写点什么

保姆级教程:教你 UniMRCP 对接华为云 ASR(Linux 版)

  • 2024-03-26
    广东
  • 本文字数:25140 字

    阅读完需:约 82 分钟

保姆级教程:教你UniMRCP对接华为云ASR(Linux版)

本文分享自华为云社区《unimrcp对接华为云ASR(Linux版)》,作者: ASR-beginer。


本篇文章提供了 unimrcp 对接华为云 ASR 的保姆级教程,根据第一到四章,可从头逐步编译+集成基于华为云 ASR 的 unimrcp 系统(授人以渔)。同时,本文第五章(直接给条鱼)提供了作者修改好的源码,直接一键编译即可。

一、安装 unimrcp


1.提前安装必要的依赖项(系统级)



2.下载 unimrcp



3.下载 unimrcp 的依赖库,并安装


进入https://www.unimrcp.org/downloads/dependencies,直接选择最新版本下载



解压后,进入 unimrcp-deps-1.6.0,执行安装命令:


./build-dep-libs.sh
复制代码


4.安装 unimrcp



之后可以在/usr/local/unimrcp 路径下,看到以下内容:



5.ASR 测试


先启动服务端./unimrcpserver


(base) [root@asr-build bin]# ./unimrcpserver 2024-03-09 10:23:39:729352 [NOTICE] UniMRCP Server [1.8.0]2024-03-09 10:23:39:729427 [INFO]   APR [1.5.2]2024-03-09 10:23:39:729436 [NOTICE] Create MRCP Server2024-03-09 10:23:39:729459 [NOTICE] Open Config File [/usr/local/unimrcp/conf/unimrcpserver.xml]2024-03-09 10:23:39:729874 [INFO]   Set Property ip:127.0.0.12024-03-09 10:23:39:729888 [INFO]   Register Codec [PCMU]2024-03-09 10:23:39:729890 [INFO]   Register Codec [PCMA]2024-03-09 10:23:39:729893 [INFO]   Register Codec [L16]2024-03-09 10:23:39:729895 [INFO]   Register Codec [G722]2024-03-09 10:23:39:729911 [NOTICE] Register Resource [speechsynth]2024-03-09 10:23:39:729916 [NOTICE] Register Resource [speechrecog]2024-03-09 10:23:39:729918 [NOTICE] Register Resource [recorder]2024-03-09 10:23:39:729922 [NOTICE] Register Resource [speakverify]2024-03-09 10:23:39:729925 [INFO]   Register Resource Factory2024-03-09 10:23:39:729938 [NOTICE] Create SofiaSIP Agent [SIP-Agent-1] [1.12.11-239-g54ef3e2] sip:127.0.0.1:8060;transport=udp,tcp2024-03-09 10:23:39:729943 [INFO]   Register Signaling Agent [SIP-Agent-1]2024-03-09 10:23:39:729954 [NOTICE] Create RTSP Server [RTSP-Agent-1] 127.0.0.1:1554 [100] connection timeout [600 sec]2024-03-09 10:23:39:730021 [INFO]   Register Signaling Agent [RTSP-Agent-1]2024-03-09 10:23:39:730027 [NOTICE] Create MRCPv2 Agent [MRCPv2-Agent-1] 127.0.0.1:1544 [100]2024-03-09 10:23:39:730047 [INFO]   Register Connection Agent [MRCPv2-Agent-1]2024-03-09 10:23:39:730051 [NOTICE] Create Media Engine [Media-Engine-1]2024-03-09 10:23:39:730058 [INFO]   Register Media Engine [Media-Engine-1]2024-03-09 10:23:39:730062 [NOTICE] Create RTP Termination Factory 127.0.0.1:[5000,6000]2024-03-09 10:23:39:730064 [INFO]   Register RTP Termination Factory [RTP-Factory-1]2024-03-09 10:23:39:730069 [INFO]   Load Plugin [Demo-Synth-1] [/usr/local/unimrcp/plugin/demosynth.so]2024-03-09 10:23:39:730166 [INFO]   Register MRCP Engine [Demo-Synth-1]2024-03-09 10:23:39:730171 [INFO]   Load Plugin [Huawei-Recog-1] [/usr/local/unimrcp/plugin/huaweirecog.so]2024-03-09 10:23:39:730223 [INFO]   Register MRCP Engine [Huawei-Recog-1]2024-03-09 10:23:39:730226 [INFO]   Load Plugin [Demo-Verifier-1] [/usr/local/unimrcp/plugin/demoverifier.so]2024-03-09 10:23:39:730276 [INFO]   Register MRCP Engine [Demo-Verifier-1]2024-03-09 10:23:39:730279 [INFO]   Load Plugin [Recorder-1] [/usr/local/unimrcp/plugin/mrcprecorder.so]2024-03-09 10:23:39:730323 [INFO]   Register MRCP Engine [Recorder-1]2024-03-09 10:23:39:730333 [INFO]   Register RTP Settings [RTP-Settings-1]2024-03-09 10:23:39:730338 [NOTICE] Create MRCPv2 Profile [uni2]2024-03-09 10:23:39:730344 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni2]2024-03-09 10:23:39:730347 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni2]2024-03-09 10:23:39:730349 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni2]2024-03-09 10:23:39:730351 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni2]2024-03-09 10:23:39:730354 [INFO]   Register Profile [uni2]2024-03-09 10:23:39:730356 [NOTICE] Create MRCPv1 Profile [uni1]2024-03-09 10:23:39:730358 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni1]2024-03-09 10:23:39:730360 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni1]2024-03-09 10:23:39:730362 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni1]2024-03-09 10:23:39:730364 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni1]2024-03-09 10:23:39:730367 [INFO]   Register Profile [uni1]2024-03-09 10:23:39:730370 [INFO]   Start Task [MRCP Server]>2024-03-09 10:23:39:730456 [INFO]   Open Engine [Demo-Verifier-1]2024-03-09 10:23:39:730471 [INFO]   Start Task [Demo Verifier Engine]2024-03-09 10:23:39:730526 [INFO]   Open Engine [Huawei-Recog-1]2024-03-09 10:23:39:730530 [INFO]   Start Task [Huawei Recog Engine]2024-03-09 10:23:39:730552 [INFO]   Open Engine [Recorder-1]2024-03-09 10:23:39:730555 [INFO]   Open Engine [Demo-Synth-1]2024-03-09 10:23:39:730557 [INFO]   Start Task [Demo Synth Engine]2024-03-09 10:23:39:730580 [INFO]   Start Task [SIP-Agent-1]2024-03-09 10:23:39:730599 [INFO]   Start Task [RTSP-Agent-1]2024-03-09 10:23:39:730619 [INFO]   Start Task [MRCPv2-Agent-1]2024-03-09 10:23:39:730642 [INFO]   Start Task [Media-Engine-1]sres: /etc/resolv.conf: unknown option                      2024-03-09 10:23:39:731672 [NOTICE] MRCP Server Started
复制代码


再启动客户端,测试 ASR



>run recog>[1]2024-03-09 10:24:46:654383 [NOTICE] Create MRCP Handle 0x7f902c035630 [uni2]2024-03-09 10:24:46:654420 [INFO]   Create Channel umc-1 <new>2024-03-09 10:24:46:654466 [INFO]   Receive App Request umc-1 <new> [2]2024-03-09 10:24:46:654535 [INFO]   Add MRCP Handle umc-1 <new>2024-03-09 10:24:46:654573 [NOTICE] Add Control Channel umc-1 <new@speechrecog>2024-03-09 10:24:46:656847 [INFO]   Send Offer umc-1 <new> [c:1 a:1 v:0] to 127.0.0.1:80602024-03-09 10:24:46:656879 [INFO]   Local SDP umc-1 <new>v=0o=UniMRCPClient 0 0 IN IP4 127.0.0.1s=-c=IN IP4 127.0.0.1t=0 0m=application 9 TCP/MRCPv2 1a=setup:activea=connection:newa=resource:speechrecoga=cmid:1m=audio 4000 RTP/AVP 0 8 9 96 101a=rtpmap:0 PCMU/8000a=rtpmap:8 PCMA/8000a=rtpmap:9 G722/8000a=rtpmap:96 L16/8000a=rtpmap:101 telephone-event/8000a=fmtp:101 0-15a=sendonlya=ptime:20a=mid:1
2024-03-09 10:24:46:657606 [INFO] Receive SIP Event [nua_i_state] Status 0 INVITE sent [SIP-Agent-1]2024-03-09 10:24:46:657624 [NOTICE] SIP Call State umc-1 [calling]2024-03-09 10:24:46:661600 [INFO] Receive SIP Event [nua_r_invite] Status 200 OK [SIP-Agent-1]2024-03-09 10:24:46:661658 [INFO] Receive SIP Event [nua_i_state] Status 200 OK [SIP-Agent-1]2024-03-09 10:24:46:661664 [NOTICE] SIP Call State umc-1 [ready]2024-03-09 10:24:46:661668 [INFO] Remote SDP umc-1 <new>v=0o=UniMRCPServer 8207428304472510678 6899681545498287106 IN IP4 192.168.34.32s=-c=IN IP4 127.0.0.1t=0 0m=application 1544 TCP/MRCPv2 1a=setup:passivea=connection:newa=channel:328a56e4ddbc11ee@speechrecoga=cmid:1m=audio 5000 RTP/AVP 0 101a=rtpmap:0 PCMU/8000a=rtpmap:101 telephone-event/8000a=fmtp:101 0-15a=recvonlya=ptime:20a=mid:1
2024-03-09 10:24:46:661738 [INFO] Receive SIP Event [nua_i_active] Status 200 Call active [SIP-Agent-1]2024-03-09 10:24:46:661756 [INFO] Receive Answer umc-1 <new> [c:1 a:1 v:0] Status 2002024-03-09 10:24:46:661892 [NOTICE] Established TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:15442024-03-09 10:24:46:661906 [INFO] Add Control Channel <328a56e4ddbc11ee@speechrecog> 127.0.0.1:59830 <-> 127.0.0.1:1544 [1]2024-03-09 10:24:46:666844 [INFO] Enable RTP Session 127.0.0.1:40002024-03-09 10:24:46:666873 [INFO] Open RTP Transmitter 127.0.0.1:4000 -> 127.0.0.1:50002024-03-09 10:24:46:666878 [INFO] Media Path umc-1 Source->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMU/8000/1]->Sink2024-03-09 10:24:46:666900 [INFO] Raise App Response umc-1 <328a56e4ddbc11ee> [2] SUCCESS [0]2024-03-09 10:24:46:666975 [INFO] Receive App MRCP Request umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:46:666979 [INFO] Send MRCP Request umc-1 <328a56e4ddbc11ee@speechrecog> [1]2024-03-09 10:24:46:667028 [INFO] Send MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [442 bytes]MRCP/2.0 442 DEFINE-GRAMMAR 1Channel-Identifier: 328a56e4ddbc11ee@speechrecogContent-Type: application/srgs+xmlContent-Id: request1@form-levelContent-Length: 269
<?xml version="1.0"?><grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en-US" version="1.0" mode="voice" root="digit"> <rule id="digit"> <one-of> <item>one</item> <item>two</item> <item>three</item> </one-of> </rule></grammar>2024-03-09 10:24:46:667298 [INFO] Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [112 bytes]MRCP/2.0 112 1 200 COMPLETEChannel-Identifier: 328a56e4ddbc11ee@speechrecogCompletion-Cause: 000 success2024-03-09 10:24:46:667336 [INFO] Raise App MRCP Response umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:46:667403 [INFO] Receive App MRCP Request umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:46:667406 [INFO] Send MRCP Request umc-1 <328a56e4ddbc11ee@speechrecog> [2]2024-03-09 10:24:46:667414 [INFO] Set [/usr/local/unimrcp/data/one-8kHz.pcm] as Speech Source2024-03-09 10:24:46:667428 [INFO] Send MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [304 bytes]MRCP/2.0 304 RECOGNIZE 2Channel-Identifier: 328a56e4ddbc11ee@speechrecogContent-Type: text/uri-listCancel-If-Queue: falseNo-Input-Timeout: 5000Recognition-Timeout: 10000Start-Input-Timers: trueConfidence-Threshold: 0.5Save-Waveform: trueContent-Length: 27
session:request1@form-level2024-03-09 10:24:46:667664 [INFO] Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [83 bytes]MRCP/2.0 83 2 200 IN-PROGRESSChannel-Identifier: 328a56e4ddbc11ee@speechrecog2024-03-09 10:24:46:667686 [INFO] Raise App MRCP Response umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:47:541082 [INFO] Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [94 bytes]MRCP/2.0 94 START-OF-INPUT 2 IN-PROGRESSChannel-Identifier: 328a56e4ddbc11ee@speechrecog2024-03-09 10:24:47:541165 [INFO] Raise App MRCP Event umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:48:011172 [INFO] Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [393 bytes]MRCP/2.0 393 RECOGNITION-COMPLETE 2 COMPLETEChannel-Identifier: 328a56e4ddbc11ee@speechrecogCompletion-Cause: 000 successContent-Type: application/x-nlsmlContent-Length: 208
<?xml version="1.0"?><result> <interpretation grammar="session:request1@form-level.store" confidence="0.97"> <instance>one</instance> <input mode="speech">one</input> </interpretation></result>2024-03-09 10:24:48:011247 [INFO] Raise App MRCP Event umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:48:011354 [INFO] Interpretation[0].confidence: 0.972024-03-09 10:24:48:011363 [INFO] Interpretation[0].grammar: session:request1@form-level.store2024-03-09 10:24:48:011380 [INFO] Interpretation[0].instance[0]: one2024-03-09 10:24:48:011386 [INFO] Interpretation[0].input: one2024-03-09 10:24:48:011388 [INFO] Interpretation[0].input.mode: speech2024-03-09 10:24:48:011391 [INFO] Interpretation[0].input.confidence: 1.002024-03-09 10:24:48:011412 [INFO] Receive App Request umc-1 <328a56e4ddbc11ee> [1]2024-03-09 10:24:48:011424 [INFO] Terminate Session umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:48:011454 [INFO] Remove Control Channel <328a56e4ddbc11ee@speechrecog> [0]2024-03-09 10:24:48:011459 [INFO] Close TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:15442024-03-09 10:24:48:011791 [INFO] Receive SIP Event [nua_r_bye] Status 200 OK [SIP-Agent-1]2024-03-09 10:24:48:011805 [INFO] Receive SIP Event [nua_i_state] Status 200 to BYE [SIP-Agent-1]2024-03-09 10:24:48:011810 [NOTICE] SIP Call State umc-1 [terminated]2024-03-09 10:24:48:011871 [INFO] Session Terminated umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:48:016774 [INFO] Close RTP Transmitter 127.0.0.1:4000 -> 127.0.0.1:5000 [s:67 o:10720]2024-03-09 10:24:48:016791 [INFO] Remove RTP Session 127.0.0.1:40002024-03-09 10:24:48:016831 [NOTICE] Destroy TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:15442024-03-09 10:24:48:016851 [INFO] Remove MRCP Handle umc-1 <328a56e4ddbc11ee>2024-03-09 10:24:48:016855 [INFO] Raise App Response umc-1 <328a56e4ddbc11ee> [1] SUCCESS [0]2024-03-09 10:24:48:016953 [NOTICE] Destroy MRCP Handle umc-1
复制代码


第 124~130 行即为模拟的识别结果


二、安装华为 ASR SDK


华为 ASR C++(Linux 版)SDK 安装过程参考之前的文章:https://bbs.huaweicloud.com/blogs/392949


这里推荐一键安装,因为本 SDK 将所有依赖均打包在仓库中,可以在任意 Linux 机器上编译通过(g++ 4.8.5 及以上,cmake 版本至少是 3.14)

三、unimrcp 集成华为 ASR SDK


本章节用从底到高的顺序进行记录,即先修改底层代码/配置,再修改顶层代码


1.将(上个步骤 安装华为 ASR SDK)编译好的 SDK 打包起来,并命名为 huaweicloud_asr,作为备用,其代码结构如下:



2.进入 unimrcp/plugins,调整代码结构,如下图所示


cd umimrcp/pluginsmv demo-recog huawei-recog    # 将原始的demo-recog重命名成huawei-recogmv huawei-recog/src/demo_recog_engine.c huawei-recog/src/huawei_recog_engine.cccp -r path_to_sdk/huaweicloud_asr huawei-recog/huaweicloud_asr   # 将华为asr sdk拷贝到umimrcp/plugins/huawei-recog/huaweicloud_asr处
复制代码



3.修改 huawei-recog/src/huawei_recog_engine.cc 文件,具体内容如下

















/** Process START-INPUT-TIMERS request */static apt_bool_t huawei_recog_channel_timers_start(    mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response){    huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)channel->method_obj;    recog_channel->timers_started = TRUE;    return mrcp_engine_channel_message_send(channel, response);}
/** Dispatch MRCP request */static apt_bool_t huawei_recog_channel_request_dispatch(mrcp_engine_channel_t *channel, mrcp_message_t *request){ apt_bool_t processed = FALSE; mrcp_message_t *response = mrcp_response_create(request, request->pool); switch (request->start_line.method_id) { case RECOGNIZER_SET_PARAMS: break; case RECOGNIZER_GET_PARAMS: break; case RECOGNIZER_DEFINE_GRAMMAR: break; case RECOGNIZER_RECOGNIZE: processed = huawei_recog_channel_recognize(channel, request, response); break; case RECOGNIZER_GET_RESULT: break; case RECOGNIZER_START_INPUT_TIMERS: processed = huawei_recog_channel_timers_start(channel, request, response); break; case RECOGNIZER_STOP: processed = huawei_recog_channel_stop(channel, request, response); break; default: break; } if (processed == FALSE) { /* send asynchronous response for not handled request */ mrcp_engine_channel_message_send(channel, response); } return TRUE;}
/** Callback is called from MPF engine context to destroy any additional data associated with audio stream */static apt_bool_t huawei_recog_stream_destroy(mpf_audio_stream_t *stream){ return TRUE;}
/** Callback is called from MPF engine context to perform any action before open */static apt_bool_t huawei_recog_stream_open(mpf_audio_stream_t *stream, mpf_codec_t *codec){ return TRUE;}
/** Callback is called from MPF engine context to perform any action after close */static apt_bool_t huawei_recog_stream_close(mpf_audio_stream_t *stream){ return TRUE;}
/* Raise huawei START-OF-INPUT event */static apt_bool_t huawei_recog_start_of_input(huawei_recog_channel_t *recog_channel){ /* create START-OF-INPUT event */ mrcp_message_t *message = mrcp_event_create(recog_channel->recog_request, RECOGNIZER_START_OF_INPUT, recog_channel->recog_request->pool); if (!message) { return FALSE; }
/* set request state */ message->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS; /* send asynch event */ return mrcp_engine_channel_message_send(recog_channel->channel, message);}
/* Load demo recognition result */static apt_bool_t huawei_recog_result_load(huawei_recog_channel_t *recog_channel, mrcp_message_t *message){ mrcp_engine_channel_t *channel = recog_channel->channel; const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
/* read the demo result from file 这里是从asr真实结果中读取*/ mrcp_generic_header_t *generic_header;
apt_string_assign_n(&message->body, recog_channel->call_back->Result().c_str(), recog_channel->call_back->Result().size(), message->pool);
/* get/allocate generic header */ generic_header = mrcp_generic_header_prepare(message); if (generic_header) { /* set content types */ apt_string_assign(&generic_header->content_type, "application/x-nlsml", message->pool); mrcp_generic_header_property_add(message, GENERIC_HEADER_CONTENT_TYPE); } return TRUE;}
/* Raise huawei RECOGNITION-COMPLETE event */static apt_bool_t huawei_recog_recognition_complete( huawei_recog_channel_t *recog_channel, mrcp_recog_completion_cause_e cause){ mrcp_recog_header_t *recog_header; /* create RECOGNITION-COMPLETE event */ mrcp_message_t *message = mrcp_event_create( recog_channel->recog_request, RECOGNIZER_RECOGNITION_COMPLETE, recog_channel->recog_request->pool); if (!message) { return FALSE; }
/* get/allocate recognizer header */ recog_header = (mrcp_recog_header_t *)mrcp_resource_header_prepare(message); if (recog_header) { /* set completion cause */ recog_header->completion_cause = cause; mrcp_resource_header_property_add(message, RECOGNIZER_HEADER_COMPLETION_CAUSE); } /* set request state */ message->start_line.request_state = MRCP_REQUEST_STATE_COMPLETE;
if (cause == RECOGNIZER_COMPLETION_CAUSE_SUCCESS) { huawei_recog_result_load(recog_channel, message); }
recog_channel->recog_request = NULL; /* send asynch event */ return mrcp_engine_channel_message_send(recog_channel->channel, message);}
/** Callback is called from MPF engine context to write/send new frame */static apt_bool_t huawei_recog_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame){ huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)stream->obj; if (recog_channel->stop_response) { /* send asynchronous response to STOP request */ mrcp_engine_channel_message_send(recog_channel->channel, recog_channel->stop_response); recog_channel->stop_response = NULL; recog_channel->recog_request = NULL; return TRUE; } bool finish = false;
// 接收的数据帧转给SDK if (recog_channel->recog_request) { // 第622行至660行 是unimrcp提供的断句规则,这里禁用中间两个case(因为效果差),直接用华为云的断句规则 // mpf_detector_event_e det_event = mpf_activity_detector_process(recog_channel->detector, frame); // switch (det_event) { // case MPF_DETECTOR_EVENT_ACTIVITY: // apt_log(RECOG_LOG_MARK, // APT_PRIO_INFO, // "Detected Voice Activity " APT_SIDRES_FMT, // MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); // huawei_recog_start_of_input(recog_channel); // break; // case MPF_DETECTOR_EVENT_INACTIVITY: // apt_log(RECOG_LOG_MARK, // APT_PRIO_INFO, // "Detected Voice Inactivity " APT_SIDRES_FMT, // MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); // if (recog_channel->rasr_client) { // recog_channel->rasr_client->SendEnd(); // recog_channel->rasr_client->Close(); // } // finish = true; // huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS); // apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog"); // break; // case MPF_DETECTOR_EVENT_NOINPUT: // apt_log(RECOG_LOG_MARK, // APT_PRIO_INFO, // "Detected Noinput " APT_SIDRES_FMT, // MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); // if (recog_channel->timers_started == TRUE) { // if (recog_channel->rasr_client) { // recog_channel->rasr_client->SendEnd(); // recog_channel->rasr_client->Close(); // } // huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT); // } // finish = true; // break; // default: // break; // } Status cur_status = recog_channel->call_back->GetStatus(); switch (cur_status) { case Status::voice_start: apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Detected Voice Activity " APT_SIDRES_FMT, MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); huawei_recog_start_of_input(recog_channel); recog_channel->call_back->SetStatus(Status::recognizing); break; case Status::voice_end: apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Detected Voice Inactivity " APT_SIDRES_FMT, MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); if (recog_channel->rasr_client) { recog_channel->rasr_client->SendEnd(); recog_channel->rasr_client->Close(); } finish = true; huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS); apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog"); break; case Status::exceeded_silence: apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Detected Noinput " APT_SIDRES_FMT, MRCP_MESSAGE_SIDRES(recog_channel->recog_request)); if (recog_channel->timers_started == TRUE) { if (recog_channel->rasr_client) { recog_channel->rasr_client->SendEnd(); recog_channel->rasr_client->Close(); } huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT); } finish = true; break; default: break; }
if (recog_channel->recog_request) { if ((frame->type & MEDIA_FRAME_TYPE_EVENT) == MEDIA_FRAME_TYPE_EVENT) { if (frame->marker == MPF_MARKER_START_OF_EVENT) { apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Detected Start of Event " APT_SIDRES_FMT " id:%d", MRCP_MESSAGE_SIDRES(recog_channel->recog_request), frame->event_frame.event_id); } else if (frame->marker == MPF_MARKER_END_OF_EVENT) { apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Detected End of Event " APT_SIDRES_FMT " id:%d duration:%d ts", MRCP_MESSAGE_SIDRES(recog_channel->recog_request), frame->event_frame.event_id, frame->event_frame.duration); } } }
if (recog_channel->audio_out) { fwrite(frame->codec_frame.buffer, 1, frame->codec_frame.size, recog_channel->audio_out); }
if (recog_channel->rasr_client && !finish) { recog_channel->rasr_client->SendBinary((unsigned char*)frame->codec_frame.buffer, frame->codec_frame.size); apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Sending [%d] bytes", frame->codec_frame.size); } } return TRUE;}
static apt_bool_t huawei_recog_msg_signal( huawei_recog_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request){ apt_bool_t status = FALSE; huawei_recog_channel_t *huawei_channel = (huawei_recog_channel_t *)channel->method_obj; huawei_recog_engine_t *huawei_engine = huawei_channel->huawei_engine; apt_task_t *task = apt_consumer_task_base_get(huawei_engine->task); apt_task_msg_t *msg = apt_task_msg_get(task); if (msg) { huawei_recog_msg_t *huawei_msg; msg->type = TASK_MSG_USER; huawei_msg = (huawei_recog_msg_t *)msg->data;
huawei_msg->type = type; huawei_msg->channel = channel; huawei_msg->request = request; status = apt_task_msg_signal(task, msg); } return status;}
static apt_bool_t huawei_recog_msg_process(apt_task_t *task, apt_task_msg_t *msg){ huawei_recog_msg_t *huawei_msg = (huawei_recog_msg_t *)msg->data; switch (huawei_msg->type) { case HUAWEI_RECOG_MSG_OPEN_CHANNEL: /* open channel and send asynch response */ mrcp_engine_channel_open_respond(huawei_msg->channel, TRUE); apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_OPEN_CHANNEL"); break; case HUAWEI_RECOG_MSG_CLOSE_CHANNEL: { /* close channel, make sure there is no activity and send asynch response */ huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)huawei_msg->channel->method_obj; if (recog_channel->audio_out) { fclose(recog_channel->audio_out); recog_channel->audio_out = NULL; } apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_CLOSE_CHANNEL"); mrcp_engine_channel_close_respond(huawei_msg->channel); break; } case HUAWEI_RECOG_MSG_REQUEST_PROCESS: apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_REQUEST_PROCESS"); huawei_recog_channel_request_dispatch(huawei_msg->channel, huawei_msg->request); break; default: break; } return TRUE;}
复制代码


4.修改 asr 插件的 makefile.am 文件(进入路径:unimrcp/plugins): vim huawei-recog/Makefile.am


AM_CPPFLAGS                = $(UNIMRCP_PLUGIN_INCLUDES)
plugin_LTLIBRARIES = huaweirecog.la
huaweirecog_la_SOURCES = src/huawei_recog_engine.cchuaweirecog_la_LDFLAGS = $(UNIMRCP_PLUGIN_OPTS) \ -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party \ -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/lib64 \ -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/lib \ -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/glog-install/lib64 \ ${CMAKE_SOURCE_DIR}/third_party/openssl-install/lib \ -lhuawei_rasr -ssl -lcrypto -ljsoncpp -lgflags_nothreads -stdc++=14huaweirecog_ladir = $(libdir)huaweirecog_la_DATA = $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/libhuawei_rasr.so \ $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/lib/libcrypto.so \ $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/lib/libssl.so \ $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/lib64/libjsoncpp.so \ $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/lib/libgflags_nothreads.soinclude $(top_srcdir)/build/rules/uniplugin.am
UNIMRCP_PLUGIN_INCLUDES += -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/include \ -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/websocketpp-src \ -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/include \ -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/boost-src/ \ -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/include \ -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/include
复制代码


5.修改插件的 Makefile.am(进入路径:unimrcp/plugins) vim Makefile.am



6.修改最顶层的 configure.ac(进入 unimrcp 根目录):


dnldnl Autoconf configuration file for UniMRCP.dnldnl Use ./bootstrap to produce a configure script.dnl
AC_PREREQ(2.59)
AC_INIT([unimrcp],[1.8.0])
AC_CONFIG_AUX_DIR([build])AC_CONFIG_MACRO_DIR([build/acmacros])
dnl Set ac_macro_dir variable manually for autoconf 2.61 and above.ac_macro_dir="build/acmacros"
AC_SUBST(ac_aux_dir)AC_SUBST(ac_macro_dir)
dnl Include m4 macros for libtool 2.sinclude(build/acmacros/libtool.m4)sinclude(build/acmacros/ltoptions.m4)sinclude(build/acmacros/ltsugar.m4)sinclude(build/acmacros/ltversion.m4)sinclude(build/acmacros/lt~obsolete.m4)
AC_PREFIX_DEFAULT(/usr/local/unimrcp)
dnl Define the directory layout.APR_ENABLE_LAYOUT(classic, [plugindir logdir vardir])AC_SUBST(plugindir)AC_SUBST(logdir)AC_SUBST(vardir)
dnl Reparse the configure arguments.APR_PARSE_ARGUMENTS
dnl Generate ./config.nice to reuse ./configure command-line.APR_CONFIG_NICE(config.nice)
AM_INIT_AUTOMAKE([no-define nostdinc foreign subdir-objects])
dnl Enable silent build rules available since automake 1.11.m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
dnl Set default language.AC_LANG_C
AC_PROG_CCAC_PROG_CXXAM_PROG_CC_C_OAC_PROG_INSTALL
dnl Skip detection of Fortran.m4_undefine([AC_PROG_F77])m4_defun([AC_PROG_F77],[])AC_PROG_LIBTOOL
dnl Suppress warning: ar: 'u' modifier ignored since 'D' is the defaultAC_SUBST(AR_FLAGS, [cr])
dnl Do not use autoconf generated compiler DEFS.rm confdefs.htouch confdefs.h
dnl Check for C compiler vendor.AX_COMPILER_VENDOR
dnl Search for pkg-config.AC_PATH_PROG(PKG_CONFIG, pkg-config, no)if test "x$PKG_CONFIG" = "xno"; then AC_MSG_ERROR([pkg-config is missing])fi
dnl Get version information.get_version="$srcdir/build/get-version.sh"version_hdr="$srcdir/build/uni_version.h"plugin_version_hdr="$srcdir/libs/mrcp-engine/include/mrcp_engine_plugin.h"UNI_DOTTED_VERSION="`$get_version all $version_hdr UNI`"UNI_LT_VERSION="-version-info `$get_version libtool $version_hdr UNI`"PLUGIN_LT_VERSION="-version-info `$get_version libtool $plugin_version_hdr PLUGIN`"
AC_SUBST(UNI_DOTTED_VERSION)AC_SUBST(UNI_LT_VERSION)AC_SUBST(PLUGIN_LT_VERSION)
echo "UniMRCP Version: ${UNI_DOTTED_VERSION}"
dnl Check for the APR and APR-util libraries.UNIMRCP_CHECK_APRdnl Check for the Sofia-SIP library.UNIMRCP_CHECK_SOFIA
dnl Enable AMR codec.AC_ARG_ENABLE(amr-codec, [AC_HELP_STRING([--enable-amr-codec ],[enable AMR codec])], [enable_amr_codec="$enableval"], [enable_amr_codec="no"])
AC_MSG_NOTICE([enable AMR codec: $enable_amr_codec])if test "${enable_amr_codec}" != "no"; then dnl Check for the OpenCORE AMR library. UNIMRCP_CHECK_OPENCORE_AMR
dnl Check for the VO AMRWBENC library. UNIMRCP_CHECK_VO_AMRWBENCfi
AM_CONDITIONAL([UNIMRCP_AMR_CODEC],[test "${enable_amr_codec}" != "no"])
dnl Enable inter-library dependencies.AC_ARG_ENABLE(interlib-deps, [AC_HELP_STRING([--disable-interlib-deps ],[disable inter-library dependencies (might break builds)])], [enable_interlib_deps="$enableval"], [enable_interlib_deps="yes"])
AC_MSG_NOTICE([enable inter-library dependencies: $enable_interlib_deps])if test "${enable_interlib_deps}" == "yes"; then link_all_deplibs=yes link_all_deplibs_CXX=yeselse link_all_deplibs=no link_all_deplibs_CXX=nofi
dnl Enable maintainer mode.AC_ARG_ENABLE(maintainer-mode, [AC_HELP_STRING([--enable-maintainer-mode ],[turn on debugging and compile time warnings])], [enable_maintainer_mode="$enableval"], [enable_maintainer_mode="no"])
AC_MSG_NOTICE([enable maintainer mode: $enable_maintainer_mode])if test "${enable_maintainer_mode}" != "no"; then APR_ADDTO(CFLAGS,-g) if test "x${ax_cv_c_compiler_vendor}" = "xgnu" ; then APR_ADDTO(CFLAGS,-Wall -Werror) fifi
dnl UniMRCP client library.AC_ARG_ENABLE(client-lib, [AC_HELP_STRING([--disable-client-lib ],[exclude unimrcpclient lib from build])], [enable_client_lib="$enableval"], [enable_client_lib="yes"])
AM_CONDITIONAL([UNIMRCP_CLIENT_LIB],[test "${enable_client_lib}" = "yes"])
dnl Sample UniMRCP client application in C.AC_ARG_ENABLE(client-app, [AC_HELP_STRING([--disable-client-app ],[exclude sample unimrcpclient app from build])], [enable_client_app="$enableval"], [enable_client_app="yes"])
AM_CONDITIONAL([UNIMRCP_CLIENT_APP],[test "${enable_client_lib}" = "yes" && test "${enable_client_app}" = "yes"])
dnl Sample UniMRCP client application in C++.AC_ARG_ENABLE(umc, [AC_HELP_STRING([--disable-umc ],[exclude sample unimrcpclient C++ app from build])], [enable_umc="$enableval"], [enable_umc="yes"])
AM_CONDITIONAL([UMC],[test "${enable_client_lib}" = "yes" && test "${enable_umc}" = "yes"])
dnl Miscellaneous ASR client library and application.AC_ARG_ENABLE(asr-client, [AC_HELP_STRING([--disable-asr-client ],[exclude misc ASR client lib and app from build])], [enable_asr_client="$enableval"], [enable_asr_client="yes"])
AM_CONDITIONAL([ASR_CLIENT],[test "${enable_client_lib}" = "yes" && test "${enable_asr_client}" = "yes"])
AM_CONDITIONAL([COMMON_CLIENT_DATA],[test "${enable_client_app}" = "yes" || test "${enable_umc}" = "yes" ||test "${enable_asr_client}" = "yes"])
dnl UniMRCP server library.AC_ARG_ENABLE(server-lib, [AC_HELP_STRING([--disable-server-lib ],[exclude unimrcpserver lib from build])], [enable_server_lib="$enableval"], [enable_server_lib="yes"])
AM_CONDITIONAL([UNIMRCP_SERVER_LIB],[test "${enable_server_lib}" = "yes"])
dnl UniMRCP server application.AC_ARG_ENABLE(server-app, [AC_HELP_STRING([--disable-server-app ],[exclude unimrcpserver app from build])], [enable_server_app="$enableval"], [enable_server_app="yes"])
AM_CONDITIONAL([UNIMRCP_SERVER_APP],[test "${enable_server_lib}" = "yes" && test "${enable_server_app}" = "yes"])
dnl Demo synthesizer plugin.UNI_PLUGIN_ENABLED(demosynth)
AM_CONDITIONAL([DEMOSYNTH_PLUGIN],[test "${enable_demosynth_plugin}" = "yes"])
dnl Huawei recognizer plugin.UNI_PLUGIN_ENABLED(huaweirecog)
AM_CONDITIONAL([HUAWEIRECOG_PLUGIN],[test "${enable_huaweirecog_plugin}" = "yes"])
dnl Demo verifier plugin.UNI_PLUGIN_ENABLED(demoverifier)
AM_CONDITIONAL([DEMOVERIFIER_PLUGIN],[test "${enable_demoverifier_plugin}" = "yes"])
dnl Recorder plugin.UNI_PLUGIN_ENABLED(recorder)
AM_CONDITIONAL([RECORDER_PLUGIN],[test "${enable_recorder_plugin}" = "yes"])
dnl Enable test suites.AC_ARG_ENABLE(test-suites, [AC_HELP_STRING([--enable-test-suites ],[build test suites])], [enable_test_suites="$enableval"], [enable_test_suites="no"])
AM_CONDITIONAL([TEST_SUITES],[test "${enable_test_suites}" != "no"])
AM_CONDITIONAL(ISMAC, [test `uname -s` = Darwin])
AC_CONFIG_FILES([ Makefile libs/Makefile libs/apr-toolkit/Makefile libs/mpf/Makefile libs/mrcp/Makefile libs/mrcp-signaling/Makefile libs/mrcpv2-transport/Makefile libs/mrcp-engine/Makefile libs/mrcp-server/Makefile libs/mrcp-client/Makefile libs/uni-rtsp/Makefile modules/Makefile modules/mrcp-sofiasip/Makefile modules/mrcp-unirtsp/Makefile plugins/Makefile plugins/mrcp-recorder/Makefile plugins/demo-synth/Makefile plugins/huawei-recog/Makefile plugins/demo-verifier/Makefile platforms/Makefile platforms/libunimrcp-server/Makefile platforms/libunimrcp-client/Makefile platforms/unimrcp-server/Makefile platforms/unimrcp-client/Makefile platforms/libasr-client/Makefile platforms/asr-client/Makefile platforms/umc/Makefile tests/Makefile tests/apttest/Makefile tests/mpftest/Makefile tests/mrcptest/Makefile tests/rtsptest/Makefile tests/strtablegen/Makefile build/Makefile build/pkgconfig/Makefile build/pkgconfig/unimrcpclient.pc build/pkgconfig/unimrcpserver.pc build/pkgconfig/unimrcpplugin.pc conf/Makefile data/Makefile docs/doxygen.conf])
AC_OUTPUT
echoecho '****************************** REPORT ******************************'echoecho UniMRCP version............... : $UNI_DOTTED_VERSIONechoecho APR version................... : $apr_versionecho APR-util version.............. : $apu_versionecho Sofia-SIP version............. : $sofia_versionif test "${enable_amr_codec}" != "no"; thenechoecho OpenCORE AMR version.......... : $opencore_amr_versionecho VO AMRWBENC version........... : $vo_amrwbenc_versionfiechoecho Compiler...................... : $CCecho Compiler flags................ : $CFLAGSecho Preprocessor definitions...... : $CPPFLAGSecho Linker flags.................. : $LDFLAGSechoecho UniMRCP client lib............ : $enable_client_libecho Sample UniMRCP client app..... : $enable_client_appecho Sample UMC C++ client app..... : $enable_umcecho Misc ASR client lib and app... : $enable_asr_clientechoecho UniMRCP server lib............ : $enable_server_libecho UniMRCP server app............ : $enable_server_appechoecho Demo synthesizer plugin....... : $enable_demosynth_pluginecho Huawei recognizer plugin........ : $enable_huaweirecog_pluginecho Demo verifier plugin.......... : $enable_demoverifier_pluginecho Recorder plugin............... : $enable_recorder_pluginechoecho Installation layout........... : $layout_nameecho Installation directory........ : $prefixechoecho '********************************************************************'
复制代码


7.添加 asr 配置文件(进入 unimrcp 根目录),这些都是必填项,按照备注获取相应的 asr 鉴权信息:vim conf/asr_info.conf



8.重新编译 unimrcp,默认会安装在/usr/local/unimrcp 中


./bootstrapmake -jmake install
复制代码


9.unimrcp 代码结构展示



/usr/local/unimrcp├── bin                 # 可执行文件 ├── conf                # 配置文件,可以设置访问IP和端口号├── data                # 默认提供的一些音频文件├── include             # 程序必须的头文件├── lib                 # 依赖的库文件├── log                 # 日志路径├── plugin              # 插件所依赖的库文件└── var                 # 保存的音频文件
复制代码


10.运行结果展示


启动服务端 /usr/local/unimrcp/bin/unimrcpserver


(base) [root@asr-build bin]# ./unimrcpserver 2024-03-11 17:17:14:704844 [NOTICE] UniMRCP Server [1.8.0]2024-03-11 17:17:14:704913 [INFO]   APR [1.5.2]2024-03-11 17:17:14:704924 [NOTICE] Create MRCP Server2024-03-11 17:17:14:704946 [NOTICE] Open Config File [/usr/local/unimrcp/conf/unimrcpserver.xml]2024-03-11 17:17:14:705077 [INFO]   Set Property ip:127.0.0.12024-03-11 17:17:14:705086 [INFO]   Register Codec [PCMU]2024-03-11 17:17:14:705088 [INFO]   Register Codec [PCMA]2024-03-11 17:17:14:705090 [INFO]   Register Codec [L16]2024-03-11 17:17:14:705092 [INFO]   Register Codec [G722]2024-03-11 17:17:14:705127 [NOTICE] Register Resource [speechsynth]2024-03-11 17:17:14:705132 [NOTICE] Register Resource [speechrecog]2024-03-11 17:17:14:705135 [NOTICE] Register Resource [recorder]2024-03-11 17:17:14:705138 [NOTICE] Register Resource [speakverify]2024-03-11 17:17:14:705141 [INFO]   Register Resource Factory2024-03-11 17:17:14:705157 [NOTICE] Create SofiaSIP Agent [SIP-Agent-1] [1.12.11-239-g54ef3e2] sip:127.0.0.1:8060;transport=udp,tcp2024-03-11 17:17:14:705163 [INFO]   Register Signaling Agent [SIP-Agent-1]2024-03-11 17:17:14:705178 [NOTICE] Create RTSP Server [RTSP-Agent-1] 127.0.0.1:1554 [100] connection timeout [600 sec]2024-03-11 17:17:14:705258 [INFO]   Register Signaling Agent [RTSP-Agent-1]2024-03-11 17:17:14:705266 [NOTICE] Create MRCPv2 Agent [MRCPv2-Agent-1] 127.0.0.1:1544 [100]2024-03-11 17:17:14:705290 [INFO]   Register Connection Agent [MRCPv2-Agent-1]2024-03-11 17:17:14:705297 [NOTICE] Create Media Engine [Media-Engine-1]2024-03-11 17:17:14:705305 [INFO]   Register Media Engine [Media-Engine-1]2024-03-11 17:17:14:705309 [NOTICE] Create RTP Termination Factory 127.0.0.1:[5000,6000]2024-03-11 17:17:14:705311 [INFO]   Register RTP Termination Factory [RTP-Factory-1]2024-03-11 17:17:14:705316 [INFO]   Load Plugin [Demo-Synth-1] [/usr/local/unimrcp/plugin/demosynth.so]2024-03-11 17:17:14:705393 [INFO]   Register MRCP Engine [Demo-Synth-1]2024-03-11 17:17:14:705397 [INFO]   Load Plugin [Huawei-Recog-1] [/usr/local/unimrcp/plugin/huaweirecog.so]2024-03-11 17:17:14:712580 [INFO]   Register MRCP Engine [Huawei-Recog-1]2024-03-11 17:17:14:712599 [INFO]   Load Plugin [Demo-Verifier-1] [/usr/local/unimrcp/plugin/demoverifier.so]2024-03-11 17:17:14:712677 [INFO]   Register MRCP Engine [Demo-Verifier-1]2024-03-11 17:17:14:712682 [INFO]   Load Plugin [Recorder-1] [/usr/local/unimrcp/plugin/mrcprecorder.so]2024-03-11 17:17:14:712739 [INFO]   Register MRCP Engine [Recorder-1]2024-03-11 17:17:14:712755 [INFO]   Register RTP Settings [RTP-Settings-1]2024-03-11 17:17:14:712764 [NOTICE] Create MRCPv2 Profile [uni2]2024-03-11 17:17:14:712773 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni2]2024-03-11 17:17:14:712776 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni2]2024-03-11 17:17:14:712779 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni2]2024-03-11 17:17:14:712782 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni2]2024-03-11 17:17:14:712786 [INFO]   Register Profile [uni2]2024-03-11 17:17:14:712790 [NOTICE] Create MRCPv1 Profile [uni1]2024-03-11 17:17:14:712793 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni1]2024-03-11 17:17:14:712796 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni1]2024-03-11 17:17:14:712799 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni1]2024-03-11 17:17:14:712802 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni1]2024-03-11 17:17:14:712805 [INFO]   Register Profile [uni1]2024-03-11 17:17:14:712810 [INFO]   Start Task [MRCP Server]>2024-03-11 17:17:14:712891 [INFO]   Open Engine [Demo-Verifier-1]2024-03-11 17:17:14:712906 [INFO]   Start Task [Demo Verifier Engine]2024-03-11 17:17:14:712963 [INFO]   Open Engine [Huawei-Recog-1]2024-03-11 17:17:14:712966 [INFO]   Start Task [Huawei Recog Engine]2024-03-11 17:17:14:712990 [INFO]   Open Engine [Recorder-1]2024-03-11 17:17:14:712993 [INFO]   Open Engine [Demo-Synth-1]2024-03-11 17:17:14:712995 [INFO]   Start Task [Demo Synth Engine]2024-03-11 17:17:14:713019 [INFO]   Start Task [SIP-Agent-1]2024-03-11 17:17:14:713037 [INFO]   Start Task [RTSP-Agent-1]2024-03-11 17:17:14:713159 [INFO]   Start Task [MRCPv2-Agent-1]2024-03-11 17:17:14:713313 [INFO]   Start Task [Media-Engine-1]sres: /etc/resolv.conf: unknown option                      2024-03-11 17:17:14:714108 [NOTICE] MRCP Server Started
复制代码


启动客户端:/usr/local/unimrcp/umc,然后在交互窗口执行 run recog,获取识别结果


四、unimrcp 定制化改动

1. 修改 xml 文件(在 conf 文件夹中),指定访问地址和端口号。端口号我们采用默认端口,IP 地址指定为本地 IP


unimrcpclient.xml 修改:



unimrcpserver.xml 修改:


2. 利用 unimrcp 调整 ASR 断句参数


unimrcp 的 header 里面提供了一组断句参数,来控制响应时间/句子长段(比如,说完一段话之后,等待多少毫秒,给出结果)。各参数名及其含义如下(在路径 data/params_default.txt 下)


No-Input-Timeout: 5000       -----> 用户接到电话后超过5000ms没有声音,就结束识别 等价于华为云ASR的vad_headRecognition-Timeout: 20000    ------> 用户一直说话,最长能说多久(默认20s),结束识别。等价于华为云ASR的max_secondsSpeech-Complete-Timeout: 400  -------> 用户开始说话后,停顿多久(默认400ms)就结束说话
复制代码
2.1 通过请求参数传递断句参数


以上参数均是通过请求体的 header 进行传递,参考文档:https://www.unimrcp.org/manuals/html/GoogleSRUsageManual.html


可以通过 unimrcp 打印的日志,查看参数设置是否生效:


2.2 强制指定断句参数


为了方便管理,建议对所有通话保留相同的断句参数,具体设置如下:


if(recog_header) {		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_START_INPUT_TIMERS) == TRUE) {			recog_channel->timers_started = recog_header->start_input_timers;		}		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_NO_INPUT_TIMEOUT) == TRUE) {			mpf_activity_detector_noinput_timeout_set(recog_channel->detector,10000); // 强制指定 如果10s中无人说话,就结束		}		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_SPEECH_COMPLETE_TIMEOUT) == TRUE) {			mpf_activity_detector_silence_timeout_set(recog_channel->detector, 600); // 强制指定600ms		}	}
复制代码

3. 利用华为云 ASR 做智能短句


华为云 ASR 提供的 sentence 模式,可适用于外呼、数字人场景,通过设置 vad_head/vad_tail/max_seconds 参数可以智能短句,相关代码变动如下


  if (recog_channel->recog_request) {        // 第622行至660行 是unimrcp提供的断句规则,这里禁用中间两个case(因为效果差),直接用华为云的断句规则        // mpf_detector_event_e det_event = mpf_activity_detector_process(recog_channel->detector, frame);        // switch (det_event) {        //     case MPF_DETECTOR_EVENT_ACTIVITY:        //         apt_log(RECOG_LOG_MARK,        //             APT_PRIO_INFO,        //             "Detected Voice Activity " APT_SIDRES_FMT,        //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));        //         huawei_recog_start_of_input(recog_channel);        //         break;        //     case MPF_DETECTOR_EVENT_INACTIVITY:        //         apt_log(RECOG_LOG_MARK,        //             APT_PRIO_INFO,        //             "Detected Voice Inactivity " APT_SIDRES_FMT,        //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));        //         if (recog_channel->rasr_client) {        //             recog_channel->rasr_client->SendEnd();        //             recog_channel->rasr_client->Close();        //         }        //         finish = true;        //         huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);        //         apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");        //         break;        //     case MPF_DETECTOR_EVENT_NOINPUT:        //         apt_log(RECOG_LOG_MARK,        //             APT_PRIO_INFO,        //             "Detected Noinput " APT_SIDRES_FMT,        //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));        //         if (recog_channel->timers_started == TRUE) {        //             if (recog_channel->rasr_client) {        //                 recog_channel->rasr_client->SendEnd();        //                 recog_channel->rasr_client->Close();        //             }        //             huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);        //         }        //         finish = true;        //         break;        //     default:        //         break;        // }        Status cur_status = recog_channel->call_back->GetStatus();        switch (cur_status) {            case Status::voice_start:                apt_log(RECOG_LOG_MARK,                    APT_PRIO_INFO,                    "Detected Voice Activity " APT_SIDRES_FMT,                    MRCP_MESSAGE_SIDRES(recog_channel->recog_request));                huawei_recog_start_of_input(recog_channel);                recog_channel->call_back->SetStatus(Status::recognizing);                break;            case Status::voice_end:                apt_log(RECOG_LOG_MARK,                    APT_PRIO_INFO,                    "Detected Voice Inactivity " APT_SIDRES_FMT,                    MRCP_MESSAGE_SIDRES(recog_channel->recog_request));                if (recog_channel->rasr_client) {                    recog_channel->rasr_client->SendEnd();                    recog_channel->rasr_client->Close();                }                finish = true;                huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);                apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");                break;            case Status::exceeded_silence:                apt_log(RECOG_LOG_MARK,                    APT_PRIO_INFO,                    "Detected Noinput " APT_SIDRES_FMT,                    MRCP_MESSAGE_SIDRES(recog_channel->recog_request));                if (recog_channel->timers_started == TRUE) {                    if (recog_channel->rasr_client) {                        recog_channel->rasr_client->SendEnd();                        recog_channel->rasr_client->Close();                    }                    huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);                }                finish = true;                break;            default:                break;        }    
复制代码

五、快速搭建 unimrcp 服务


作者按照前四章逐步执行,并在 GitHub 和 Gitee 上发布了修改后的代码,大家可以直接拿来使用。



本代码仓提供两种安装方式供用户选择

方法一: 源码安装


  1. 先下载安装华为云实时语音识别 SDK,安装指南参考https://bbs.huaweicloud.com/blogs/392949

  2. 将安装的华为云 ASR SDK 放入 plugins/huawei_recog/huaweicloud_asr 】

  3. 进入根目录/unimcrp-deps-1.6.0,安装安装 unimrcp 的依赖库


./build-dep-libs.sh
复制代码


4.进入根目录,安装 unimrcp


./bootstrap./configuremakemake install
复制代码


5.unimrcp 默认安装在/usr/local/unimrcp 中,使用前需要手动修改 asr_info.conf 中的配置项

方法二:直接使用 release 库


wget https://gitee.com/computervisionlearner/unimrcp_with_huaweicloud_asr/releases/download/v0.0.1/unimrcp-1.8.0_unimrcp-deps-1.6.0_binary.tar.gztar -xzvf unimrcp-1.8.0_unimrcp-deps-1.6.0_binary.tar.gz
复制代码


点击关注,第一时间了解华为云新鲜技术~ 

发布于: 刚刚阅读数: 3
用户头像

提供全面深入的云计算技术干货 2020-07-14 加入

生于云,长于云,让开发者成为决定性力量

评论

发布
暂无评论
保姆级教程:教你UniMRCP对接华为云ASR(Linux版)_开源_华为云开发者联盟_InfoQ写作社区