Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Getting No NLSML data available on speech recognition input validation [AzureSR] #325

Open
ackris opened this issue Mar 6, 2024 · 0 comments
Open

Getting No NLSML data available on speech recognition input validation [AzureSR] #325

ackris opened this issue Mar 6, 2024 · 0 comments

Comments

@ackris
Copy link

ackris commented Mar 6, 2024

Hi Team,

We are getting no nlsml data available error when we had executed run bsr1.

2024-03-06 06:52:18:992359 [INFO]   Raise App MRCP Response umc-1 <9567c86c26324318>
2024-03-06 06:52:19:238253 [INFO]   Receive MRCPv2 Data 10.0.0.4:47574 <-> 10.0.0.4:1544 [154 bytes]
MRCP/2.0 154 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: 9567c86c26324318@speechrecog
Completion-Cause: 006 recognizer-error
Waveform-Uri: 


2024-03-06 06:52:19:238341 [INFO]   Raise App MRCP Event umc-1 <9567c86c26324318>
2024-03-06 06:52:19:238404 [WARN]   No NLSML data available
2024-03-06 06:52:19:238448 [INFO]   Receive App Request umc-1 <9567c86c26324318> [1]
2024-03-06 06:52:19:238463 [INFO]   Terminate Session umc-1 <9567c86c26324318>
2024-03-06 06:52:19:238576 [INFO]   Remove Control Channel <9567c86c26324318@speechrecog> [0]
2024-03-06 06:52:19:238616 [INFO]   Close TCP/MRCPv2 Connection 10.0.0.4:47574 <-> 10.0.0.4:1544
2024-03-06 06:52:19:239188 [INFO]   Receive SIP Event [nua_r_bye] Status 200 OK [SIP-Agent-1]
2024-03-06 06:52:19:239386 [INFO]   Receive SIP Event [nua_i_state] Status 200 to BYE [SIP-Agent-1]
2024-03-06 06:52:19:239423 [NOTICE] SIP Call State umc-1 [terminated]
2024-03-06 06:52:19:239502 [INFO]   Session Terminated umc-1 <9567c86c26324318>
2024-03-06 06:52:19:241109 [INFO]   Close RTP Transmitter 10.0.0.4:4000 -> 10.0.0.4:5020 [s:12 o:3840]
2024-03-06 06:52:19:241152 [INFO]   Remove RTP Session 10.0.0.4:4000
2024-03-06 06:52:19:241220 [NOTICE] Destroy TCP/MRCPv2 Connection 10.0.0.4:47574 <-> 10.0.0.4:1544
2024-03-06 06:52:19:241248 [INFO]   Remove MRCP Handle umc-1 <9567c86c26324318>
2024-03-06 06:52:19:241257 [INFO]   Raise App Response umc-1 <9567c86c26324318> [1] SUCCESS [0]
2024-03-06 06:52:19:241329 [NOTICE] Destroy MRCP Handle umc-1

However, we are able to 200 Ok for DTMF on executing run bsr2.


2024-03-06 06:53:03:881693 [INFO]   Raise App MRCP Response umc-2 <846f2c75a1f34801>
2024-03-06 06:53:03:942430 [INFO]   Receive MRCPv2 Data 10.0.0.4:40564 <-> 10.0.0.4:1544 [113 bytes]
MRCP/2.0 113 START-OF-INPUT 1 IN-PROGRESS
Channel-Identifier: 846f2c75a1f34801@speechrecog
Input-Type: dtmf


2024-03-06 06:53:03:942519 [INFO]   Raise App MRCP Event umc-2 <846f2c75a1f34801>
2024-03-06 06:53:04:302730 [INFO]   Receive MRCPv2 Data 10.0.0.4:40564 <-> 10.0.0.4:1544 [378 bytes]
MRCP/2.0 378 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: 846f2c75a1f34801@speechrecog
Completion-Cause: 000 success
Content-Type: application/x-nlsml
Content-Length: 193

<?xml version="1.0"?>
<result>
  <interpretation grammar="builtin:dtmf/digits" confidence="1">
    <instance>1234</instance>
    <input mode="dtmf">1 2 3 4</input>
  </interpretation>
</result>
2024-03-06 06:53:04:302865 [INFO]   Raise App MRCP Event umc-2 <846f2c75a1f34801>
2024-03-06 06:53:04:303056 [INFO]   Interpretation[0].confidence: 1.00
2024-03-06 06:53:04:303082 [INFO]   Interpretation[0].grammar: builtin:dtmf/digits
2024-03-06 06:53:04:303098 [INFO]   Interpretation[0].instance[0]: 1234
2024-03-06 06:53:04:303112 [INFO]   Interpretation[0].input: 1 2 3 4
2024-03-06 06:53:04:303119 [INFO]   Interpretation[0].input.mode: dtmf
2024-03-06 06:53:04:303126 [INFO]   Interpretation[0].input.confidence: 1.00
2024-03-06 06:53:04:303183 [INFO]   Receive App Request umc-2 <846f2c75a1f34801> [1]
2024-03-06 06:53:04:303203 [INFO]   Terminate Session umc-2 <846f2c75a1f34801>
2024-03-06 06:53:04:303301 [INFO]   Remove Control Channel <846f2c75a1f34801@speechrecog> [0]
2024-03-06 06:53:04:303335 [INFO]   Close TCP/MRCPv2 Connection 10.0.0.4:40564 <-> 10.0.0.4:1544
2024-03-06 06:53:04:303853 [INFO]   Receive SIP Event [nua_r_bye] Status 200 OK [SIP-Agent-1]
2024-03-06 06:53:04:303874 [INFO]   Receive SIP Event [nua_i_state] Status 200 to BYE [SIP-Agent-1]
2024-03-06 06:53:04:303883 [NOTICE] SIP Call State umc-2 [terminated]
2024-03-06 06:53:04:303977 [INFO]   Session Terminated umc-2 <846f2c75a1f34801>
2024-03-06 06:53:04:311096 [INFO]   Close RTP Transmitter 10.0.0.4:4002 -> 10.0.0.4:5022 [s:33 o:132]
2024-03-06 06:53:04:311139 [INFO]   Remove RTP Session 10.0.0.4:4002
2024-03-06 06:53:04:311269 [NOTICE] Destroy TCP/MRCPv2 Connection 10.0.0.4:40564 <-> 10.0.0.4:1544
2024-03-06 06:53:04:311367 [INFO]   Remove MRCP Handle umc-2 <846f2c75a1f34801>
2024-03-06 06:53:04:311394 [INFO]   Raise App Response umc-2 <846f2c75a1f34801> [1] SUCCESS [0]
2024-03-06 06:53:04:311436 [NOTICE] Destroy MRCP Handle umc-2

The below are the logs.

2024-03-06 06:53:44:772364 [NOTICE] Add Session <a43daf0874ea42f6>
2024-03-06 06:53:44:772377 [INFO]   Receive Offer 0x7f99a8004838 <a43daf0874ea42f6> [c:1 a:1 v:0]
2024-03-06 06:53:44:772386 [INFO]   Found MRCP Engine [Azure-SR-1] for Resource [speechrecog] 0x7f99a8004838 <a43daf0874ea42f6>
2024-03-06 06:53:44:772436 [INFO]   Add Pending Control Channel <a43daf0874ea42f6@speechrecog> [1]
2024-03-06 06:53:44:782085 [INFO]   Enable RTP Session 10.0.0.4:5024
2024-03-06 06:53:44:782111 [INFO]   Open RTP Receiver 10.0.0.4:5024 <- 10.0.0.4:4004 playout [50 ms] bounds [0 - 600 ms] adaptive [1] skew detection [1]
2024-03-06 06:53:44:782117 [INFO]   Media Path 0x7f99a8004838 Source->[PCMU/16000/1]->Decoder->[LPCM/16000/1]->Bridge->[LPCM/16000/1]->Sink
2024-03-06 06:53:44:782177 [INFO]   Open <a43daf0874ea42f6@azuresr>
2024-03-06 06:53:44:782197 [NOTICE] AZURESR Usage: 1/1/2
2024-03-06 06:53:44:782208 [NOTICE] Usage [umsazuresr] min [0] cur [1] max [1]
2024-03-06 06:53:44:782224 [INFO]   Send Answer 0x7f99a8004838 <a43daf0874ea42f6> [c:1 a:1 v:0] Status OK
2024-03-06 06:53:44:782239 [INFO]   Local SDP 0x7f99a8004838 <a43daf0874ea42f6>
v=0
o=UniMRCPServer 0 0 IN IP4 10.0.0.4
s=-
c=IN IP4 10.0.0.4
t=0 0
m=application 1544 TCP/MRCPv2 1
a=setup:passive
a=connection:new
a=channel:a43daf0874ea42f6@speechrecog
a=cmid:1
m=audio 5024 RTP/AVP 97 102
a=rtpmap:97 PCMU/16000
a=rtpmap:102 telephone-event/16000
a=fmtp:102 0-15
a=recvonly
a=ptime:20
a=mid:1

2024-03-06 06:53:44:782549 [INFO]   Receive SIP Event [nua_i_state] Status 200 OK [SIP-Agent-1]
2024-03-06 06:53:44:782560 [NOTICE] SIP Call State 0x7f99a8004838 [completed]
2024-03-06 06:53:44:782744 [INFO]   Receive SIP Event [nua_i_ack] Status 200 OK [SIP-Agent-1]
2024-03-06 06:53:44:782757 [INFO]   Receive SIP Event [nua_i_state] Status 200 OK [SIP-Agent-1]
2024-03-06 06:53:44:782761 [NOTICE] SIP Call State 0x7f99a8004838 [ready]
2024-03-06 06:53:44:782765 [INFO]   Receive SIP Event [nua_i_active] Status 200 Call active [SIP-Agent-1]
2024-03-06 06:53:44:783035 [NOTICE] Accepted TCP/MRCPv2 Connection 10.0.0.4:1544 <-> 10.0.0.4:44106
2024-03-06 06:53:44:791756 [INFO]   Receive MRCPv2 Data 10.0.0.4:1544 <-> 10.0.0.4:44106 [335 bytes]
MRCP/2.0 335 RECOGNIZE 1
Channel-Identifier: a43daf0874ea42f6@speechrecog
Content-Id: request1@form-level
Content-Type: text/uri-list
Cancel-If-Queue: false
No-Input-Timeout: 5000
Recognition-Timeout: 10000
Start-Input-Timers: true
Confidence-Threshold: 0.5
Save-Waveform: true
Content-Length: 25

builtin:speech/transcribe
2024-03-06 06:53:44:791823 [INFO]   Assign Control Channel <a43daf0874ea42f6@speechrecog> to Connection 10.0.0.4:1544 <-> 10.0.0.4:44106 [0] -> [1]
2024-03-06 06:53:44:791888 [INFO]   Process RECOGNIZE Request <a43daf0874ea42f6@speechrecog> [1]
2024-03-06 06:53:44:791955 [INFO]   Init Speech Detector: frame-duration=10 ms, frame-size=320, max-frame-count=1550, output-frame-count=20, vad-mode=2, noinput-timeout=5000 ms, input-timeout=10000 ms, start-timeout=50 ms, complete-timeout=1000 ms, incomplete-timeout=15000 ms, leading-silence=300 ms, trailing-silence=300 ms, interim-results=1, start-of-input=external <a43daf0874ea42f6>
2024-03-06 06:53:44:791970 [INFO]   Start No-Input Timer [5000 ms] <a43daf0874ea42f6>
2024-03-06 06:53:44:791974 [INFO]   Open Waveform File for Writing /opt/unimrcp/var/atomax-utterance-a43daf0874ea42f6-1-16-kHz.wav, sampling-rate [16000]
2024-03-06 06:53:44:792041 [INFO]   Set Subscription Profile [cognitive.subscription.key] <a43daf0874ea42f6@azuresr>
2024-03-06 06:53:44:792046 [INFO]   Set Recognition Mode [interactive] <a43daf0874ea42f6@azuresr>
2024-03-06 06:53:44:792089 [INFO]   Process RECOGNIZE Response <a43daf0874ea42f6@speechrecog> [1]
2024-03-06 06:53:44:792093 [INFO]   State Transition IDLE -> RECOGNIZING <a43daf0874ea42f6@speechrecog>
2024-03-06 06:53:44:792102 [INFO]   Initiate WS connection <a43daf0874ea42f6> guid [aca0dbae3cbb4195a4d3c8e14001070e] uri [https://eastus.stt.speech.microsoft.com/speech/recognition/interactive/cognitiveservices/v1?format=detailed&language=en-US]
2024-03-06 06:53:44:792110 [INFO]   Send MRCPv2 Data 10.0.0.4:1544 <-> 10.0.0.4:44106 [83 bytes]
MRCP/2.0 83 1 200 IN-PROGRESS
Channel-Identifier: a43daf0874ea42f6@speechrecog


2024-03-06 06:53:44:804328 [INFO]   WS connected <a43daf0874ea42f6>
2024-03-06 06:53:44:804342 [INFO]   WS upgrade guid [aca0dbae3cbb4195a4d3c8e14001070e] <a43daf0874ea42f6>
2024-03-06 06:53:44:815755 [INFO]   WS upgraded <a43daf0874ea42f6>
HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Date: Wed, 06 Mar 2024 06:53:44 GMT
Server: Kestrel
Upgrade: websocket
Strict-Transport-Security: max-age=31536000; includeSubDomains
Sec-WebSocket-Accept: gvh9ZueaP4ZYkOBGy9Ch0gHvxmo=


2024-03-06 06:53:44:815800 [INFO]   Send WS msg [232 bytes] <a43daf0874ea42f6>
Path:speech.config
X-Timestamp:2024-03-06T06:53:44.815778
Content-Type:application/json; charset=utf-8

{"context":{"system":{"version":"2.0.12341"}},"device":{"manufacturer":"UniSpeech","model":"umsazuresr","version":"1.27.0"}}
2024-03-06 06:53:45:032092 [INFO]   Speech Detector State Transition NO-INPUT -> IN-PROGRESS [250 ms] <a43daf0874ea42f6>
2024-03-06 06:53:45:032148 [INFO]   Start Input Timer [10000 ms] <a43daf0874ea42f6>
2024-03-06 06:53:45:032403 [INFO]   Send WS msg [517 bytes] <a43daf0874ea42f6>
Path:speech.context
X-RequestId:ed1816c4fc674573a11f3f99cf99144c
X-Timestamp:2024-03-06T06:53:45.032369
Content-Type:application/json; charset=utf-8

{"languageId":{"languages":[],"onSuccess":{"action":"Recognize"},"onUnknown":{"action":"None"},"mode":"DetectAtAudioStart","Priority":"PrioritizeLatency"},"phraseDetection":{"onSuccess":{"action":"None"},"onUnknown":{"action":"None"}},"phraseOutput":{"interimResults":{"resultType":"Auto"},"phraseResults":{"resultType":"Always"}},"audio":{"streams":{"1":null}}}
2024-03-06 06:53:45:032571 [INFO]   Send WS bin msg [5930 bytes] <a43daf0874ea42f6>
Path:audio
X-RequestId:ed1816c4fc674573a11f3f99cf99144c
X-Timestamp:2024-03-06T06:53:45.032555
Content-Type:audio/x-wav

2024-03-06 06:53:45:036553 [INFO]   Received WS msg [186 bytes] <a43daf0874ea42f6>
X-RequestId:ed1816c4fc674573a11f3f99cf99144c
Path:turn.start
Content-Type:application/json; charset=utf-8

{
  "context": {
    "serviceTag": "3de1a8d6595c44e7a5cff086ae8bf10b"
  }
}
2024-03-06 06:53:45:037637 [INFO]   Received WS close [36 bytes] status code [1007] reason [Could not validate speech context.] <a43daf0874ea42f6>
2024-03-06 06:53:45:037678 [INFO]   Respond to WS close [36 bytes] status code [1007] reason [Could not validate speech context.] <a43daf0874ea42f6>
2024-03-06 06:53:45:037696 [INFO]   Close HTTP connection <a43daf0874ea42f6>
2024-03-06 06:53:45:037711 [INFO]   Received WS close [36 bytes] status code [1007] reason [Could not validate speech context.] <a43daf0874ea42f6>
2024-03-06 06:53:45:037722 [INFO]   Close HTTP connection <a43daf0874ea42f6>
2024-03-06 06:53:45:038010 [NOTICE] UTTM total file entries [16] timers [16]
2024-03-06 06:53:45:038115 [NOTICE] RDRM total file entries [16] timers [16]
2024-03-06 06:53:45:038150 [INFO]   Process RECOGNITION-COMPLETE Event <a43daf0874ea42f6@speechrecog> [1]
2024-03-06 06:53:45:038178 [INFO]   State Transition RECOGNIZING -> RECOGNIZED <a43daf0874ea42f6@speechrecog>
2024-03-06 06:53:45:038226 [INFO]   Send MRCPv2 Data 10.0.0.4:1544 <-> 10.0.0.4:44106 [154 bytes]
MRCP/2.0 154 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: a43daf0874ea42f6@speechrecog
Completion-Cause: 006 recognizer-error
Waveform-Uri: 


2024-03-06 06:53:45:038775 [INFO]   TCP/MRCPv2 Peer Disconnected 10.0.0.4:1544 <-> 10.0.0.4:44106
2024-03-06 06:53:45:039041 [INFO]   Receive SIP Event [nua_i_bye] Status 200 Session Terminated [SIP-Agent-1]
2024-03-06 06:53:45:039065 [INFO]   Receive SIP Event [nua_i_state] Status 200 Session Terminated [SIP-Agent-1]
2024-03-06 06:53:45:039076 [NOTICE] SIP Call State 0x7f99a8004838 [terminated]
2024-03-06 06:53:45:039096 [INFO]   Receive SIP Event [nua_i_terminated] Status 200 Session Terminated [SIP-Agent-1]
2024-03-06 06:53:45:039117 [INFO]   Deactivate Session 0x7f99a8004838 <a43daf0874ea42f6>
2024-03-06 06:53:45:039128 [INFO]   Terminate Session 0x7f99a8004838 <a43daf0874ea42f6>
2024-03-06 06:53:45:039173 [INFO]   Remove Control Channel <a43daf0874ea42f6@speechrecog> [0]
2024-03-06 06:53:45:039178 [INFO]   Close <a43daf0874ea42f6@azuresr>
2024-03-06 06:53:45:039293 [INFO]   Close HTTP connection <a43daf0874ea42f6>
2024-03-06 06:53:45:039357 [NOTICE] AZURESR Usage: 0/1/2
2024-03-06 06:53:45:039363 [NOTICE] Usage [umsazuresr] min [0] cur [0] max [1]
2024-03-06 06:53:45:042183 [INFO]   Close RTP Receiver 10.0.0.4:5024 <- 10.0.0.4:4004 [r:12 l:0 j:70 p:50 d:0 i:0]
2024-03-06 06:53:45:042216 [INFO]   Remove RTP Session 10.0.0.4:5024
2024-03-06 06:53:45:042302 [NOTICE] Destroy TCP/MRCPv2 Connection 10.0.0.4:1544 <-> 10.0.0.4:44106
2024-03-06 06:53:45:042332 [NOTICE] Remove Session <a43daf0874ea42f6>
2024-03-06 06:53:45:042341 [INFO]   Session Terminated 0x7f99a8004838 <a43daf0874ea42f6>
2024-03-06 06:53:45:042384 [NOTICE] Destroy Session <a43daf0874ea42f6>

Please let us know the reason for this error. I can see that it is properly sending websocket request to port 1544.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ackris