isFinal missing on streamingRecognize

[blerest]

Sometime the Speech API stuck when I say only one word using streaming recognize. The API recognize the end of the sentence as I receive correctly END_OF_SINGLE_UTTERANCE, but I never receive the transcription with isFinal=true.

This is a big problem for me as I use isFinal to reload the API connection. I can reproduce the issue on both API v1 and v1p1beta1.

{ config:
   { encoding: 1,
     sampleRateHertz: 8000,
     languageCode: 'fr-FR',
     maxAlternatives: 0,
     profanityFilter: true },
  singleUtterance: true,
  interimResults: true }

long sentence:

{"results":[{"alternatives":[{"words":[],"transcript":"pour","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"Bonjour","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça m'a","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça marche","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce","confidence":0}],"isFinal":false,"stability":0.8999999761581421},{"alternatives":[{"words":[],"transcript":" que ça marche","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que","confidence":0}],"isFinal":false,"stability":0.8999999761581421},{"alternatives":[{"words":[],"transcript":" ça marche bien","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça","confidence":0}],"isFinal":false,"stability":0.8999999761581421},{"alternatives":[{"words":[],"transcript":" marche bien","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça marche","confidence":0}],"isFinal":false,"stability":0.8999999761581421},{"alternatives":[{"words":[],"transcript":" bien","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça marche bien","confidence":0}],"isFinal":false,"stability":0.8999999761581421}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[],"error":null,"speechEventType":"END_OF_SINGLE_UTTERANCE"}
{"results":[{"alternatives":[{"words":[],"transcript":"bonjour est-ce que ça marche bien","confidence":0.9081912636756897}],"isFinal":true,"stability":0}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}

one word sentence:

{"results":[{"alternatives":[{"words":[],"transcript":"un","confidence":0}],"isFinal":false,"stability":0.009999999776482582}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[{"alternatives":[{"words":[],"transcript":"un","confidence":0}],"isFinal":false,"stability":0.8999999761581421}],"error":null,"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{"results":[],"error":null,"speechEventType":"END_OF_SINGLE_UTTERANCE"}
{"results":[],"error":{"details":[],"code":11,"message":"Exceeded maximum allowed stream duration of 65 seconds."},"speechEventType":"SPEECH_EVENT_UNSPECIFIED"}
{ Error: 11 OUT_OF_RANGE: Exceeded maximum allowed stream duration of 65 seconds.
    at createStatusError (node_modules/grpc/src/client.js:64:15)
    at ClientDuplexStream._emitStatusIfDone (node_modules/grpc/src/client.js:270:19)
    at ClientDuplexStream._receiveStatus (node_modules/grpc/src/client.js:248:8)
    at node_modules/grpc/src/client.js:804:12
  code: 11,
  metadata:
   Metadata {
     _internal_repr: { 'content-disposition': [Array], 'x-goog-trace-id': [Array] } },
  details: 'Exceeded maximum allowed stream duration of 65 seconds.' }

[alisher-agzamov]

I have the same issue but not only when I'm saying one word. Sometimes it happens while long recognition. I'm using confidence result in my app every time when I receive isFinial=true. But sometimes while long recognition I receive END_OF_SINGLE_UTTERANCE and I cannot use confidence because it always returns only with isFinal=true. And this is really important thing in my app.

Can you return confidence value even if I'm receiving END_OF_SINGLE_UTTERANCE or while interim results?

[blerest]

I still have issue. Do you have information?

[pohsiu]

How about restart the service every single minute

[beccasaurus]

Just saw this get bumped. Following up on this, will report back.

[callmehiphop]

@beccasaurus gentle ping!

[0xTea]

Has this issue ever been resolved , @blerest ... I have a similar issues of time , I have noticed the same issue where the commands is on only final after an a new world or sound has been registered between the utterances.

[kidplug]

@blerest
When you get the END_OF_SINGLE_UTTERANCE event, are you then calling stream.end() ?
That's what I'm doing and it works fine... but you still have to keep listening for the "data" event from the stream itself, which should give you the final data event with isFinal:true. Then you'll also get an end event, at which time you can "unlisten" from the stream events: data, error, and end.

From docs:
`

END_OF_SINGLE_UTTERANCE

This event indicates that the server has detected the end of the user's speech utterance and expects no additional speech. Therefore, the server will not process additional audio (although it may subsequently return additional results). The client should stop sending additional audio data, half-close the gRPC connection, and wait for any additional results until the server closes the gRPC connection. This event is only sent if single_utterance was set to true, and is not used otherwise.

`

[nnegrey]

+1 to what @kidplug said.

Also tried this myself and got:

# First result is a interim result
results {
  alternatives {
    transcript: "check"
  }
  stability: 0.009999999776482582
  result_end_time {
    nanos: 940000000
  }
}
# Second result is a interim result
results {
  alternatives {
    transcript: "check"
  }
  stability: 0.8999999761581421
  result_end_time {
    seconds: 1
    nanos: 510000000
  }
}
# Speech api detects the end of my utterance
speech_event_type: END_OF_SINGLE_UTTERANCE
# Speech api returns is_final=true
results {
  alternatives {
    transcript: "check"
    confidence: 0.9628270864486694
  }
  is_final: true
  result_end_time {
    seconds: 3
    nanos: 670000000
  }
}

If anyone is still having an issue, please comment again here and I'll try to reproduce. But I'm going to close this for now.