Speech: Queuing messages for text-to-speech

[corinagum]

Reference: #1714

Customer had an issue where text-to-speech has been cutting out on Web Chat. To quote the conclusion:

So when the "additional questions" type message gets into the channel it interrupts the reading task to focus on the new task.
A workaround for now is to stop using "additional questions" type messages.
On the other hand it would be great if the webchat.js could queue messages to read in queued fashion.

(emphasis mine)

Queuing messages would prevent the bot from 'cutting out' and interrupting whatever current utterance is being made. Is queuing messages to prevent interruption of speech something we want to support in the near future? @compulim in particular has done the most work with Speech recently and may have thoughts on the matter.

[compulim]

We tested on Mockbot that the new incoming activity did not interrupt the current speaking activity. However, the new incoming activity is not spoken too.

Repro steps

1. Go to webchat-playground
2. Click the microphone button
3. Say "proactive"
4. The bot should come back in 2 batches of activity
   • The second batch come in when the first batch is being spoken

Expected

The second activity will be spoken immediately following the completion of the first activity. The second activity will not interrupt the first activity.

Actual

The second activity did not interrupt the speaking of the first activity. However, the second activity is not spoken either.

[compulim]

We START_DICTATE immediately after the first MARK_ACTIVITY completed. Looks like something wrong with startDictateAfterSpeakActivitySaga.js.

[darrenj]

I believe this issue is the one we've hit with testing, here are some examples to help validate the change:

• If one response is sent back WebChat appears to perform TTS reliably, eg:

• If two responses are sent back WebChat fails to perform TTS on anything (despite Speak properties being set). Occasionally it will play the last message, sometimes truncated. Other times nothing is heard. I suspect the first is being played, the second comes in and does not wait until the first is completed. Due to the race condition it’s luck of the drawer if you hear anything.

• This I believe is another example of “two responses” – “I found the following” and then a carousel of adaptive cards. Speak properties are set on both yet nothing is heard.

@ryanlengel can let you internally have a pointer to a bot so you can validate the change.

[alokraj68]

How to stop the bot from speaking and clear all the queues? This was available in v3 as we could reset the queue. in v4, the queue seems empty and this functionality seems missing.

This is not a how-to question, this is a missing functionality in v4.

[corinagum]

@alokraj68, this is feature work done with the Cognitive Service team for 4.6, which is when we will bump the dependency in Web Chat. This is not yet testable in our latest bits but will be released at the beginning of November.

[alokraj68]

@alokraj68, this is feature work done with the Cognitive Service team for 4.6, which is when we will bump the dependency in Web Chat. This is not yet testable in our latest bits but will be released at the beginning of November.

Please do let me know once it's in the latest bits so that we can test it.