Voice recorder button throws an exception when qnabot is embedded on website

[chanbi-aws]

Describe the bug
Selecting the mic button when the qnabot is embedded in a HTML website (parent.html) results in an exception: "Uncaught (in promise) DOMException: The element has no supported sources." The error is on line 29 of lex-web-ui.min.js, as part of setAudioAutoPlay.

To Reproduce
Deploy the bot as an embedded UI and click the mic button.

Expected behavior
Launch the audio recorder associated with the browser.

Please complete the following information about the solution:

• Version: [e.g. v0.0.1]

To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0189) QnABot [...] v5.4.5".

• Region: [us-east-1]
• Was the solution modified from the version published on this repository? No
• If the answer to the previous question was yes, are the changes available on GitHub?
• Have you checked your service quotas for the services this solution uses? Yes
• Were there any errors in the CloudWatch Logs? No, client side error

Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context
Add any other context about the problem here.

[dougtoppin]

@chanbi-aws thanks for your report, we will investigate and get back to you

[michaelin-96]

Hi @chanbi-aws, I assume you are using v5.4.5 from your latest post. I recommend trying to upgrade to the latest version v5.5.0. I wasn't able to replicate this issue when deploying qnabot v5.5.0. Let me know if you still come across issues! Thanks.

[michaelin-96]

After discussion with @chanbi-aws, this seems to be a bug from lex-web-ui, which our QnABot solution does not manage.

[chanbi-aws]

This is a different issue to the 'invalid ssml'.

[bobpskier]

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

[chanbi-aws]

The issue occurs prior to any request to the LLM. The error is thrown when you click the 'mic' icon to record your voice input. Maybe this is an issue that needs to be raised with https://github.com/aws-samples/aws-lex-web-ui.

[bobpskier]

@chanbi-aws Is the embedded version of LexWebUi being used or was the full version from https://github.com/aws-samples/aws-lex-web-ui installed? Sounds like it is the full version based on the use of parent.html but want to confirm. If its the full version then this should be submitted against LexWebUi. Note a number of problems with voice mode were resolved in LexWebUi 0.20.2 which was released on 2023-11-28. So first step is going to be to upgrade to the latest LexWebUi at least greater than or equal to 0.20.2.

[michaelin-96]

@bobpskier This comment should refer to this ticket 694.
In terms of the mic issue in the embedded page, I recommend using the latest lex build (20.6), otherwise this may be a bug for lex-web-ui. Closing this ticket out now

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

[chanbi-aws]

I will raise an issue with aws-lex-web-ui. Updating the stack with the master.yaml (https://github.com/aws-samples/aws-lex-web-ui/blob/master/templates/master.yaml) successfully upgrades to v0.20.6, but when the UI initialises with version 0.20.5. Browser console output: "lex-web-ui.min.js:29 successfully initialized lex web ui version: 0.20.5"