Custom ResponseBot not matching custom vocabulary in voice mode

[amendlik]

Describe the bug
I have a custom ResponseBot built to capture city names. This includes a custom slot type containing a list of all the cities, with the Use slot values as custom vocabulary box checked.

When the custom ResponseBot is tested directly using the Lex console, it matches perfectly in text or voice mode.
When I test the QnABot entry point bot using the Lex console, it matches perfectly in text mode and fails nearly 100% in voice mode.

To Reproduce

1. Create a custom ResponseBot to match city names.
   • In the Lex console, create new bot with a name beginning with 'QNA'.
   • Add a blank slot type.
   • Select Restrict to slot values.
   • Add the city names as values. Do not use famous cities like New York, Paris, etc.
   • Check the box labeled Use slot values as custom vocabulary. According to the documentation, this is appropriate for proper nounes, like place names.
2. Add an empty intent
   • Define a slot and set the slot type to the custom type created above
   • Add one or more utterances that include the slot token. If the slot is named City, the utterance would be {City}
   • Activate Confirmation
   • Add a Confirmation prompt: Is {City} correct (Yes or No)?
   • Add a Decline response: Please provide the city name again.
   • Activate Closing response
   • Set the Message to OK.
3. Build the bot
4. Take note of the Bot ID
5. Create a QnABot document
   • Question: Ask me for a city
   • Answer: Give me a city name
   • ResponseBot Hook: lexv2::BOTID/TSTALIASID/en_US where BOTID is the ID of the bot created above
   • Response Session Attribute Namespace: city
6. Open the Lex console and locate the QnABot entry point bot. This name will be a combination of the CloudFormation stack name and the string "QnaBot"
   • Select the live alias.
   • Select the associated version
   • Select English (US)
   • Click the Test button
   • Confirm that the live alias should be tested
7. Test the bot using voice (this works fine in text mode, but fails in voice)
   • Click the microphone in the Lex console test window.
   • Say aloud, "Ask me for a city" and click the check mark
   • The system will respond with, "Give me a city name"
   • Say the name of a city in your custom slot type, defined above
   • The city name will fail to match to one listed in the custom vocabulary, even after multiple attempts.

Expected behavior
The expected behavior is that the city name should be recognized as one from the custom vocabulary with the same accuracy as if I test the custom ResponseBot directly, which is nearly 100% on the first try in voice mode.

Please complete the following information about the solution:

• Version: v5.4.5
• Region: us-west-2
• Was the solution modified from the version published on this repository? No
• If the answer to the previous question was yes, are the changes available on GitHub? N/A
• Have you checked your service quotas for the services this solution uses? Yes
• Were there any errors in the CloudWatch Logs? No

[bobpskier]

@amendlik Setting up a separate intent and slot type will only work in voice mode if you also train QnABot with the same city data for voice processing. In this situation, the Lex bot, QnABot, is being invoked using the utterance prior to calling the response bot. It does not pass the voice utterance through to the response bot but instead passes the text transcript which is most likely in error in this case as QnABot/Lex was not trained to understand it in the QnABot context.

The best way to solve this is to use QnABot to define a new slottype and a new question which uses the new slot type and creates a dedicated bot intent. This is in lieu of building a new response bot. I've attached qnabot json file that can be imported that demonstrates this technique. You can cache this slot value in a SessionAttribute for later use in QnABot in another question use it in a question flow where another qid is called via Document Chaining. When using this technique be sure to perform a Lex Rebuild from the hamburger menu in the upper right area of the admin ui tool bar.

mycityqna.json

[fhoueto-amz]

Thanks @bobpskier. @amendlik plase let us know if the provided solution works for you

[amendlik]

Thank you @bobpskier. I can see this solution working in some cases, but only when you always want to collect the city name within the first step of the conversation. If you have multiple steps, and the city name would be collected conditionally in a later step, I don't think this works.

Are there any other solutions that would support multi-step conversations?

[bobpskier]

@amendlik I think using QnABot conditional chaining would work quite nicely for this scenario. This lets one question chain to another question in QnABot. There will be a question that decides to conditionally chain into the city question. Once the city question is complete and the session attribute set, it would chain to the later step. I can work up an example if that would help.

[amendlik]

Thanks @bobpskier - What about cases where I only want to ask for a city name in some cases? If the city name is a required slot in the supervisory bot, how would I ever avoid asking for it?

BTW - I'm having some trouble testing this scenario due to #684

[amendlik]

@bobpskier I received an update on #684 that one cannot use slots and chaining within the same QnA document.

[bobpskier]

@amendlik Not the response I was expecting from the QnABot solutions team. I'll see if I can come up with a solution. This should work regardless if using a custom intent/slot or if using an elicit response bot prompting for a value. The custom intent will work better from a training perspective than using an elicit response bot.

[fhoueto-amz]

@amendlik Are you still blocked here or do you have a path based on the input from #684 ?