voice input example?

[stephenxp04]

Hi i'm trying to do implement this with voice input request but the current sample only supports text input. Do i configure the config file to include audioIn and remove the 'delete request.audio_in' and 'text_query' to enable audio request mode?

I have searched online but cant seem to find any documentations on how to do this. Helps will be much appreciated! Thanks~!

[Fleker]

Yeah, you'll need to create an AudioInConfig instead of the text query and then start sending audio_in bytes up until you decide to close the connection or you receive AssistResponse.event_type.END_OF_UTTERANCE.

A conversation is one or more gRPC connections, each consisting of several streamed requests and responses. For example, the user says Add to my shopping list and the Assistant responds What do you want to add?. The sequence of streamed requests and responses in the first gRPC message could be:
AssistRequest.config
AssistRequest.audio_in
AssistRequest.audio_in
AssistRequest.audio_in
AssistRequest.audio_in
AssistResponse.event_type.END_OF_UTTERANCE
AssistResponse.speech_results.transcript "add to my shopping list"
AssistResponse.dialog_state_out.microphone_mode.DIALOG_FOLLOW_ON
AssistResponse.audio_out
AssistResponse.audio_out
AssistResponse.audio_out

The user then says bagels and the Assistant responds OK, I've added bagels to your shopping list. This is sent as another gRPC connection call to the Assist method, again with streamed requests and responses, such as:
AssistRequest.config
AssistRequest.audio_in
AssistRequest.audio_in
AssistRequest.audio_in
AssistResponse.event_type.END_OF_UTTERANCE
AssistResponse.dialog_state_out.microphone_mode.CLOSE_MICROPHONE
AssistResponse.audio_out
AssistResponse.audio_out
AssistResponse.audio_out
AssistResponse.audio_out