| page_type | languages | products | description | urlFragment | ||
|---|---|---|---|---|---|---|
sample |
|
|
The Direct Line Speech Client is a Windows Presentation Foundation (WPF) application in C# that makes it easy to test interactions with your bot before creating a custom client application. |
Cognitive-Services-Direct-Line-Speech-Client |
The Direct Line Speech Client is a Windows Presentation Foundation (WPF) application in C# that makes it easy to test interactions with your bot before creating a custom client application. It demonstrates how to use the Azure Speech Services Speech SDK to manage communication with your Azure Bot-Framework bot. To use this client, you need to register your bot with the Direct Line Speech channel. Direct Line Speech Client is used in the tutorial Voice-enable your bot using the Speech SDK.
Following the introduction of Custom Commands (in preview) in Speech SDK 1.8, this tool was updated to accept a custom command application ID. This allows you to test your task completion or command-and-control scenario hosted on the Custom Command service.
- Fully configurable to support any bot registered with the Direct Line Speech channel
- Accepts typed text and speech captured by a microphone as inputs for your bot
- Supports playback of audio response
- Supports use of custom wake-words
- Supports sending custom Bot-Framework Activities as JSON to the bot
- Displays Adaptive Cards sent from your bot (with some limitations)
- Exports the transcript and activity logs to a file
Let's review the hardware, software, and subscriptions that you'll need to use this client application.
- Windows 10 PC
- Visual Studio 2017 or higher
- Microphone access
- An Azure Speech Services Key
- A Bot-Framework bot service registered with Direct Line Speech channel
- The first step is to clone the repository:
git clone https://github.com/Azure-Samples/Cognitive-Services-Direct-Line-Speech-Client.git
- Then change directories:
cd Cognitive-Services-Direct-Line-Speech-Client - Launch Visual Studio 201 or newer, then open the solution for the Direct Line Speech Client:
DLSpeechClient.sln. The solution is in the root of the cloned repository. - Run the executable. For example, for Release x64 build:
DLSpeechClient\bin\x64\Release\DLSpeechClient.exe. - When you first run the application, the Setting page will open. The first two fields are required (all others are optional):
- Enter Subscription key. This is your Azure Speech Services Key
- Enter Subscription key region. This is the Azure region of your key in the format specified by the "Speech SDK Parameter" column in this table (for example "westus")
- Leave the field Custom commands app Id empty (unless you plan to use Custom Commands)
- The default input language is "en-us" (US English). Update the Language field as needed to select a different language code from the "Speech-to-text" list.
- Press Ok when you're done.
- Your entires will be saved and populated automatically when you launch the app again.
- Press Reconnect. The application will try to connect to your bot via Direct Line Speech channel.
The message New conversation started -- type or press the microphone button will appear below the text bar if the connection succeeded. - You'll be prompted to allow microphone access. If you want to use the microphone, allow access.
- Press the microphone icon to begin recording. While speaking, intermediate recognition results will be shown in the application. The microphone icon will turn red while recording is in progress. It will automatically detect end of speech and stop recording.
- If everything works, you should see your bot's response on the screen and hear it speak the response. You can click on lines in the Activity Log window to see the full activity payload from the bot in JSON.
Note: You'll only hear the bot's voice response if the Speak field in the bot's output activity was set.
If an error messages was shown in red in the main application window, use this table to troubleshoot:
| Error | What should you do? |
|---|---|
| Error AuthenticationFailure : WebSocket Upgrade failed with an authentication error (401). Please check for correct subscription key (or authorization token) and region name | In the Settings page of the application, make sure you entered the Speech Subscription key and its region correctly. |
| Error ConnectionFailure : Connection was closed by the remote host. Error code: 1011. Error details: We could not connect to the bot before sending a message | Make sure you checked the "Enable Streaming Endpoint" box and/or toggled "Web sockets" to On Make sure your Azure App Service is running. If it is, try restarting your App Service. |
| Error ConnectionFailure : Connection was closed by the remote host. Error code: 1011. Error details: Response status code does not indicate success: 500 (InternalServerError) | Your bot specified a Neural Voice in its output Activity Speak field, but the Azure region associated with your Speech subscription key does not support Neural Voices. See Standard and neural voices. |
| Error ConnectionFailure : Connection was closed by the remote host. Error code: 1000. Error details: Exceeded maximum websocket connection idle duration(> 300000ms) | This is an expected error when the client left the connection to the channel open for more than 5 minutes without any activity |
See also Debugging section in Voice-first virtual assistants Preview: Frequently asked questions
Direct Line Speech Client allows you to author and send a custom JSON activity to your bot. This is done using the "Custom Activity" bar at the bottom of the main window and the "New", "Edit" and "Send" buttons. Enter a valid JSON format that conforms to the Bot-Framework Activity schema. An example is given in the file example.json.
See section Add custom keyword activation in the tutorial.
The Speech Studio Portal allows you to create Custom Speech in order to build, analyze, and deploy custom speech recognition models. The portal will provide an "Endpoint ID" (a GUID). Enter this GUID in the "Endpoint Id" field in the settings page (under "Custom SR settings"), and check the "Enabled" box below it to have it used. Direct Line Speech channel will use your custom SR endpoint to transcribe voice.
The Speech Studio Portal allows you to create a Custom Voice, where you can record and upload training data to create a unique voice font for your applications. The portal will provide a deployment ID (a GUID). Enter this GUID in the "Voice deployment Ids" field in the settings page (under "Custom TTS settings"), and check the "Enabled" box below it to have it used. Direct Line Speech channel will use your custom TTS endpoint to create the bot's voice response. Note that the "speak" field in the bot's reply Activity must contain the name of the voice you created in the portal.
If you built your dialog using the Custom Commands service (instead of a Bot-Framework bot registered with Direct Line Speech channel), enter your custom commands application ID in the setting page ("custom commands app id"). This client application will connect to the custom commands service that hosts your dialog.
- Bot Framework docs:
- Speech SDK docs:
- About voice assistants
- Tutorial: Voice-enable your bot using the Speech SDK
- What is custom Speech?
- Create a custom voice
- Create a custom wake word by using the Speech service
- Try Speech Services for free
- Language and region support for the Speech Services
- Speech Service supported regions
- Speech Synthesis Markup Language (SSML)
- Adaptive Cards