This application demonstrates the combination of the Conversation and Discovery services. First, users pose questions to the Conversation service. If Conversation is not able to confidently answer, Conversation with Discovery executes a call to Discovery to provide the user with a list of helpful answers.
Either way you deploy this app, you must have a Bluemix account and run some steps within Bluemix.
How the app works
Commands may be issued to the car to perform simple operations. These commands are run against a small set of sample data trained with intents like "turn_on", "weather", and "capabilities".
Example commands that can be executed by the Conversation service are:
turn on windshield wipers play music
This app has also ingested and trained itself based on a car manual. In addition to conversational commands, you can also ask questions that you would expect to have answered in your car manual. For example:
How do I check my tire pressure How do I turn on cruise control How do I improve fuel efficiency How do I connect my phone to bluetooth
Getting Started using Bluemix
Deploy the App
Log in with an existing Bluemix account or sign up.
Name your app and then select Create. This performs multiple actions:
- Creates the web application
- Creates a Conversation service instance
- Creates a Discovery service instance
The status of the deployment is shown. This can take some time.
When setup is complete, you are informed that you need to add environment variables.
Navigate to the Conversations service in your Bluemix Dashboard and import a workspace. Set up your workspace then return to these steps.
Navigate to the Discovery service in your Bluemix Dashboard and ingest the provided documents into a new Discovery collection. Afterwards, return to these steps
After you have set up a Conversations workspace and a Discovery collection, add your environment variables to your Bluemix app
Getting Started locally
Before you begin
- Ensure that you have a Bluemix account. While you can do part of this deployment locally, you must still use Bluemix.
- Ensure that you have Websphere Liberty Profile Server.
Create the services
In Bluemix, create a Conversation Service instance.
In Bluemix, create a Discovery Service instance.
To build the application:
Clone the repository
git clone https://github.com/watson-developer-cloud/conversation-with-discovery
Navigate to the
For Windows, type
gradlew.bat build. Otherwise, type
The built WAR file (conversation-with-discovery-0.1-SNAPSHOT.war) is in the
- Copy the WAR file generated above into the Liberty install directory's dropins folder. For example,
<liberty install directory>/usr/servers/<server profile>/dropins.
- Navigate to the
conversation-with-discovery/src/main/resourcesfolder. Copy the
- Navigate to the
<liberty install directory>/usr/servers/<server name>/folder (where < server name > is the name of the Liberty server you wish to use). Paste the
- In the
server.envfile, in the "conversation" section.
- Populate the "password" field.
- Populate the "username" field.
- Add the WORKSPACE_ID that you copied earlier.
- In the
server.envfile, in the "discovery" section.
- Populate the "password" field.
- Populate the "username" field.
- Add the COLLECTION_ID and ENVIRONMENT_ID that you copied from the Discovery UI
- Start the server using Eclipse or CLI with the command
server run <server name>(use the name you gave your server).
- Liberty notifies you when the server starts and includes the port information.
- Open your browser of choice and go to the URL displayed in Step 6. By default, this is
Create a collection and ingest documents in Discovery
- Navigate to your Discovery instance in your Bluemix dashboard
Create a new data collection, name it whatever you like, and select the default configuration.
After you're done, there should be a new private collection in the UI
Set up the custom configuration using one of the two options
Download and unzip the manualdocs.zip in this repo to reveal a set of JSON documents
In the tooling interface, drag and drop (or browse and select) all of the JSON files into the "Add data to this collection" box
- This may take a few minutes -- you will see a notification when the process is finished
Set up a custom configuration with the Discovery API
Download the FordConfig.json in this repo
Open your computer's command line interface or terminal
Copy and paste the curl command below into your command line
Run the command
curl -X POST -u "<username>:<password>" \ -H "Content-Type: application/json" \ –data “@<path_to_config> \ "https://gateway.watsonplatform.net/discovery/api/v1/environments/<environment_id>/configurations?version=2016-12-01"
Set up a custom configuration with the Discovery Tooling
The configuration for this app includes several updates from the default that are meant to help improve the results for long-tail searches. To create this configuration in the tooling, go into the collection and where the DefaultConfiguration is listed, select “Switch”
This brings up the configuration editor. There are three steps in a configuration, Convert, Enrich, and Normalize. The configuration editor allows you to upload a sample document to preview the results of a configuration as you make changes.
- To use the preview, add manual_0.json into the pane on the right
For the Convert step, only JSON cleanup is needed for these documents. In this case what is needed is to create two new fields that are copies of the original body and title fields so that we can use the copies in a later step to create a searchable text field.
- To create these fields, select “Add field”, set the action to “copy” and enter title to searchTitle, and repeat for body to searchText
- To see how the preview is affected by this step and ensure it is creating the new fields, click Apply & Save at the bottom of the page then select manual_0.json. The right side preview pane should update to display the results of the changes.
The next step in configuration is Enrich. Select “Enrich” from the top bar. In this we’ll set the configuration to enrich the body field of each document so we can use the enriched metadata to improve search.
- First remove the existing “text” field that is being enriched with the default configuration by clicking the – to the right of the field.
- Under “Add a field” enter “body” or select “body” from the dropdown
- This will add a new field to be enriched below. You can then select the enrichments that should be applied to this field by clicking “Add enrichments”. In this case we can apply Keyword Extraction, Concept Extraction, Taxonomy Classification, and Entity Extraction. These enrichments add meta data to the documents that help improve search.
- Once the enrichments have been selected, choose “Done” then again choose “Apply and Save” to see the results of the changes. Now there should be a body_enriched field in the preview that shows all the applied enrichments over the document
The final step in configuration is Normalize. This step allow you to clean up the data and fields so that you have a consistent structure for your use cases.
- For these documents, we want to create searchable fields that can be use to improve the quality of results we get back for long tail questions
- We will create two fields, one searchText that contains the combination of title and body, and one enrichedText field that contains the combination of extracted concepts and keywords from the body field.
- To do this the concepts and keywords text first need to be copied into intermediate fields. To do this, click “Add field” and enter the fully qualified path to the keyword text (body_enriched.keywords.text) to enrichedText. Hint: Use the preview pane to find the paths to fields you need. Repeat this step for body_enirched.concepts.text to conceptText
- Now we can merge the two intermediate fields together into the enrichedText field. Again select “Add a field”, choose the “merge” action and enter conceptText to enrichedText. This will merge the conceptText field into the enrichedText field, removing conceptText and preserving the combined enirchedText
- Similarly we want to combine searchTitle and searchText, so click “Add a field” and merge searchTitle into searchText
- Finally, again select apply&save to store the updated the configuration and see the preview results.
- In the preview pane scroll to the bottom of the new document, and you should see a searchText field containing title + body and an enrichedText field containing a list of the concepts and keywords extracted from the data.
Go to the Bluemix Dashboard and select the Conversation/Discovery service instance. Once there, select the Service Credentials menu item.
Select New Credential. Name your credentials then select Add.
Copy the credentials (or remember this location) for later use.
Import a workspace
To use the app you're creating, you need to add a workspace to your Conversation service. A workspace is a container for all the artifacts that define the behavior of your service (ie: intents, entities and chat flows). For this sample app, a workspace is provided.
For more information on workspaces, see the full Conversation service documentation.
Navigate to the Bluemix dashboard and select the Conversation service you created.
Click the Launch Tool button under the Manage tab. This opens a new tab in your browser, where you are prompted to login if you have not done so before. Use your Bluemix credentials.
Download the exported JSON file that contains the Workspace contents.
Select the import icon: . Browse to (or drag and drop) the JSON file that you downloaded in Step 3. Choose to import Everything(Intents, Entities, and Dialog). Then select Import to finish importing the workspace.
Refresh your browser. A new workspace tile is created within the tooling. Select the menu button within the workspace tile, then select View details:
Return to the deploy steps that you were following:
Adding environment variables in Bluemix
- In Bluemix, open the application from the Dashboard. Select Runtime and then Environment Variables.
- In the User Defined section, add the following Conversations environment variables:
- Then add the following four Discovery environment variables to this section:
- DISCOVERY_PASSWORD: Use your Discovery service credentials
- DISCOVERY_USERNAME: Use your Discovery service credentials
- DISCOVERY_COLLECTION_ID: Find your collection ID in the Discovery collection you created
- DISCOVERY_ENVIRONMENT_ID: Find your environment ID in the Discovery collection you created
- Select SAVE.
- Restart your application.
Troubleshooting in Bluemix
- Log in to Bluemix, you'll be taken to the dashboard.
- Navigate to the the application you previously created.
- Select Logs.
This sample code is licensed under Apache 2.0. Full license text is available in LICENSE.
Open Source @ IBM
Find more open source projects on the IBM Github Page.