Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Damian Cummins
committed
Apr 1, 2017
0 parents
commit 30a0c39
Showing
16 changed files
with
2,195 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# Logs | ||
logs | ||
*.log | ||
npm-debug.log* | ||
|
||
# Runtime data | ||
pids | ||
*.pid | ||
*.seed | ||
*.pid.lock | ||
|
||
# Directory for instrumented libs generated by jscoverage/JSCover | ||
lib-cov | ||
|
||
# Coverage directory used by tools like istanbul | ||
coverage | ||
|
||
# nyc test coverage | ||
.nyc_output | ||
|
||
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) | ||
.grunt | ||
|
||
# node-waf configuration | ||
.lock-wscript | ||
|
||
# Compiled binary addons (http://nodejs.org/api/addons.html) | ||
build/Release | ||
|
||
# Dependency directories | ||
node_modules | ||
jspm_packages | ||
|
||
# Optional npm cache directory | ||
.npm | ||
|
||
# Optional eslint cache | ||
.eslintcache | ||
|
||
# Optional REPL history | ||
.node_repl_history | ||
|
||
# Output of 'npm pack' | ||
*.tgz | ||
|
||
# .DS_Store files | ||
.DS_Store | ||
|
||
#output.wav | ||
output.wav | ||
|
||
#config | ||
config.js |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
# Tell the Time | ||
|
||
> Build a robot that can tell the time in different cities with [Watson](https://www.ibm.com/watson/developercloud/conversation.html) | ||
### "Watson, what time is it in Berlin?" | ||
|
||
This module provides Node.js code to get your Raspberry Pi to tell the time in any city you choose to tell Watson about. It uses [Watson Speech to Text](https://www.ibm.com/watson/developercloud/speech-to-text.html) to parse audio from the microphone, uses [Watson Conversation](https://www.ibm.com/watson/developercloud/conversation.html) to generate a response, and uses [Watson Text to Speech](https://www.ibm.com/watson/developercloud/text-to-speech.html) to "read" out this response! | ||
|
||
**This will only run on the Raspberry Pi.** | ||
|
||
|
||
## How It Works | ||
- Listens for voice commands | ||
- Sends audio from the microphone to the Watson Speech to Text Service - STT to transcribe [Watson Speech to Text](https://www.ibm.com/watson/developercloud/speech-to-text.html) | ||
- Parses the text looking for the attention word | ||
- Once the attention word is recognized, the text is sent to [Watson Conversation](https://www.ibm.com/watson/developercloud/conversation.html) to generate the response. | ||
- The response is sent to [Watson Text to Speech](https://www.ibm.com/watson/developercloud/text-to-speech.html) to generate the audio file. | ||
- The robot speaks the response via the Alsa audio playback tools | ||
|
||
## Hardware | ||
Check out [this instructable] (http://www.instructables.com/id/Build-a-Talking-Robot-With-Watson-and-Raspberry-Pi/) to prepare your system. You will need a Raspberry Pi 3, Microphone, Speaker, and [the TJBot cardboard](https://ibmtjbot.github.io/#gettj). | ||
|
||
## Build | ||
> We recommend starting with our [step by step instructions] (http://www.instructables.com/id/Build-a-Talking-Robot-With-Watson-and-Raspberry-Pi/) to build this recipe. | ||
Get the sample code and go to the application folder. Please see this [instruction on how to clone](https://help.github.com/articles/cloning-a-repository/) a repository. | ||
|
||
|
||
Install ALSA tools (required for recording audio on Raspberry Pi) | ||
|
||
sudo apt-get install alsa-base alsa-utils | ||
|
||
Install Dependencies | ||
|
||
npm install | ||
|
||
Set the audio output to your audio jack. For more audio channels, check the [config guide. ](https://www.raspberrypi.org/documentation/configuration/audio-config.md) | ||
|
||
amixer cset numid=3 1 | ||
// This sets the audio output to option 1 which is your Pi's Audio Jack. Option 0 = Auto, Option 2 = HDMI. An alternative is to type sudo raspi-config and change the audio to 3.5mm audio jack. | ||
|
||
Update the Config file with your Bluemix credentials for all three Watson services. | ||
|
||
edit config.js | ||
enter your watson usernames, passwords and versions. | ||
|
||
## Creating a Conversation Flow | ||
You need to train your robot with what to say and when to say it. For that, we use [Watson Conversation] (https://www.ibm.com/watson/developercloud/conversation.html). Open a browser and go to [IBM Watson Conversation link](http://www.ibmwatsonconversation.com) | ||
From the top right corner, select the name of your conversation service and click 'create' to create a new workspace for your robot. You can create intents and dialogs there. [Here](http://www.instructables.com/id/Build-a-Talking-Robot-With-Watson-and-Raspberry-Pi/#step6) is a step-by-step instructions to create a conversation flow. | ||
|
||
### Creating intents | ||
|
||
An intent for telling the time might look like this: | ||
|
||
![tellTheTime intent](https://github.com/DamianCummins/tell_the_time/blob/master/images/tellTheTimeIntent.PNG) | ||
|
||
|
||
### Creating dialogs | ||
|
||
A Dialog node for telling the time for different timezones might look like this: | ||
|
||
![tellTheTime dialog](https://github.com/DamianCummins/tell_the_time/blob/master/images/tellTheTimeDialog.PNG) | ||
|
||
The response should contain a placeholder (`todays_date`) that the node application can replace with a javascript Date. When a specific city entity is found, the response for each city should return the timezone offset in the output context: | ||
|
||
![timezone in offset output](https://github.com/DamianCummins/tell_the_time/blob/master/images/tellTheTimeDialogDetails.PNG) | ||
|
||
|
||
## Running | ||
|
||
Start the application | ||
|
||
node app.js | ||
|
||
Then you should be able to speak to the microphone. | ||
The robot gets better with training. You can go to your [Watson conversation module](http://www.ibmwatsonconversation.com) to train the robot with more intents and responses. | ||
|
||
## Monitoring | ||
|
||
The app serves up a Monitoring web interface locally on port `3000`. You should be able to point to this from another device on the same network as your Rasberry Pi. The Monitoring UI allows you to stop and resume listening on TJBot's microphone. You can also tell whether the TJBot module has stopped due to an unexpected error. | ||
|
||
![tellTheTime monitor app](https://github.com/DamianCummins/tell_the_time/blob/master/images/monitorApp.gif) | ||
|
||
The logs show the REST calls made from the Monitoring UI to control TJBot: | ||
|
||
``` | ||
TJBot is listening, you may speak now. | ||
TJBot monitor app listening on port 3000! | ||
Fri, 24 Mar 2017 22:38:53 GMT LOG Router request GET /tjbot | ||
Fri, 24 Mar 2017 22:38:54 GMT LOG Router request GET /tjbot/status | ||
Fri, 24 Mar 2017 22:39:00 GMT LOG Router request POST /tjbot/pause | ||
Fri, 24 Mar 2017 22:39:19 GMT LOG Router request POST /tjbot/resume | ||
TJBot is listening, you may speak now. | ||
``` | ||
|
||
## Customization | ||
The attention word is the word you say to get the attention of the robot. | ||
The default attention word is set to 'Watson' but you can change it from config.js. Some words are easier for the robot to recognize. If decided to change the attention word, experiment with multiple words and pick the one that is easier for the robot to recognize. | ||
|
||
The default voice of TJBot is set to a male voice (`en-US_MichaelVoice`) but you can change it from config.js. Two female voices are available for TJBot (`en-US_AllisonVoice` and `en-US_LisaVoice`). | ||
|
||
// The attention word to wake up the robot. | ||
exports.attentionWord ='watson'; | ||
|
||
// You can change the voice of the robot to your favorite voice. | ||
exports.voice = 'en-US_MichaelVoice'; | ||
// Some of the available options are: | ||
// en-US_AllisonVoice | ||
// en-US_LisaVoice | ||
// en-US_MichaelVoice (the default) | ||
|
||
# Dependencies List | ||
|
||
- Watson Developer Cloud - [Watson Speech to Text](https://www.ibm.com/watson/developercloud/speech-to-text.html), [Watson Conversation](https://www.ibm.com/watson/developercloud/conversation.html), and [Watson Text to Speech](https://www.ibm.com/watson/developercloud/text-to-speech.html). | ||
- mic npm package : for reading audio input | ||
|
||
|
||
## Contributing | ||
See [CONTRIBUTING.md](../../CONTRIBUTING.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
"use strict"; | ||
|
||
var express = require("express"); | ||
var app = express(); | ||
var bodyParser = require("body-parser"); | ||
var path = require("path"); | ||
|
||
var tellTheTime = require("./tellTheTime.js")(); | ||
|
||
var port = process.env.VCAP_APP_PORT || process.env.PORT || 3000; | ||
|
||
var router = express.Router(); | ||
|
||
tellTheTime.start(); | ||
|
||
app.use(bodyParser.urlencoded({"extended": true})); | ||
app.use(bodyParser.json()); | ||
|
||
router.use(function(req, res, next) { | ||
console.log(new Date().toUTCString() + " LOG Router request " + req.method + " " + req.originalUrl); | ||
next(); | ||
}); | ||
|
||
router.get("/", function(req, res) { | ||
res.sendFile(path.join(__dirname + "/index.html")); | ||
}); | ||
|
||
router.route("/status") | ||
.get(function(req, res) { | ||
tellTheTime.isMicStopped(function(status) { | ||
if (status) { | ||
res.json({'message':'SNOOZING'}); | ||
} else { | ||
res.json({'message':'AWAKE'}); | ||
} | ||
}); | ||
}); | ||
|
||
|
||
router.route("/pause") | ||
.post(function(req, res) { | ||
tellTheTime.pause(function(err) { | ||
if (!err) { | ||
res.json({'message':'SNOOZING'}); | ||
} | ||
}); | ||
}); | ||
|
||
router.route("/resume") | ||
.post(function(req, res) { | ||
tellTheTime.start(); | ||
tellTheTime.isMicStopped(function(status) { | ||
if (status) { | ||
res.json({'message':'SNOOZING'}); | ||
} else { | ||
res.json({'message':'AWAKE'}); | ||
} | ||
}); | ||
}); | ||
|
||
app.use("/tjbot", router); | ||
|
||
app.use(function(req, res, next) { | ||
res.status(404); | ||
console.error("ERROR: Page not found"); | ||
res.json({"code": 404, "message": "Page not found."}); | ||
}); | ||
|
||
app.listen(port, function() { | ||
console.log("TJBot monitor app listening on port "+port+"!"); | ||
}); |
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
"use strict"; | ||
var fs = require('fs'); | ||
var csv = require('fast-csv'); | ||
|
||
var fs = require('fs'); | ||
var workspace = JSON.parse(fs.readFileSync('tellTheTime_workspace.json', 'utf8')); | ||
|
||
var stream = fs.createReadStream("timezones.csv"); | ||
|
||
var entityValues = []; | ||
|
||
var dialogNodeCounter = 0; | ||
|
||
var dialogs = []; | ||
|
||
|
||
|
||
var csvStream = csv() | ||
.on("data", function(data){ | ||
var zone = data[0]; | ||
var zoneTokens = zone.split("/"); | ||
var city = zoneTokens[zoneTokens.length -1].replace("_", " "); | ||
|
||
var entityValue = { | ||
value: city, | ||
synonyms: [], | ||
metadata: null | ||
}; | ||
|
||
entityValues.push(entityValue); | ||
|
||
if (city.indexOf(" ") > -1) { | ||
city = "(" + city + ")"; | ||
} | ||
var offset = parseFloat(data[1]); | ||
var dst = data[2]; | ||
var dialogResponse = { | ||
"type":"response_condition", | ||
"go_to":null, | ||
"output":{ | ||
"text":{ | ||
"values":[ | ||
"The time in @city is todays_date" | ||
], | ||
"selection_policy":"sequential" | ||
}, | ||
"context":{ | ||
"timezoneOffset": offset, | ||
"dst": dst | ||
} | ||
}, | ||
"parent":"Tell the Time", | ||
"context":null, | ||
"metadata":null, | ||
"conditions":" @city:"+city, | ||
"description":null, | ||
"dialog_node":"dialog_node_" + dialogNodeCounter, | ||
"previous_sibling":"dialog_node_" + (dialogNodeCounter - 1) | ||
}; | ||
if(dialogNodeCounter === 0) { | ||
dialogResponse["previous_sibling"] = "node_3_1490294223086"; | ||
} | ||
dialogNodeCounter++; | ||
dialogs.push(dialogResponse); | ||
|
||
console.log(data); | ||
console.log(JSON.stringify(entityValue)); | ||
console.log(JSON.stringify(dialogResponse)); | ||
|
||
}) | ||
.on("end", function(){ | ||
console.log("done"); | ||
console.log(dialogs.length); | ||
console.log(entityValues.length); | ||
workspace.entities[0].values = workspace.entities[0].values.concat(entityValues); | ||
workspace.dialog_nodes = workspace.dialog_nodes.concat(dialogs); | ||
fs.writeFileSync('tellTheTime_workspace_new.json', JSON.stringify(workspace, null, 3)); | ||
}); | ||
stream.pipe(csvStream); |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
<!DOCTYPE html> | ||
<html lang="en"> | ||
<head> | ||
<meta charset="UTF-8"> | ||
<title>TJBot Monitor</title> | ||
<meta name="viewport" content="width=device-width, initial-scale=1"> | ||
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css"> | ||
<style> | ||
body { padding-top:50px; } | ||
</style> | ||
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script> | ||
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script> | ||
<script> | ||
function refresh() { | ||
$.getJSON('/tjbot/status', | ||
function(response) { | ||
$('#status').empty().append(response.message); | ||
}); | ||
}; | ||
|
||
function pause() { | ||
// Send the request | ||
$.post('/tjbot/pause', {}, function(response) { | ||
$('#status').empty().append(response.message); | ||
alert("TJBot is snoozing... Zzz"); | ||
}, 'json'); | ||
}; | ||
|
||
function resume() { | ||
// Send the request | ||
$.post('/tjbot/resume', {}, function(response) { | ||
$('#status').empty().append(response.message); | ||
alert("TJBot woke up!"); | ||
}, 'json'); | ||
}; | ||
|
||
refresh(); | ||
</script> | ||
</head> | ||
<body> | ||
|
||
<div class="container"> | ||
<div class="jumbotron"> | ||
<h1>TJBot Monitor</h1> | ||
<p>Current status of TJBot:</p><p id="status"></p> | ||
</div> | ||
<div class="container text-center col-sm-12"> | ||
<div type="button" class="btn btn-block btn-info" onclick="refresh()">Refresh</div><br/> | ||
<div type="button" class="btn btn-block btn-warning" onclick="pause()">Let TJBot snooze</div><br/> | ||
<div type="button" class="btn btn-block btn-success" onclick="resume()">Wake TJBot up</div> | ||
</div> | ||
</div> | ||
|
||
</body> | ||
</html> |
Oops, something went wrong.