Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
881cca4
commit c267ecb
Showing
22 changed files
with
781 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# audioLocalization | ||
|
||
This sample shows how to code Misty to localize audio. When this code runs, Misty starts sending audio localization event messages, and prints the degree of any arrival speech as a debug message. | ||
|
||
In this sample, we use two methods from Misty's JavaScript API to register for audio localization (`SourceTrackDataMessage`) events. We use the `misty.RegisterEvent()` method to create a new event listener for messages from Misty's audio localization system, and we use the `misty.AddReturnProperty()` method to tell the system which SourceTrackDataMessage property values those event messages should include. | ||
|
||
When Misty sends an audio localization event message, that message data gets passed into a callback function where we write the code | ||
that defines how the robot should respond. | ||
|
||
You can run this code on your robot by uploading the files from this folder to Misty via the Skill Runner web tool. Alternately, refer to this code sample (or copy and paste it into your own skills) when working on similar functionality. | ||
|
||
**Tip:** This sample prints the degree of arrival speech as a debug message. You can extend this sample by adding code to find Misty's current heading (yaw value from IMU), and to calculate a new heading for Misty, so that she can turn to face the person she hears speaking. | ||
|
||
**Note:** For audio localization events, the direction Misty's head is facing is her 0/360 degrees. The system calculates the degree of arrival speech relative to this position. We start this sample by positioning Misty's head to face the same direction as her body, but keep in mind that this won't always be the case. |
91 changes: 91 additions & 0 deletions
91
JavaScript SDK Code Samples/audioLocalization/audioLocalization.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
/********************************************************************** | ||
Copyright 2019 Misty Robotics, Inc. | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
imitations under the License. | ||
**********************************************************************/ | ||
|
||
/********************************************************************** | ||
audioLocalization | ||
This sample shows how to code Misty to localize audio. When this code | ||
runs, Misty starts sending audio localization event messages, and | ||
prints the degree of any arrival speech as a debug message. | ||
In this sample, we use two methods from Misty's JavaScript API to | ||
register for audio localization (SourceTrackDataMessage) events. We use | ||
the misty.RegisterEvent() method to create a new event listener for | ||
messages from Misty's audio localization system, and we use the | ||
misty.AddReturnProperty() method to tell the system which | ||
SourceTrackDataMessage property values those event messages should | ||
include. | ||
When Misty sends an audio localization event message, that message | ||
data gets passed into a callback function where we write the code | ||
that defines how the robot should respond. | ||
We've left a lot of comments in this code for developers new to the | ||
Misty platform. If you already know your way around, feel free to | ||
ignore them! | ||
Tip: This sample prints the degree of arrival speech as a debug | ||
message. You can extend this sample by adding code to find Misty's | ||
current heading (yaw value from IMU), and to calculate a new heading | ||
for Misty, so that she can turn to face the person she hears speaking. | ||
Note: For audio localization events, the direction Misty's head is | ||
facing is her 0/360 degrees. The system calculates the degree of | ||
arrival speech relative to this position. We start this sample by | ||
positioning Misty's head to face the same direction as her body, | ||
but keep in mind that this won't always be the case. | ||
**********************************************************************/ | ||
|
||
// Sets up Misty to start listening for audio. | ||
misty.MoveHeadDegrees(-15, 0, 0, 50); | ||
registerAudioLocalization(); | ||
|
||
// Misty must be recording audio to stream audio localization data. | ||
// When this sample starts, Misty starts recording audio to a new file | ||
// called "deleteThis.wav" | ||
misty.StartRecordingAudio("deleteThis.wav"); | ||
misty.ChangeLED(0, 0, 255); // Changes LED to blue; "I'm listening!" | ||
// Stops recording after 10 seconds. Extend this duration to keep Misty | ||
// listening longer. | ||
misty.Pause(10000); | ||
misty.StopRecordingAudio(); | ||
misty.ChangeLED(0, 255, 0); // Changes LED to green; "Done listening!" | ||
|
||
// Sets up our SourceTrackDataMessage event listener. | ||
function registerAudioLocalization() { | ||
// Tells the system to print the value of the DegreeOfArrivalSpeech | ||
// property in the AdditionalResults array that comes back with | ||
// "DegreeOfArrivalSpeech" events. This value indicates the angle | ||
// relative to the direction Misty's head is facing where she | ||
// detected the loudest voice. | ||
misty.AddReturnProperty("soundIn", "DegreeOfArrivalSpeech"); | ||
// Registers a new event listener for SourceTrackDataMessage events. | ||
// (We call this event listener soundIn, but you can use any name | ||
// you like. Giving event listeners a custom name means you can | ||
// create multiple event listeners for the same type of event in a | ||
// single skill.) Our soundIn event listener has a debounce | ||
// of 100 ms, and we set the fourth argument (keepAlive) to true, | ||
// which tells the system to keep listening for | ||
// SourceTarckDataMessage events after the first message comes back | ||
misty.RegisterEvent("soundIn", "SourceTrackDataMessage", 100, true); | ||
} | ||
|
||
// Defines how Misty should respond to soundIn events. Data from each | ||
// soundIn event is passed into this callback function. | ||
function _soundIn(data) { | ||
// Prints the degree of arrival speech as a debug message. | ||
misty.Debug(data.AdditionalResults[0].toString() + " <- degree of arrival for detected audio"); | ||
} |
19 changes: 19 additions & 0 deletions
19
JavaScript SDK Code Samples/audioLocalization/audioLocalization.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
{ | ||
"Name": "audioLocalization", | ||
"UniqueId": "c6bbc564-412c-4b91-8ede-31b976e9e680", | ||
"Description": "Misty prints the degree of arrival speech she detects as a debug message", | ||
"StartupRules": [ | ||
"Manual", | ||
"Robot" | ||
], | ||
"Language": "javascript", | ||
"BroadcastMode": "debug", | ||
"TimeoutInSeconds": 36000, | ||
"CleanupOnCancel": true, | ||
"WriteToLog": false, | ||
"Parameters": { | ||
"int": 10, | ||
"double": 20.5, | ||
"string": "twenty" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# hazardNotification | ||
|
||
This sample shows how to register for hazard notification (`HazardNotification`) event messages. Once you've registered for these | ||
messages, you can use them to program Misty to back up and find another direction when she stops driving due to an obstacle or a cliff in her | ||
path. | ||
|
||
In this sample, we use the `misty.RegisterEvent()` method from Misty's JavaScript API to register for `HazardNotification` event messages. Data from `HazardNotification` event messages gets passed into a callback function where we write the code that defines how the robot should respond. In our case, this callback function prints an array with the names of the triggered hazards as a debug message. If the message shows that Misty is in a hazards state, we change her LED to white, meaning it's not safe for her to drive. If misty is NOT in a hazard | ||
state, we change the LED to white, meaning it is safe for her to drive. | ||
|
||
You can run this code on your robot by uploading the files from this folder to Misty via the Skill Runner web tool. Alternately, refer to this code sample (or copy and paste it into your own skills) when working on similar functionality. | ||
|
||
**Tip:** You can extend this sample to write a skill that has Misty autonomously roam her environment, programmatically changing direction each time she enters a hazard state. Because the `DriveStopped` hazards are each associated with a particular "region", you can check which "regions" are unsafe, and use Misty's driving commands to program her to back up and choose a new direction. |
115 changes: 115 additions & 0 deletions
115
JavaScript SDK Code Samples/hazardNotification/hazardNotification.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
/********************************************************************** | ||
Copyright 2019 Misty Robotics, Inc. | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
imitations under the License. | ||
**********************************************************************/ | ||
|
||
/********************************************************************** | ||
hazardNotification | ||
This sample shows how to register for hazard notification | ||
(HazardNotification) event messages. Once you've registered for these | ||
messages, you can use them to program Misty to back up and find another | ||
direction when she stops driving due to an obstacle or a cliff in her | ||
path. | ||
In this sample, we use the misty.RegisterEvent() method from Misty's | ||
JavaScript API to register for HazardNotification event messages. Data | ||
from HazardNotification event messages gets passed into a callback | ||
function where we write the code that defines how the robot should | ||
respond. In our case, this callback function prints an array with the | ||
names of the triggered hazards as a debug message. If the message | ||
shows that Misty is in a hazards state, we change her LED to white, | ||
meaning it's not safe for her to drive. If misty is NOT in a hazard | ||
state, we change the LED to white, meaning it is safe for her to drive. | ||
We've left a lot of comments in this code for developers new to the | ||
Misty platform. If you already know your way around, feel free to | ||
ignore them! | ||
**********************************************************************/ | ||
|
||
// Changes LED to white | ||
misty.ChangeLED(255, 255, 255); | ||
|
||
// Tells the system to print the value of the DriveStopped property in | ||
// the AdditionalResults array that comes back with "Hazard" event | ||
// messages. This value tells us which hazard state an event message is | ||
// associated with. This sample only uses data from the DriveStopped | ||
// hazard state; however, you could add additional return properties of | ||
// HazardNotification event messages to programmatically respond to | ||
// other types of hazard states. | ||
misty.AddReturnProperty("Hazard", "DriveStopped"); | ||
|
||
// Registers a new event listener for HazardNotification events. (We | ||
// call this event listener Hazard, but you can use any name you like. | ||
// Giving event listeners a custom name means you can create multiple | ||
// event listeners for the same type of event in a single skill.) Our | ||
// Hazard event listener has a debounce of 0 ms - HazardNotification | ||
// event messages are sent whenever a hazard state changes, and do not | ||
// use timed intervals - and we set the fourth argument (keepAlive) to | ||
// true, which tells the system to keep listening for Hazard events | ||
// after the first message comes back. | ||
misty.RegisterEvent("Hazard", "HazardNotification", 0, true); | ||
|
||
/* | ||
Note: By default, when the system sends an event message, our skill | ||
passes that message into a callback function with the same name as our | ||
registered event listener, prefixed with an underscore. (In this case, | ||
that's "_Hazard()"). You can customize the name of this callback method | ||
by passing in a different name as an optional argument when you call | ||
the misty.RegisterEvent() method. | ||
*/ | ||
|
||
// Defines how Misty should respond to Hazard event messages. The | ||
// data from each Hazard event is passed into this callback function. | ||
function _Hazard(data) { | ||
|
||
var safe = false; | ||
|
||
// Prints a debug message with the contents of the | ||
// AdditionalResults array. Then, assigns this array to a variable. | ||
misty.Debug(JSON.stringify(data.AdditionalResults)); | ||
const dataIn = data.AdditionalResults; | ||
|
||
// Loops through the dataIn array to check which DriveStopped | ||
// hazards are in a hazards state, and stores the SensorNames of | ||
// those hazards in the triggers array. | ||
var triggers = []; | ||
dataIn.forEach(sensor => { | ||
sensor.forEach(sensorData => { | ||
sensorData.InHazard ? triggers.push(sensorData.SensorName) : {} | ||
}); | ||
}); | ||
// Checks the length of the triggers array. If not empty, prints | ||
// a debug message with the contents of the triggers array. If | ||
// empty, the DriveStopped hazards are in a hazards state, and we | ||
// toggle safe to be true. | ||
triggers.length ? misty.Debug(triggers) : safe = true; | ||
|
||
// If safe is true, then we change the LED to white (it's safe for | ||
// Misty to drive). If safe is still false, we change the LED to | ||
// red (it's not safe to drive). | ||
safe ? misty.ChangeLED(255, 255, 255) : misty.ChangeLED(255, 0, 0); | ||
} | ||
|
||
|
||
|
||
/* | ||
Tip: You can extend this sample to write a skill that has Misty | ||
autonomously roam her environment, programmatically changing direction | ||
each time she enters a hazard state. Because the DriveStopped hazards | ||
are each associated with a particular "region", you can check the | ||
contents of the triggers array to find out which "regions" are unsafe, | ||
and use Misty's driving commands to program her to back up and choose | ||
a new direction. | ||
*/ |
19 changes: 19 additions & 0 deletions
19
JavaScript SDK Code Samples/hazardNotification/hazardNotification.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
{ | ||
"Name": "hazardNotification", | ||
"UniqueId": "67478ed7-3ad1-486b-8f91-355b86d823ca", | ||
"Description": "Misty changes the color of her LED when she's in a hazard state", | ||
"StartupRules": [ | ||
"Manual", | ||
"Robot" | ||
], | ||
"Language": "javascript", | ||
"BroadcastMode": "debug", | ||
"TimeoutInSeconds": 36000, | ||
"CleanupOnCancel": false, | ||
"WriteToLog": false, | ||
"Parameters": { | ||
"int": 10, | ||
"double": 20.5, | ||
"string": "twenty" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# propertyTest | ||
|
||
This sample shows how to code Misty to apply a property test to event listeners, so that Misty ignores messages from an event that do not meet the requirements you define in your code. | ||
|
||
You can use property tests any time you register a new event listener. In this sample, we filter messages from Misty's time-of-flight sensors, so that our event callback only triggers when the front-center sensor detects an obstacle closer than 15 cm. This event listener ignores messages that come from the other time-of-flight sensors, and it doesn't pass along data from the front center sensor unless its | ||
distance reading is less than or equal to 15 cm. | ||
|
||
You can run this code on your robot by uploading the files from this folder to Misty via the Skill Runner web tool. Alternately, refer to this code sample (or copy and paste it into your own skills) when working on similar functionality. | ||
|
||
**Note:** To test this code, start the skill without any obstacles in front of Misty's front center time-of-flight sensor. Then, slowly bring your hand closer to the front of Misty's base. When the front center sensor detects that your hand is closer than 0.15 cm, Misty's LED should change to red. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
/********************************************************************** | ||
Copyright 2019 Misty Robotics, Inc. | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
imitations under the License. | ||
**********************************************************************/ | ||
|
||
/********************************************************************** | ||
propertyTest | ||
This sample shows how to code Misty to apply a property test to event | ||
listeners, so that Misty ignores messages from an event that do not | ||
meet the requirements you define in your code. | ||
You can use property tests any time you register a new event listener. | ||
In this sample, we filter messages from Misty's time-of-flight sensors, | ||
so that our event callback only triggers when the front-center sensor | ||
detects an obstacle closer than 15 cm. This event listener ignores | ||
messages that come from the other time-of-flight sensors, and it | ||
doesn't pass along data from the front center sensor unless its | ||
distance reading is less than or equal to 15 cm. | ||
To test this code, start the skill without any obstacles in front of | ||
Misty's front center time-of-flight sensor. Then, slowly bring your | ||
hand closer to the front of Misty's base. When the front center sensor | ||
detects that your hand is closer than 0.15 cm, Misty's LED should | ||
change to red. | ||
We've left a lot of comments in this code for developers new to the | ||
Misty platform. If you already know your way around, feel free to | ||
ignore them! | ||
**********************************************************************/ | ||
|
||
// Creates a property test for FrontTOF event messages to check | ||
// whether the value of the SensorPosition property is equal to | ||
// "Center". The FrontTOF listener ignores any TimeOfFlight messages | ||
// where this is not the case. | ||
|
||
misty.AddPropertyTest("FrontTOF", "SensorPosition", "==", "Center", "string"); | ||
|
||
/* | ||
Note: All possible time-of-flight sensor positions are: | ||
* Center | ||
* Left | ||
* Right | ||
* Back | ||
* DownFrontRight | ||
* DownFrontLeft | ||
* DownBackRight | ||
* DownBackLeft | ||
*/ | ||
|
||
// Creates a property test for FrontTOF event messages to check | ||
// whether the value of the DistanceInMeters property is less than or | ||
// equal to 0.15. The FrontTOF listener ignores any TimeOfFlight | ||
// messages where this is not the case. | ||
misty.AddPropertyTest("FrontTOF", "DistanceInMeters", "<=", 0.15, "double"); | ||
|
||
// Registers a new event listener for TimeOfFlight events. (We call | ||
// this event listener FrontTOF, but you can use any name you like.) | ||
// Our FrontTOF event listener has a debounce of 0 ms, and we set the | ||
// fourth argument (keepAlive) to false, which tells the system to stop | ||
// listening for FrontTOF events after the first message comes back. | ||
misty.RegisterEvent("FrontTOF", "TimeOfFlight", 0, false); | ||
|
||
// Changes LED to purple | ||
misty.ChangeLED(144, 0, 230); | ||
|
||
// Defines how Misty should respond to FrontTOF events. Remember, this | ||
// callback function ONLY triggers when Misty detects on obstacle | ||
// closer than 0.15 cm with her front center sensor. | ||
function _FrontTOF(data) { | ||
// Prints the value of the DistanceInMeters property from our front | ||
// center time-of-flight sensor. Event messages that pass a | ||
// property test are pushed to the PropertyTestResults array in the | ||
// data that gets passed into our event callback. | ||
misty.Debug(data.PropertyTestResults[0].PropertyParent.DistanceInMeters); | ||
misty.ChangeLED(255, 0, 0); // Changes LED to red | ||
misty.PlayAudio("s_Amazement.wav"); // Plays an amazed sound | ||
} |
Oops, something went wrong.