Talk to me

RAPP communication module

Ability to interact with the robot is one of the most important when creating human-oriented applications. Rapp communication module is implemented as rapp::robot::communication class.

Guessing game

To show some robot abilities, let's create a guessing game. Scenario is as follows:

• robot asks human to think of a number between 1 and 100
• human confirms that he or she is ready
• robot asks few questions that can be answered YES or NO
• last question is (or should be) whether robot guessed the number

Coding time!

First of all, you should [create a new application](Create your first RApp). When it's ready, new header must be added and proper communication object should be created. In header section add following line:

#include <rapp-robots-api/communication/communication.hpp>

and after info object add new one:

int main(int argc, char * argv[]) {
    rapp::robot::info info(argc, argv);
    // add following line
    rapp::robot::communication comm(argc, argv);
    // ...

Next step is to ask first question. There is convenient method text_to_speech, that takes text to be said and language as an argument. As English is set as default, we'll stick with it and just pass text:

    comm.text_to_speech("Think of a number between 1 and 100. When ready, please say the word start.");

Listening to answer from human player is possible using word_spotting method. As an argument vector of words should be passed. Method will return one of those words after successful recognition.

    // create list of words for recognition
    std::vector<std::string> words;
    // add two words
    words.push_back("start");
    words.push_back("stop");
    std::string result;


    // start word spotting until proper word will be recognized
    do {
        result = comm.word_spotting(words);

        // if player wants to end the game
        if (result == "stop") {
            comm.text_to_speech("I hope we will play again soon!");
            return 0;
        }

        // notify human 
        if (result != "start") {
            comm.text_to_speech("Say start when ready or stop if you want to finish game.");
        }
    } while (result != "start");

Now, when we know, that player is ready, it's time to start asking questions. As it is only simple tutorial, we will use divide-and-conquer strategy.

    // create variables for lower and upper limit
    int lower = 1;
    int upper = 100;

    // prepare new list of words
    words.clear();
    words.push_back("yes");
    words.push_back("no");
    words.push_back("stop");

    // loop until both limits are equal
    while(lower < upper) {
        // calculate middle number
        int middle = lower + (upper - lower) / 2;
        // prepare question
        std::string question;
        question += "Is your number greater than " + std::to_string(middle) + "?";
        // ask the question
        comm.text_to_speech(question);
        // wait for an answer
        do {
            result = comm.word_spotting(words);

            // if player wants to end the game
            if (result == "stop") {
                comm.text_to_speech("I hope we will play again soon!");
                return 0;
            }

            // notify human 
            if (result != "yes" && result != "no") {
                comm.text_to_speech("Yes or no!");
            }
        } while (result != "yes" && result != "no");

        // if the number is greater
        if (result == "yes") {
            lower = middle + 1;
        }
        // if the number is lower or equal
        if (result == "no") {
            upper = middle;
        }
    }

At this moment, when loop is finished we can be sure, that we know the answer! Let's shout it out:

    std::string text = "I know the answer! ";
    text += "You thought about " + std::to_string(lower);
    comm.text_to_speech(text);

It will be nice, if the robot will play some happy music at the end. To add the music file, few things should be done. First of all, file should be added to the source directory. Let's add any music file and name it win.ogg. Next, you should extend create_rapp command invocation in CMakeLists.txt:

create_rapp(
    NAME ${PROJECT_NAME}
    SRCS main.cpp
    FILES win.ogg
)

This will instruct build system to copy win.ogg to share subdirectory of your application.

Next, you have to get the path to your file using rapp::robot::info class (method get_path), and pass it to play_audio:

    comm.play_audio(info.get_path("share/talk_to_me/cheer.wav"));

And that's all. More methods for communication (like sound recording, silence detection etc.) can be found in rapp::robot::communication docs.