Remote application for NAO in Python: Detect and track QR tags (normal)

The current tutorial describes a more complicated application, as it will employ both the robot-agnostic RAPP robot API and the RAPP Platform API, aiming to implement a simple detection of a QR code, as well as a reaction from NAO. This application will be executed remotely in a PC.

For this tutorial we will use the following tools:

• A real NAO robot
• the NaoQi Python libraries
• the rapp-robots-api Github repository
• the rapp-api Github repository
• the rapp-platform up and running

Of course the standard prerequisites are a functional installation of Ubuntu 14.04, an editor and a terminal.

##Preparation steps

####NaoQI Python libraries setup Please check here.

####RAPP Robots Python API setup Please check here.

####RAPP Platform API setup

It is assumed that till now you have created the ~/rapp_nao folder where the NaoQi Python libraries and the RAPP Robots API Python libraries exist. The next step is to clone the RAPP Platform API:

cd ~/rapp_nao
git clone https://github.com/rapp-project/rapp-api.git

In order to utilize the RAPP Platform API the PYTHONPATH environmental variable has to be updated:

echo 'export PYTHONPATH=$PYTHONPATH:~/rapp_nao/rapp-api/python' >> ~/.bashrc
source ~/.bashrc

It should be stated that if you have already set-up the RAPP Platform in your PC, the RAPP Platform API has been already installed and the previous step is not necessary.

####RAPP Platform setup

In order to set-up the RAPP Platform you can follow this tutorial. Have in mind that the RAPP Platform installation is quite time consuming.

##Writing the application

Let's create a Python file for our application and give it execution rights:

cd ~/rapp_nao
mkdir rapps && cd rapps
touch qr_app.py
chmod +x qr_app.py

The robotic application follows:

#!/usr/bin/env python

import time, threading
from os.path import expanduser

# Import the RAPP Robot API
from rapp_robot_api import RappRobot
# Import the RAPP Platform API
from RappCloud import RappPlatformAPI

# Create an object in order to call the desired functions
rh = RappRobot()
ch = RappPlatformAPI()

# Enable the NAO motors for the head to move
rh.motion.enableMotors()

def callback():
    print "Callback called!"
    # Capture an image from the NAO cameras
    rh.vision.capturePhoto("/home/nao/qr.jpg", "front", "640x480")
    # Get the photo to the PC
    rh.utilities.moveFileToPC("/home/nao/qr.jpg", expanduser("~") + "/Pictures/qr.jpg")
    # Check if QRs exist
    imageFilepath = expanduser("~") + "/Pictures/qr.jpg"
    res = ch.qrDetection(imageFilepath)
    print "Call to platform finished"

    if len(res['qr_centers']) == 0: # No QR codes were detected
        print "No QR codes were detected"
    else: # One or more QR codes detected
        print "QR code detected"
        qr_center = res['qr_centers'][0]
        qr_message = res['qr_messages'][0]

        # Directions are computed bounded in [-1,1]
        dir_x = (qr_center['x'] - (640.0/2.0)) / (640.0 / 2.0)
        dir_y = (qr_center['y'] - (480.0/2.0)) / (480.0 / 2.0)
        angle_x = -dir_x * 30.0 / 180.0 * 3.1415
        angle_y = dir_y * 23.7 / 180.0 * 3.1415

        # Get NAO head angles
        ans = rh.humanoid_motion.getJointAngles(["HeadYaw", "HeadPitch"]) 
        angles = ans['angles']
        
        # Set NAO angles according to the QR center
        rh.humanoid_motion.setJointAngles(["HeadYaw", "HeadPitch"], \
                [angle_x + angles[0], angle_y + angles[1]], 0.1)

        if callback.qr_found == False:
            rh.audio.speak("I found a QR with message: " + qr_message)
            rh.audio.speak("I will track it")
            callback.qr_found = True

        # Capture an image from the NAO cameras
        rh.vision.capturePhoto("/home/nao/qr.jpg", "front", "640x480")
        # Get the photo to the PC
        rh.utilities.moveFileToPC("/home/nao/qr.jpg", expanduser("~") + "/Pictures/qr.jpg")
    
    threading.Timer(0, callback).start()


callback.qr_found = False
callback()

In order to execute the application the RAPP Platform must be up and running. This can be done by executing the two deployment scripts:

bash ~/rapp_platform/rapp-platform-catkin-ws/src/rapp-platform/rapp_scripts/deploy/deploy_rapp_ros.sh

and

bash ~/rapp_platform/rapp-platform-catkin-ws/src/rapp-platform/rapp_scripts/deploy/deploy_web_services.sh

Finally, you just have to execute python ~/rapp_nao/rapps/qr_app.py to deploy the application.