SystemOverview

The system overview of the Handyman is described below.

System Configuration

The system configuration for this competitive challenge is outlined below.

The Windows computer runs the Handyman program. The Handyman program has been created in Unity.
The Ubuntu computer runs the rosbridge server, the SIGVerse rosbridge server, and the robot controller created by the competition participants.

The Handyman program and robot controller communicate through the basic rosbridge server, but communication with a large amount of data (sensor data, etc.) is transmitted through the SIGVerse rosbridge server.

In the Handyman program, robots move in accordance with the instructions from the robot controller when the human avatar issues commands to the robot. The robot controller sends ROS messages such as Twist and JointTrajectory to the Handyman program to move the robot in the Handyman program.

The Handyman program distributes JointState, TF, sensor information, and other ROS messages at a regular interval to the Robot Controller.

Flow

The flow of the competitive challenge is outlined below.

1. Launch the robot controller, the SIGVerse rosbridge, etc on Ubuntu side.
2. Launch the Handyman program on Windows side.
3. Initialize the position and direction of the robot and object to grasp.
4. The avatar sends the “Are_you_ready?” message to the robot.
   And “Environment” message is sent at the same time.
5. The robot sends the “I_am_ready” message to the avatar.
6. The avatar sends “Instruction” message to the robot.
   (e.g.: Go to the XXXX, grasp the YYYY and bring it here.)
7. The robot moves to the room for instruction.
8. The robot sends the “Room_reached” message to the avatar.
9. The avatar checks the first statement.
   If successful, points are awarded and the challenge moves on to the next task.
   If unsuccessful, the task ends.
10. The robot looks for the object.
    If could find the target, try to grasp the object.
    If could not find the target, sends the “Does_not_exist” message to the avatar.
    1. If the indication is correct, points are awarded and the avatar sends “Corrected_instruction” message to the robot and looks for again.
    2. If the indication is not correct, the task ends.
11. The robot grasps the object.
12. The robot sends the “Object_grasped” message to the avatar.
13. The avatar checks the second statement.
    If successful, points are awarded and the challenge moves on to the next task.
    If unsuccessful, the task ends.
14. The robot carries out the instruction after grasping.
15. The robot returns to the position of the avatar once all of the instructions are completed successfully.
16. The robot sends the “Task_finished” message to the avatar.
17. The avatar checks the position of the robot.
    If successful, points are awarded. The task ends.

• If the task ends (successfully or unsuccessfully):
  The avatar sends a “Task_succeeded” (successful task) or “Task_failed” (unsuccessful task) message to the robot to start the next task when participants still have attempts left.
  The avatar sends the “Mission_complete” message to the robot to end the competitive challenge when participants have no attempts left.
• If the time limit has passed:
  The avatar sends the “Task_failed” message to the robot to indicate the task was unsuccessful.
• If users would like to withdraw from the task:
  Users press the Give Up button on the screen to have the avatar send the “Task_failed” message to the robot and indicate the task was unsuccessful.

---  Note:                                                                           ---
---    The method to switch rooms will be announced later.                           ---
---    Avatar will send an environment ID to robot at first.                         ---
---                                                                                  ---
---    Additional situations would be added in which grasping targets do not exist.  ---

Notes

Destination of Grasped Objects

Basically please put the grasped object on a desk or the like.
However, if the destination is a person, move the arm so that the center of the grasped object is within the sphere of the following image.