ROCK, PAPER, SCISSORS ROBOT
See it on my portfolio: https://codynichoson.com/rockpaperscissors.html
In this solo project, I set out to create a rock, paper, scissors robotic partner that would allow a user to play in a similar fashion as they would another human. Meeting this requirement meant that some creative computer vision techniques were necessary to facilitate gameplay using natural gestures. The project made use of the Wonik Allegro Robotic Hand, OpenCV computer vision library, and MediaPipe machine learning library in order to create the final result.
- Wonik Allegro Robotic Hand Version 4.0 (http://wiki.wonikrobotics.com/AllegroHandWiki/index.php/Allegro_Hand_v4.0)
- PCAN-USB Adapter (https://www.peak-system.com/PCAN-USB.199.0.html?&L=1)
- Allegro code is written for PCAN devices
- Robot Operating System (ROS)
- MediaPipe Machine Learning Library
- BHand Library (http://wiki.wonikrobotics.com/AllegroHandWiki/index.php/BHand_library)
Software Setup (Some of this is from Wonik Robotics' website)
- Install necessary packages
sudo apt-get install cmake gcc g++ libpopt-dev
- Install MediaPipe
pip install mediapipe
- Download, build, and install PCAN-USB driver for Linux (http://www.peak-system.com/fileadmin/media/linux/index.htm#download)
tar -xzvf peak-linux-driver-x.x.tar.gz cd peak-linux-driver-x.x make NET=NO sudo make install sudo modprobe pcan
- Create a
catkin_ws/srcfor the necessary packages, then
mkdir catkin_ws/src cd catkin_ws/src
- Download my fork of the
git clone email@example.com:codynichoson/allegro_hand_ros_v4.git
- Download this package into
git clone firstname.lastname@example.org:codynichoson/RockPaperScissorsRobot.git
- Return to the base of the workspace, make, and source:
cd .. catkin_make source devel/setup.bash
- Plug the power/CAN connector into the Allegro hand.
- Plug the USB/PEAK-CAN connector into your computer.
- Turn the hand on using the toggle switch on the side. You should hear an audible beep and see the red light flash.
- To play rock, paper, scissors, run:
roslaunch rockpaperscissors rockpaperscissors.launch
- A video window should open using your default camera.
- Hold your hand clearly in frame, and move it up and down to "countdown" in order to begin the game. You should choose a rock, paper, scissors pose when the timer hits '4'. The Allegro hand will also choose a pose, and the score will be updated accordingly.
This node opens the default camera on your computer and uses the MediaPipe machine learning library to identify hand gestures in frame. These gestures are then used to play and score the game of rock, paper, scissors.
This node takes in gesture commands from the perception node after the game begins and sets the Allegro hand to a specific configuration of joint states.