This is source code for master's thesis of Huong Do Van in KIST campus under major of HCI - Robotics.
The name of thesis is:
"Efficient Reinforcement Learning in Robotics Manipulation Control by mimicking real world environment with simulator"
Under the instruction of Professor Dr. Lee Woosub and Dr. Kim Soonkyum
From Center Medical Robotics at Korea Institute of Science and Technology
Algorithm implemented: Monte-carlo; Deep Deterministic Policy Gradient
Mean of success and sum of the reward after 3000 episodes of training
The result in simulation:
The result in testbed with the hardware:
- Install OpenCV library for ROS at here
- Install Gazebo 7.0 version simulation test and install packages plugin and control with option of install Pre-Built Debians at here
- Command for install Pre-Built Debians integrate with Gazebo:
$ sudo apt-get install ros-kinetic-gazebo-ros-pkgs ros-kinetic-gazebo-ros-control
- Install Driver package for Camera Intel Realsense D435 for developer at here
- Download latest Realsense SDK 2.0 at here
- Install virtual environment in python3 and OpenAI Baselines by follows the command as below:
$ sudo apt-get install python-pip
$ sudo apt-get install python3-pip
$ sudo pip install virtualenv
$ virtualenv -p python3 [name_virtual_env]
$ source [name_virtual_env]/bin/activate
$ pip install ipython
$ pip install jupyter
$ sudo apt-get install python-dev python-pip python3-dev python3-pip
$ sudo -H pip3 install -U pip numpy
$ pip install numpy scipy matplotlib scikit-image scikit-learn ipython
$ pip install tensorflow
$ git clone https://github.com/openai/baselines.git
$ cd baselines
$ pip install -e .
$ pip install rospkg
$ deactivate
Before execute, add the text in bash file: export GAZEBO_MODEL_PATH=~/catkin_ws/src:{GAZEBO_MODEL_PATH}
Convert from .cpp to .py library of ur_kinematic of Universal Robot under python language with swig tool:
$ cd catkin_ws/src/universal_robot/ur_kinematics/src/
$ ls ( in order to check file ur_kin.i)
$ swig -c++ -python ur_kin.i (noted to check the content and link directory)
$ python setup.py build_ext --inplace
Compile catkin_ws workspace by execute command:
$ catkin_make
$ source devel/setup.bash
Execute command to launch simulation environment of golf platform
$ roslaunch ur_gazebo_test2 ur5_golf_platform.launch
After that execute Reinforcement learning algorithms under file:
q_learning_0612.py For implement Q learning algorithms in golf platform environment.
ddpg_ur5_20190416.py For implement DDPG learning algorithms in golf platform environment.
Or you can create by your own algorithms in this environment
First, you should connect your UR5 with ROS by checking IP address of UR5 at here
Second, change the detail of IP address information inside the file: modman_comm/src/ur_comm.cpp at line 32 with the right name of your UR IP address.
Second, execute connect UR5 with ROS by run command:
$ roslaunch modman_comm ur_comm.launch
Third, execute vision system by the command:
$ roslaunch modman_comm modman_vision.launch
Fourth, execute reinforcement learning algorithm by the policy file at diretory: ur_gazebo_test2/experiment/training_results/policy_best.pkl
Fifth, execute real environment with output of best policy in directory: ur_gazebo_test2/experiment/train.py. Inside this training file, you can add your own policy in line 221. In this case the default policy file is policy_best.pkl.
The result of training in simulation environment will be saved in directory:ur_gazebo_test2/scripts/q_learning_result Maintain: Huong Do Van
Email: vanhuong.robotics@gmail.com