Humanoid-Robot-Active-Vision-DDQN

Table of Contents

1. Introduction
   • Built With
2. Getting Started
   • Prerequisites
3. Usage
4. References
5. Contact
6. Acknowledgments

Introduction

This is the first official Code Release of "Real-time Active Vision for a Humanoid Soccer Robot using Deep Reinforcement Learning" paper.

Also, hereby, a world model method is open-sourced. This world model method was introduced by Meisam Teimouri. It is converted to C++ by me, Soheil Khatibi, in order to reduce the run-time.

Unfortunately, the code, especially the world model section, is too messy. it is to be well documented and cleaned.

Built With

The project is built using Tensorflow, specifically Stable Baselines, and Webots Simulation.

(back to top)

Getting Started

In order to get started with this project, you need to use RoboCup-Humanoid-MRL-Webots-Simulation, which is the simulation of the MRL-HSL Humanoid Robot in a Kid-Size Humanoid Soccer League RoboCup 2019 environment.

Prerequisites

To run this simulated environment, you need to install:

• Webots R2023b. (Instructions)
• Tensorflow (GPU) 1.15
• stable-baselines 2.10 (Link)
• gym 0.15.7

This code was developed and tested with Python 3.6 (using Miniconda) installed on Ubuntu 20.04

(back to top)

Usage

After installing Webots R2023b, You can open this environment as follows:

1. Clone the RoboCup-Humanoid-MRL-Webots-Simulation repository and run it (instructions could be found here):

   git clone https://github.com/SoheilKhatibi/RoboCup-Humanoid-MRL-Webots-Simulation.git
   cd RoboCup-Humanoid-MRL-Webots-Simulation
   /path/to/webots worlds/Rules_2019.wbt

2. In another terminal, clone the repository:

   git clone https://github.com/SoheilKhatibi/Humanoid-Robot-Active-Vision-DDQN.git

3. Build the world model module:

   cd Humanoid-Robot-Active-Vision-DDQN
   cd RobotWorldModel
   make

4. Run the Code:

   cd Humanoid-Robot-Active-Vision-DDQN
   cd DQNController
   python DQNController.py

(back to top)

References

• This ReadMe is created using this useful template, by othneildrew.
• In the Robot World Model section, The lua parser module for C++ is copied from the code of a blog post which unfortunately I can not find to reference here.

(back to top)

Contact

Soheil Khatibi - Soheil Khatibi - soheilkhatibi1377@gmail.com

Project Link: https://github.com/SoheilKhatibi/RoboCup-Humanoid-MRL-Webots-Simulation

(back to top)

Acknowledgments

Special Thanks to:

• Meisam Teimouri for contributing in the project

(back to top)