Monocular-based-pick-and-place

This Repository is for the paper: Monocular Camera-based Robotic pick-and-place in Fusion Applications.

Dependence

Our work is depending on the Isaac Gym and rl_games.

Please first intall the Isaac Gym and rl_games according to the introduction on their pages. Then please copy the files in this repository to their corresponding locations strictly according to the file tree structure, replace it if a file with the same name exists.

Training

You can start the training by the command:

cd IsaacGymEnvs/isaacgymenvs/
python train.py task=MyNewTask

Configuring your training by modifying the MyNewTask.yaml and MyNewTaskPPO.yaml files.

Testing

If you want to test your model, you can run the command:

python train.py task=MyNewTask checkpoint= the/path/to/your/model  test=TEST

Our model is available at here.

The network in the code is a bit different from the description in the paper. It's even more miniaturized but the performance is about the same.

Citation

If you feel our work is helpfull during your research, please first cite the work of Isaac Gym and rl_games, and then cite our work, as follows:

@article{yin2023monocular,
  title={Monocular Camera-Based Robotic Pick-and-Place in Fusion Applications},
  author={Yin, Ruochen and Wu, Huapeng and Li, Ming and Cheng, Yong and Song, Yuntao and Handroos, Heikki},
  journal={Applied Sciences},
  volume={13},
  number={7},
  pages={4487},
  year={2023},
  publisher={MDPI}
}