Skip to content
/ Sim2E Public

Mujoco based Robotic arm simulator with an Esim-based neuromorphic vision sensor simulator, rendered in Unity3D

License

Notifications You must be signed in to change notification settings

Turgibot/Sim2E

Repository files navigation

alt

Sim2E

We introduce our Mujoco based, Interbotix vx300s digital twin with six degrees of freedom (DOF) robotic arm, mounted with a virtual event camera. The robotic arm 3D model is in a configurable virtual lab, rendered in the Unity3D game engine, and the events are generated using the Esim event camera simulator. The system can generate, and record simulated neuromorphic vision data while the robot arm is in motion.

This project utilizes Esim to generate events from a sequence of images. You can find a pdf of the paper here. If you use any of this code, please cite this publication.

@Article{Rebecq18corl,
  author        = {Henri Rebecq and Daniel Gehrig and Davide Scaramuzza},
  title         = {{ESIM}: an Open Event Camera Simulator},
  journal       = {Conf. on Robotics Learning (CoRL)},
  year          = 2018,
  month         = oct
}

Installation

Requirements

  • Ubuntu 18.04 or newer
  • Nvidia GPU with CUDA support
  • Nvidia Cudnn
  • Anaconda

step 0: Create a conda environment

conda create -n sim2e python=3.9
conda activate sim2e

step 1: Install python binding of Esim with GPU support

IMPORTANT: Make sure to stay on the sim2e conda environment and don't switch to the environment in the Esim installation guide.
Instructions can be found in this github page.

step 2: Install Mujoco

Install mujoco and mujoco-py. Instructions can be found in this video here . You can also follow the instructions in this document instead.

step 3: Clone the repository and install dependencies

git clone https://github.com/Turgibot/Sim2E.git
cd Sim2E
pip install -r requirements.txt

Usage

Run the following command to start the simulation

python main.py

About

Mujoco based Robotic arm simulator with an Esim-based neuromorphic vision sensor simulator, rendered in Unity3D

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published