This repository is an implementation of the NeuroMechFly model in an RL environment using NVIDIA's physics simulation environmentIsaac Gym. This implementation is a fork of Shikun Liu's minimal-isaac-gym repository adapted and improved upon coupled with the work of Ramdya Lab at epfl which released a very accurate model of the drosofilia Melanogaster called NeuroMechFly
Disclaimer:This implementation is still not perfectly accurate regarding the coherance of the units used and so the results found using it a promising but not perfect. I highly recomend to read my project report, especially the part 4.5 talking about the hyperparameters where I explain a big parts of the problems I encountered and how could one resolve them.
Note: The current implementation supports only one RL algorithms, PPO continuous. The interfacing between the environment and the NN is really simple as I am not using any third-party RL frameworks like the examples of IsaacGymEnvs do. One could easely change it at will and is encourage to do so as I think my NN is not strong enough to learn such a complex environment. But again I recommend to read my project report to have more info on the subject.
Simply run python trainer.py
--sim_device type= str default="cuda:0"
Decide on which device you want to simulate your environment and the NN, the other possible values are ["cpu", "cuda:1", "cuda:2", ...]
--compute_device_id type= int default= 0
Index of CUDA-enabled GPU to be used for simulation.
--graphics_device_id type= int default= 0
Index of GPU to be used for rendering
--num_envs type= int default= 1000
Number of environments in the simulation
--headless type= bool default= False
Choose to not display a viewer by setting True.
--save_path type= str default= None
If not None, the weights of the NN will be saved to this path. This path should be Absolute
--save_freq type= int default= 100
Frequency at which it will save, normally it should be the number of optimisation steps between each save but it doesn't for some reson I know but don't have the time to change.
--load_path type= str default= None
If not None, the weights of the NN will be loaded from this path at the start of the simulation. This path is Absolute
--record_dir_name type= str default= None
If not None, screeshots of the simulation will be taken and placed in an floder ./recording/record_dir_name. The base name "recording" can be changed in the code.
--time_steps_per_recorded_frame type= int default= 2
How many time steps between each screenshot
--testing type= bool default= False
If set to True, the variance of the action will be set to a small value and no training will take place. Use this to test your NN by loading the weights in with the command --load_path described before.
Complex run commands could look something like that:
python3 trainer.py --load True --load_path ~/isaacgym/python/isaacGymEnvs/isaacgymenvs/minimal-isaacgym/fly_bProject/save7_1_23/save1_standing_18dofs_.pth --save_path ~/isaacgym/python/isaacGymEnvs/isaacgymenvs/minimal-isaacgym/fly_bProject/save7_1_23/save2_walk_18dofs_ --num_envs 1000
If you are not in headless mode you can use the following keys:
R The R key will reset all the environment at once
E The E key will quit the simulation and do a last save if --save_path is set
P The P key will print out different informations about the rewards of the environments, that can easily be changed at will in the code
V The V key will toggle on and off the viewer. The simulation is way faster when not rendering