Failures in engine while trying to run Environment

[Le5tes]

Hi,
I'm trying to train an agent to run the Anymal Environment in jupyter notebook with python 3.8.10.

I'm getting this error in the notebook:

RuntimeError                              Traceback (most recent call last)
<ipython-input-58-238be5bf9bbb> in <module>

---> 18     env.step(action)

~/.local/lib/python3.8/site-packages/gym_jiminy/common/envs/env_generic.py in step(self, action)
    745 
    746         # Perform a single simulation step
--> 747         self.simulator.step(self.step_dt)
    748 
    749         # Update shared buffers

~/.local/lib/python3.8/site-packages/jiminy_py/simulator.py in step(self, step_dt)
    404         return_code = self.engine.step(step_dt)
    405         if return_code != jiminy.hresult_t.SUCCESS:
--> 406             raise RuntimeError("Failed to perform the simulation step.")
    407 
    408     def simulate(self,

RuntimeError: Failed to perform the simulation step.

And I'm getting this error printed to the terminal:

In /__w/jiminy/jiminy/core/src/engine/EngineMultiRobot.cc:2242: In jiminy::EngineMultiRobot::step:
error: Too many successive iteration failures. Probably something is going wrong with the physics. Aborting integration.

Simple code to reproduce (you may have to run a few times)

from gym_jiminy.envs import ANYmalJiminyEnv

env = ANYmalJiminyEnv()

env.reset()

for i in range(1000):
    action = env.action_space.sample()
    env.step(action)
  

print('do I get here?')

[duburcqa]

Indeed there is an issue. I think it comes from the contact model that is based by default on collision geometries instead of frames. The former is supposed to be more realistic but untested for now and numerically unstable model. I'm going to rework internally this part of the engine in a near future to handle generic collisions (collisions of all bodies with the ground plus auto-collisions).

For now, here is a PR fixing your issue ! Basically it is sufficient to overwrite the default options, so there is no need to fix jiminy itself. But it you want, I could make a new release of Jiminy that includes these changes.

By the way, I would advice you to use the environment with low-level PID controllers: ANYmalPDControlJiminyEnv

[duburcqa]

I see you did some work based on jiminy here. If you have any question or want me to give you some advice / code review, do not hesitate. The documentation of jiminy is quite poor and providing a comprehensive tutorial would be amazing !

[Le5tes]

Thanks!
Yes, it's an assignment for a Master's. Once we're finished I'm happy to make a tutorial from it. :)

[duburcqa]

So, in the end do you need me to release a new version fixing the issue or you are planning to overwrite the base configuration manually?

[Le5tes]

I overwrote the base configuration in my downloaded version of the package, so no worries!

[duburcqa]

Perfect !

[SDJoeKing]

Hello,
I am on the same team working on the ANYmal RL project. I managed to train it to stand for 20 seconds but it really struggles to move (maybe a few tiny steps forward only). I am thinking to use the ANYmalPDControlJiminyEnv as you recommended, but when I checked the action space for this environment, I got:

Dict(Q:Box([-0.72 -9.42477796 -9.42477796 -0.49 -9.42477796 -9.42477796 -0.72 -9.42477796 -9.42477796 -0.49 -9.42477796 -9.42477796], [0.49 9.42477796 9.42477796 0.72 9.42477796 9.42477796 0.49 9.42477796 9.42477796 0.72 9.42477796 9.42477796], (12,), float64), V:Box(-7.5, 7.5, (12,), float64))

Would it be okay if you could give us a brief explanation as what do they mean? I know they are upper and lower limits, but why separately to Q and V? Sorry for asking as I am not familiar with PID controls.

Thank you so much for your help in advance!

[duburcqa]

There is one PID controller per motor. Each of them are responsible for the sending actual torque based on the error between the current and target state for the motors. More details are available here. In this case, the action is no longer the torque directly, but the target state of each motor. These low-level PID controllers are supposed to update the current torque at a higher rate than the action. More information about why doing this is important can be found in the sub-section about the action space of my preprint available on arxiv.

For simplicity, I advice you to only ignore completely the target velocity and set it to zero systematically, so that the policy only output the desired position of the motors and not the combination of both position and velocity.

[SDJoeKing]

Thank you very much for your swiftly response! I will give that a try. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Alexis DUBURCQ ***@***.***> Sent: Friday, August 19, 2022 2:02:14 PM To: duburcqa/jiminy ***@***.***> Cc: SDJoeKing ***@***.***>; Comment ***@***.***> Subject: Re: [duburcqa/jiminy] Failures in engine while trying to run Environment (Issue #560) There is one PID controller per motor. Each of them are responsible for the sending actual torque based on the error between the current and target state for the motors. More details are available here<https://en.wikipedia.org/wiki/PID_controller>. In this case, the action is no longer the torque directly, but the target state of each motor. These low-level PID controllers are supposed to update the current torque at a higher rate than the action. More information about why doing this is important can be found in the sub-section about the action space of my preprint available on arxiv<https://arxiv.org/pdf/2203.01148.pdf>. For simplicity, I advice you to only ignore completely the target velocity and set it to zero systematically, so that the policy only output the desired position of the motors and not the combination of both position and velocity. — Reply to this email directly, view it on GitHub<#560 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQPEGDQXNAK5T6V2JKLYTKTVZ6AVNANCNFSM562QNMEQ>. You are receiving this because you commented.Message ID: ***@***.***>

[duburcqa]

No problem ! Feel free to ask if you need more details or you have any additional question :)