Timing constraints on the real robots

[luator]

Dear Challenge Participants,

We did not properly document the timing requirements for testing your policies
on the real robots.

Unlike in simulation, the real world robots cannot stop and wait for the policy
to provide the next action. When some torque is applied to the joints, the robot
moves and remains in motion while the policy is computing the next action.
Therefore, there is a limit on how long the policy can take to compute the next
action such that the expected control rate can be met.

For the push task this is ~10 ms, for the lift task (where more accurate control
is needed) it is only ~2 ms.

If your policy takes longer than that, the control loop does not run at the
expected frequency which may lead to poor performance of the policy.

We now added warnings that are printed to user_stderr.txt if the policy
repeatedly takes more time than expected. If you get this warning frequently in
your output, it most likely means that your policy is too slow, in which case
you may want to optimise it for faster execution.

We also added this information to the software documentation.

In the course of adding these warnings, we also found a bug which resulted in
episodes taking more steps than intended if the policy was too slow. This is now
fixed, so the number of environment steps should always be the same.

We apologize that we did not properly document these time constraints earlier
and hope that it does not cause too much inconvenience!

The Challenge Organisers

[kHan0809]

Is there a limitation that the actuator does not work if there is a time delay?

My policy is obviously giving the non-zero output and It's not stuck in any obstacle. But the robot doesn't move in some episodes .

The only doubt is that I have often received the following warnings.

"WARNING:rrc_2022_datasets.SimTriFingerCubeEnv:Control loop got delayed by 1 ms. The action will be applied for a shorter time to catch up. Please check if your policy is fast enough (max. computation time should be <3 ms)."

and What is the average of the computation time of the policy which used for collecting your data?

Thank you!

[nicoguertler]

Hi, the actuators should still work even if the policy is too slow. A 1 ms delay doesn't sound that bad, it shouldn't completely break the policy.

The policy we used for data collection usually needs less than 1 ms but occasionally exceeds the budget for a couple of ms. Note that if policy evaluation finishes before the time budget is used up, the code waits for the remaining time to ensure that the control frequency is consistent.

When training in simulation and deploying on the real system we sometimes got policies that would not move even though the torques oscillated strongly. Maybe this happens in your case as well. If you send your username and the ID of the job to trifinger.mpi@gmail.com then I can have a look at the data and see if something is off.

[kHan0809]

I sent two email to you about this issue.

Thank you!