Upper body control seems unstable

[zhan345]

Upper body control from loco_manip policy seems jittery and unstable. Could this be due to AMASS data being processed through PHC whereas we are passing IK data to ref_upper_dof_pos? I'm theorizing that the IK-proposed joint angles are likely out of distribution from PHC-proposed joint angles. Wondering if this has been tested or if even increasing number of reference motion frames help (I see in G1 training config file that CMU and all of AMASS dataset have been tested as well, but final version seems to just use ACCAD).

[hang0610]

Any delay warning from rate_limiter?

[zhan345]

No, it just becomes jittery. Happens both in sim + real, so I'm supposing that the IK just may be proposing sets of joint angles that are out of distribution. Do you have another intuition?

[carlosdp]

@zhan345 I'm experiencing an issue where the G1 slaps it's hands together repeatedly, non-stop when the FALCON full policy is enabled. Is this similar to what you are seeing? Would love too compare notes carlos@carlosdp.xyz

[carlosdp]

This is what I am experiencing when running FALCON, this is at 0.2 KD scaling, turning it up makes it worse.

g1-freaking-out.mp4

[zhan345]

Are you running any actions on the upper body? Mine does not do this, but maybe your actions are effecting things. Also curious what this looks like with damping set to higher since that seems like it should fix.

[zhan345]

Still waiting for author response

[carlosdp]

@zhan345 the video is with full FALCON, including upper body. I was able to get the lower body policy-only working stable (but only by starting at kp_level = 0.2 and slowly scaling it up to 1.0, otherwise it freaks out uncontrollably.

In sim, it does seem bringing it up to kp_level = 1.0 (or starting from there) does stabilize it, however in real, we can't start it at 1.0 (because it really freaks out and just trips itself on the gantry immediately), and we can't bring it up slowly (because these oscilliations get so intense, it just ends up tripping itself and losing balance, even when we have someone holding him in position).

It seems to me there's either some bug in the policy training for the published checkpoints, that differs from the examples in the paper (is this the actual checkpoint run in those examples?), or there's some bug introduced in the IK resolution perhaps for the arm waypoints? I've moved on to trying to train from scratch, for now.

For what it's worth, these oscillations replicate in-sim when you start FALCON from low kp_level. In sim, it just isn't real so you can bring it up to 1.0 and it (usually, but not always!) stabilizes. In real, the robot dangerously flails and you have to stop the policy. I feel like there is something missing between the examples in the paper, and the deployment published here. It's very onerous atm to just start the policy up from zero on the G1, as is. Whereas the built-in Unitree stand/walk policy, for example, is safe to start up and stabilizes immediately. I would expect that, more or less, at a baseline.

[zhan345]

Yes I'm not sure. I have begun having similar issues as well especially when walking and the upper body begins shaking in response. Have you been able to fix this? I don't have this issue when standing still, interestingly enough.

[carlosdp]

It's definitely something wrong with our G1. We were able to replicate it well at BAIR https://x.com/cixliv/status/1956124926030831697?s=46&t=2wUdzHKx6edXp0AY4IOI3g

Starting it at high KP is very unstable still, but once you scale it up from low KP, it's pretty stable! Works very well, handles force on arms even extended, recovers from pushes well.

[hang0610]

@zhan345 Usually the shaking issue is due to the action delay. Either check your firewall setting (sudo disable ufw) or adding more upper body action delay during training.

@carlosdp Glad to see it works! But it would be great if you can ack us while posting a policy from us : )

[carlosdp]

@zhan345 Usually the shaking issue is due to the action delay. Either check you firewall setting (sudo disable ufw) or adding more upper body action delay during training.

@carlosdp Glad to see it works! But it would be great if you can ack us while posting a policy from us : )

Yea sorry about that, he didn't know about the importance of citation 😅 I'll ensure it doesn't happen again!

[lllzheng]

@zhan345抖动问题通常是由于动作延迟造成的。请检查防火墙设置（sudo disable ufw）或在训练期间增加上肢动作延迟。

@carlosdp很高兴看到它有效！不过，如果您能在发布我们的政策时回复我们，那就太好了 : )

May I ask if the action delay is related to the sdk type? Using unitree_sdk_python and loco_manip policies, it's quite difficult to stand firm. Is it necessary to switch to unitree_sdk_c++

[carlosdp]

@zhan345 do the wrists on your robot seem weakened by any chance? #19