v2.2.0 - Anti-Suicide Reward Balance
What's Changed
Reward Rebalancing
The robot was optimizing for suicide - dying quickly yielded better rewards than living with trembling penalties.
| Reward | Before | After | Why |
|---|---|---|---|
alive |
5.0 | 10.0 | Living must be profitable |
action_rate |
-0.1 | -0.05 | Allow vibration to balance |
quiet_wrists |
-0.3 | -0.1 | Tolerate arm movement |
orientation |
-2.0 | -1.0 | Less punishment for tilting |
ang_vel_xy |
-0.2 | -0.1 | Less punishment for wobble |
The Math
Before: 1 second alive = +5.0 points, but trembling cost ~7.0 → net loss
After: 1 second alive = +10.0 points, trembling cost ~3.5 → net profit
Now the robot is incentivized to stay alive and figure out balance, even if it shakes.
Full Changelog: v2.1.0...v2.2.0