The trained walker actually walks - #3
Merged
Merged
Conversation
Four places where this port disagreed with upstream flybody. Each was
silent, each had zero error at spawn and grew from there, and together they
meant the published Vaxenburg policy could not walk the body — the demo's
forward motion was the kinematic assist gliding a falling fly.
1. world_zaxis read xmat's third COLUMN where flybody reads the third ROW
(fruitfly.py: MJCFFeature('xmat', root_body)[6:]). Those are transposes,
so pitch and roll were sign-inverted on what is, by input-weight L2 norm,
the 2nd-highest-gain row of the 741x512 input layer. The fly's righting
response therefore added to a tilt instead of opposing it. Confirmed
against the shipped normalization stats: mean(accelerometer) /
mean(world_zaxis) = 981, exactly fruitfly.xml's gravity.
2. The spawn height ignored the floor. floor.xml puts the plane at
pos="0 0 -.15"; flybody's _SPAWN_POS is calibrated against dm_control's
floors.Floor() at z=0. The lowest claw started 0.15 cm up instead of
0.00048, so every episode opened with ~17.5 ms of free fall, all six
claws off the ground and the touch/force observations reading zero,
while the policy was establishing its gait. Spawn is now floor-relative,
with the plane read out of the compiled model.
3. 455 of the 741 observation dims never read the body. Both branches of
the ref block differenced the reference against ITSELF, so a tracking
controller was told "zero error, perfectly aligned" on every tick,
straight through the capsize. ref_displacement is now
R^T_fly·(refPos − flyPos) and ref_root_quat is conj(q_fly) ⊗ refQuat,
both against the live pose, indexed from f=0 so entry 0 is the current
error, over an absolute world-space trajectory the fly can fall behind.
4. flybody's actuator filter dynamics were missing entirely. Upstream sets
dyntype='filter' programmatically in fruitfly.py:_build, so it never
appears in the shipped MJCF and a port that reads the MJCF cannot see
it. model.na was 0: every control tick drove an actuator 100% of the way
to its target instead of 1-exp(-0.002/0.01) = 18%, a plant ~5.5x stiffer
than the one the policy was trained against, and the 59-dim
actuator_activation observable was all zeros. Patched into the MJCF text
in memory at load (the runtime reads fruitfly.xml from the baked bundle,
not from public/), asserted via model.na === 78 so a no-op patch throws.
data.act is now indexed through actuator_actadr, which is only correct
once na > 0.
Measured with tools/walkbench.mjs, kinematic assist explicitly off, 3 reps:
2.004-2.021 cm per simulated second against a 2.0 cm/s command, uprightness
+0.997, never capsizing, actions ~5.9 while upright (flybody's native band
is ~6). Before: 0.15 cm/sim s, on its back in every rep. Control with the
policy disabled: 0.032 cm/sim s — it stands still, so the locomotion comes
from the policy and nothing else.
Target speed goes 1.0 -> 2.0 cm/s to match upstream's inference default.
The old comment attributed the 1.0 tuning to observation-normalization
drift; the real cause was defect 3.
The kinematic assist is also cleared under the policy now, restoring what
a9e24f1 reverted. That revert was right at the time — removing the assist
from a broken walker made the feature visibly fail. It walks now, so the
policy path no longer needs it. The CPG path still does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The RL walker test asserted only that the policy ticked and its actions were finite, with a comment explaining that forward progress was deliberately not asserted because the walker did not walk. Both are now stale. It closes its window on the sim clock and asserts net displacement per simulated second plus end-of-window uprightness, matching the neighbouring walk-gate test. Threshold 0.5 sits 4x below the measured 2.016-2.020 and ~15x above the policy-disabled floor of 0.03, and the pre-fix value of -1.174 cm fails outright — so a regression of any of the four fixes is caught here, and a regression of the actuator filter fails even earlier at the model.na assertion in create(). |action|max is deliberately not gated: __rlActionStats starts accumulating when the toggle is clicked, before the test's reset, so it captures the handover transient and has been logged at 6.3, 65.8 and 166.2 across runs that were otherwise identical. Locomotion was unaffected in all three. tools/walkbench.mjs is the harness the LIMITATIONS numbers come from. It disables the kinematic assist explicitly and samples time-resolved, because every quantity worth knowing is "while the fly is still upright" — a running max over the whole window is dominated by the post-capsize blow-up and made the policy look far worse than it was. Committing it so the published numbers are reproducible rather than asserted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two days ago these files were made honest on the evidence that the trained walker did not walk. That specific fact has changed, so the claim has to change with it. LIMITATIONS.md item 1 was "Trained RL walker does not walk". It now records what is measured — ~2.0 cm per simulated second against a 2.0 cm/s command, upright +0.997, no capsize across 3 reps with the kinematic assist off — and, more usefully, why it used to fail: four port defects, each a silent disagreement with upstream flybody. Section 8's inventory is qualified rather than rewritten. The assist row, and every "assist off = the fly pirouettes in place" measurement, is now explicitly labelled as the CPG path, where it remains true. The row about the assist being live under the policy is stale and says so. Deliberately unchanged, because none of it is affected: - The connectome still does not walk the body. Brain to spine still yields a walking magnitude and a turn bias that scale a hand-written tripod CPG; ~20.3M edges still reach the body as about one scalar per tick. What works now is a published RL policy, not the fly's own brain. - The forward pass has still never been compared against the released SavedModel. tools/verify_walking_policy.py imports no TensorFlow. The walker working is strong circumstantial evidence the architecture guess is right, but it is not that comparison, and the caveat stays. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The published Vaxenburg walking policy now walks the flybody under real physics with the kinematic assist off. Four port defects, each a silent disagreement with upstream flybody, each verified against upstream source.
Measured
tools/walkbench.mjs, kinematic assist explicitly off, 3 reps:Commanded speed was 2.0 cm/s — tracking within 1%. Control, policy disabled, assist off: 0.032 cm/sim s. The fly stands still, so the locomotion comes from the policy and nothing else.
The four defects
world_zaxisread the wrong axis of the rotation matrix — the third column where flybody reads the third row (fruitfly.py:MJCFFeature('xmat', root_body)[6:]). Transposes, so pitch and roll were sign-inverted, on what is by input-weight L2 norm the 2nd-highest-gain row of the input layer. The righting response added to a tilt. Independently confirmed against the shipped normalization stats: mean(accelerometer) ÷ mean(world_zaxis) = 981, exactly the gravity infruitfly.xml.The spawn height ignored the floor.
floor.xmlputs the plane at-0.15;_SPAWN_POSis calibrated against a plane at 0. The lowest claw started 0.15 cm up instead of 0.00048, so every episode opened with ~17.5 ms of free fall, claws off the ground and touch/force observations reading zero, exactly while the policy was establishing its gait.455 of 741 observation dims never read the body. Both branches differenced the reference against itself, so a tracking controller was told "zero error, perfectly aligned" on every tick, right through the capsize. Now
Rᵀ_fly·(refPos − flyPos)andconj(q_fly) ⊗ refQuatagainst the live pose, indexed from f=0, over an absolute world trajectory the fly can fall behind.flybody's actuator filter dynamics were missing. Upstream applies
dyntype='filter'programmatically infruitfly.py:_build, so it never reaches the shipped MJCF and a port reading the MJCF cannot see it.model.nawas 0 — every control tick drove an actuator 100% to target instead of 18%, a plant ~5.5× stiffer than the one the policy was trained on, and the 59-dimactuator_activationobservable was all zeros. Patched into the MJCF text in memory (the runtime reads it from the baked bundle, not frompublic/), asserted viamodel.na === 78so a silent no-op throws.Also
The kinematic assist is cleared under the policy again, restoring what
a9e24f1reverted. That revert was correct then — removing the assist from a broken walker made the feature visibly fail. It walks now, so the policy path no longer needs it. The CPG path still does, and every "assist off = pirouettes in place" measurement in LIMITATIONS §8 remains true for it and is now labelled as such.The RL test gates on real locomotion instead of finiteness. Threshold 0.5 cm/sim s sits 4× below measured and 15× above the policy-disabled floor; the pre-fix value of −1.174 cm fails outright.
Not changed
verify_walking_policy.pyimports no TensorFlow. The walker working is strong circumstantial evidence the architecture guess is right; it is not that comparison.Verification
tscclean · unit 5/5 · full e2e 34/34 on the baseline containing fixes 2 and 4. Two later full runs were contaminated by unrelated load on this machine (11 vitest workers from another session; 15-min load average 10.78; tests taking 6–8 min against a normal 40 s). Those runs failed 3 and 4 tests respectively with zero overlap, and all 7 pass individually on an idle machine — the signature of load, not regression. The RL test itself passed 6 separate single-test runs at 2.016–2.020 cm/sim s.🤖 Generated with Claude Code