Skip to content

v0.7.2 — A2C2 catastrophic regression eliminated

Choose a tag to compare

View all tags
@rylinjames rylinjames released this 29 Apr 11:16
· 350 commits to main since this release
v0.7.2
343db33

v0.7.2 — A2C2 catastrophic regression eliminated

The A2C2 correction head in v0.5.0+ shipped as partial after a 2026-04-26 N=50 LIBERO experiment showed catastrophic regression: ON tasks 0+1 = 0/10 vs OFF baseline 8/10. Root cause: unbounded output layer + MSE loss let the head learn magnitude-7 corrections that derailed the policy.

v0.7.2 fixes this. After the 2026-04-29 fix:

  • N=10 LIBERO ON: 8/10 (80%) — exactly matches OFF baseline
  • Chunk correction magnitude: 0.755 (was ~7.0, 9× smaller)
  • Catastrophic 0/10 eliminated

Three architectural changes

  1. Bounded head output (kernels/a2c2_correction.py): tanh(z/3.0) * 3.0 saturation. Output bounded to ±3.0 in normalized action space.
  2. Huber loss + L2 magnitude penalty (correction/a2c2_training.py): MSE → Huber(δ=0.1) + λ=0.01 magnitude penalty. Caps gradient on outliers, discourages large outputs at training time.
  3. Runtime safety net (runtime/a2c2_hook.py): refuses corrections with chunk magnitude > sqrt(chunk_size) * 3.0, falls back to base actions, emits magnitude_safety_skip metric.

Validated

  • Local sanity: 1000 random extreme inputs, max output L-inf = 0.365 << 3.0
  • Modal A100 N=10 LIBERO at --inject-latency-ms 100 (paper-fidelity setup): 8/10 = OFF parity
  • 67/67 a2c2 tests pass + 2 new bounded-output tests
  • Existing checkpoints (including the catastrophic one) load fine — output is just clamped at inference

Notes

  • This is "no regression" not "perf claim". Phase 1 head is non-destructive but conservative. The L2 penalty is aggressive (λ=0.01); corrections plateau at magnitude ~0.755 chunk-wide. Phase 2 (multi-latency training + relaxed λ=0.001) is in flight to produce actual ON > OFF positive delta.
  • --a2c2-checkpoint is still opt-in. Customers who avoided A2C2 due to the catastrophic finding can now safely enable it.
  • Modal cost for the Phase 1 work: $3 ($1.50 retrain + $1.50 LIBERO smoke).

Cross-references

  • Research synthesis: 4-lens parallel investigation in reflex_context/features/01_serve/subfeatures/_rtc_a2c2/a2c2-correction/a2c2-correction_research_revisit.md
  • Validation experiment: reflex_context/03_experiments/2026-04-29-a2c2-phase1-libero-smoke-modal.md
  • Catastrophic baseline: reflex_context/03_experiments/2026-04-26-a2c2-n50-inject-latency-ON-catastrophic.md
Assets 2
Loading