Ford: anti-weave smoothing of the angle-mode command path (opt-in strength dial)#163
Open
ghbarker wants to merge 17 commits into
Open
Ford: anti-weave smoothing of the angle-mode command path (opt-in strength dial)#163ghbarker wants to merge 17 commits into
ghbarker wants to merge 17 commits into
Conversation
FordAngleAutoCal / FordAngleAutoCalState params (persistent), ControllerStateBP fields @54-56, publisher wiring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…glue + tests) Two-anchor gain estimator with steady-state gating, driven from the angle lateral strategy in the card process; factors applied live, state persisted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the one-shot converge/verify-rounds design (whose evidence died at ignition-off — a normal drive rarely satisfied both anchors, so nothing ever visibly happened) with the automated version of the manual tuning method: - apex evidence: command/measured curvature extrema matched across the actuator lag, amplitude ratio = gain (the 'tops and bottoms of the graphs'), median-of-3 committed; steady-state evidence kept alongside - quality layer with per-cause rejection counters: bump-flick blanking (with wheel-speed corroboration), rough-road RMS on the high-passed residual, lat-accel margin down-weighting, longitudinal-accel gate, left/right divergence guard for crowned/banked roads - live nudging: factors step toward the fit (0.02 max, 20s apart, 0.10/drive cap) and are written to the params — the lateral tuning menu shows them move; per-sample applied-gain bookkeeping keeps the estimate invariant to the nudge trajectory, so the loop backs off on overshoot - evidence persists across drives: estimator sufficient statistics serialize into FordAngleAutoCalState (JSON) every 30s, restored on arming; legacy 'done' states stay honored as locked - lock when there is nothing left to adjust for 5 min of driving; user edits win instantly (soft evidence reset); toggle-off still clears everything Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tlier gate Replayed against route 00000006 (2.7h Mach-E, known-good result 1.02/1.15): - outlier-vs-fit gate removed: path-dependent (early evidence anchors the fit, later contradicting evidence gets rejected) — it alone distorted the fit to 0.95/1.22. Robustness stays with the quality layer, ratio bounds, staging cancellation and apex median-of-3. - accel 1.0->2.0 m/s^2 (was rejecting 8587 frames of ordinary braking into corners), spike 0.008->0.02 (pinion noise floor), rough RMS 0.0006->0.0015, ws jump 0.3->0.6 - decay TAU 30min->2h and lock weight 120->60s so a normally-driven car's evidence equilibrium (~90-165s/anchor) can actually reach lock - peak weight 3.0->1.5s: apex evidence supplements steady evidence instead of dominating (transient undershoot pulled the low anchor ~0.02 high) Virtual closed-loop replay: drive 1 ends applied 1.04/1.08; drive 2 (state persisted) converges to 1.02/1.14, target 1.02/1.15 = the reference, and locks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…elf-reporting errors
Field forensics from the first bp7 install (routes 00000002/00000003 on 2026-07-21):
params showed FordAngleAutoCal=1 + saved evidence at boot while every logged drive
published disarmed — and the logs couldn't say why, because telemetry re-read the
params instead of reporting the controller, and every failure path was a silent except.
- nudge writes sent f'{x:.2f}' STRINGS to FLOAT-typed params: TypeError, swallowed,
nudges silently never landed. Now writes python floats; the mock Params in tests
type-checks like the real params_pyx so this class of bug fails in CI.
- bmsAngleAutoCalibrate/bmsAngleAutoCalState now publish the controller's live state
(armed/evidence/nudges, off, locked, or an error string), never a param re-read.
- every previously-silent failure (param tick, nudge write, state save) now parks a
short 'error: ...' breadcrumb in FordAngleAutoCalState, visible in qlogs/initData.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the muted auto-cal telemetry on EVERY build to date: the publisher set bmsAngleAutoCalibrate/bmsAngleAutoCalState/angleSaturated on the python struct, but convert_to_capnp only carries DECLARED dataclass fields — undeclared setattr is silently dropped and the capnp defaults (False/'') got logged. That is why the device always appeared disarmed in logs while params (and, tonight, tracking physics at 1.000/0.993 median ratio with the nudged 1.04/1.16 factors) proved the calibrator was running the whole time. angleSaturated was equally affected: logged always-False since the field's introduction, so offline analysis never actually excluded saturated frames. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
P1: human_turn/stall_blip parameters removed from SteadyStateGate.update and AutoCalPipeline.update — provably dead False at the onboard call site (both branches early-return after idling the pipeline). Gate gains reset() replacing the seven-positional-False idle call. Offline analyzer now idles on those frames, matching onboard semantics exactly. P2: the stall-blip branch now clears bp_angle_saturated and idles the pipeline before returning, like the disengage and human-turn branches — a blip can no longer leak a stale gate baseline, straddled apex windows, or surviving staged samples into the evidence stream. P3: errors write to their own FordAngleAutoCalError param; a transient write failure before ignition-off can no longer replace (and erase) the serialized evidence in FordAngleAutoCalState. Guard from route 0000000b (2026-07-22): the low anchor holds ~5x less evidence than the high one and round-tripped 0.98->1.06->1.00 in one drive — felt as turn-in overshoot via the 1.30x curve branch. MAX_DRIVE_DELTA_LOW = 0.04 bounds one drive's low-side movement to two steps; replaying the same drive now walks 0.96->0.98->1.00 monotonically to the identical lock (1.00/1.14). 36 tests updated and passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ate + steady-window drift bound The evidence design was delay-insensitive by construction (steady windows and apex amplitude ratios), but two exposures remained: 1) No measurement-chain warmup gate: kappa_meas flows through the vehicle model with liveParameters offsets from the same locationd stack that estimates the actuation delay. Evidence now collects only while liveDelay.status is 'estimated'; before that the pipeline idles (staged samples and peak windows must not straddle the unestimated period). Offline analyzer mirrors the gate from the logged liveDelay stream. 2) Slow ramps inside the per-frame rate bound carry a lag bias: meas(t) answers cmd(t - tau), so a drifting command biases the same-frame ratio by up to tau*rate/kappa — tens of percent at the MIN_KAPPA floor at the ~0.42s delays liveDelay has measured. New STEADY_DRIFT_FRAC=0.10 bounds total command drift across the steady window, capping the error at a few percent worst case regardless of the actual delay, sign-symmetric over entries/exits. Route 0000000b replay with both gates: same qualitative outcome (low walks 0.96->0.98 monotonically, high untouched at 1.14, locks), with ~10-15% less evidence — the warmup minutes and ramp samples it was designed to drop. 37 tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…alibrator consumes (review 5.2) GAIN_CAN / GAIN_CANFD_BOF / GAIN_CANFD_SUV, the per-platform CAR sets, platform_gains(), and the V_LOW / V_HIGH / LOW_ANCHOR_BASE anchors now live in values_ext.py. Kills the inverted dependency (strategy re-importing its own gain model out of the calibrator through a noqa'd second import block) and the inline duplicate of platform_gains() in update_angle_params. The strategy's gain interp now uses the named anchors instead of hardcoded 13.5/26.82/1.30 — one source for the numbers the calibrator's fit is expressed against. Pure refactor: 37 tests pass, route 0000000b analyzer replay byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… strategy (review P4) New angle_autocal_controller.py owns everything between the pure estimator and the car: params handle, arm/disarm from the toggle, JSON evidence persistence + save cadence, user-edit debounce, nudge writes, the liveDelay warmup gate, error reporting, and the ground-truth telemetry status. FordLateralAngleExt shrinks to three calls — poll_params() at its 1 Hz cadence, feed() per lateral frame, idle() on inactive branches — plus two read-only properties (autocal_enabled, bp_autocal_status) preserving the bp_card_publisher surface. feed() returns a nudged factor pair and the strategy adopts it, so the write path to the live steering values stays in exactly one place. Removes from the strategy: 9 instance attributes, 3 _autocal_* methods, 2 module-level helpers, the module json dependency, and both inline lifecycle blocks. The estimator (angle_autocal.py) stays pure math, untouched. Glue tests target the controller surface. 37 tests pass on Windows; the TestOnboardGlue suite runs in the WSL gate. Route 0000000b analyzer replay byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Frame dataclass (frozen, no defaults): AutoCalPipeline.update() and AutoCalController.feed() take one Frame instead of a 12-parameter clump the onboard hook and offline analyzer had to keep in positional agreement; defaults on physical signals (a_ego, saturated) were a silent-wrong-answer risk and are gone from the production API. - Admission predicate computed once: SteadyStateGate exposes frame_clear (and grip_this_frame); the pipeline's apex-path frame_ok reads it back instead of maintaining a hand-synced copy of the grip/cooldown/flag logic. - PeakMatcher ring buffer entries are a NamedTuple (_PeakFrame) — no more magic f[0]/f[1]/f[4] indices or unpack-repack in poison_recent(). - User-edit detection compares floats with EDIT_TOL (half the 0.01 menu granularity) instead of round-tripping through %.2f strings; the two-tick debounce stays (independently justified by async put lag). - bp_card_publisher: the param-snapshot copies of bmsAngleAutoCalibrate/State are gone — they were dead (overwritten from the live controller every publish) and a second source of truth by construction. - safety.c debug accessors: the reported idx <= COUNT off-by-one is a FALSE POSITIVE — the table is sized COUNT+1 with slot 0 as the sentinel and real rows 1..COUNT (index rides bits 1-4 of the safety param; ford.h bounds-checks index == 0 || index > COUNT the same way). Verified by tightening the check and watching test_ford's geometry test fail on the last row. Documented at the accessors so it doesn't get 'fixed' again. (gate.reset() from this list already shipped with the P1 fix.) 37 tests pass; route 0000000b analyzer replay byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plain-language guide for drivers: what the calibrator does, enabling it on both device layouts + sunnylink, what to expect and how long it takes (honest numbers from the on-road lock drive), manual +/- interaction, locking and recalibration, hard behavior bounds, troubleshooting, and the offline analyzer for the curious. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The P4 port left Frame construction (and its wheel-speed/torque/accel reads) running unconditionally every lateral frame — including for the majority of drivers who never enable the calibrator, and controller.feed() then discarded it. Caught by the closed-loop smoothing rig downstream, whose minimal CarState mock had no reason to carry wheelSpeeds for a disabled feature. Gated on autocal_ctl.enabled: disabled cost is one attribute check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same title/description/angle-mode gating as the on-device toggles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…moothing + strength) Removes the rhythmic left-right centering motion some angle-mode Fords show on straight roads. Log analysis traced it to a closed loop: curvature noise crosses the gain-schedule interp's 0.0003-wide transition band, the crossings modulate loop gain 1.0<->1.2 every cycle (0.23 Hz limit cycle measured), and the PSCM integrates near-zero LSB toggling (962/min) into felt dither. Five input-side elements, all in one pure unit-tested module (angle_smoothing.AngleSmoother): - asymmetric filter on |kappa| feeding the gain schedule — fast fixed attack (entry speed strength-independent), slow strength-scaled release (the primary fix: removes the self-modulation) - low-pass on the model predicted curvature (inside the VLT's slack) - slew on the exit-blend ratio (bounded ramps, no 4x steps from boolean chatter) - hysteresis on the curve-entering decision - 1-LSB hold on the outgoing wire value Strength is a menu dial with factor semantics: 1.0 = stock (bit-identical to the feature not existing — fuzz-tested), 2.0 = tuned, 2.5 = max. Defaults ship neutral: toggle ON at strength 1.0 means stock until the driver steps up. Output-side lag was explicitly tried and rejected: the closed-loop rig measured it degrading station-keeping 2.5x. Every filter resets across disengage/human-turn/stall-blip discontinuities, and each element re-seeds from the live value when inactive — toggling or stepping strength mid-drive can never act on stale state. On-road A/B (same car): slow lane sway ±0.46 -> ±0.26 m (−43%), dither sway −33%; curve exits measured faster and cleaner with smoothing on. Closed-loop cost at tuned strength: +3 mm station-keeping std. Params, both settings surfaces (angle-mode gated), sunnylink entries, 13 pure unit tests + closed-loop ext tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Driver-facing guide matching the auto-calibration one: the measured before/after numbers, the strength dial semantics (1.0 = provably stock), the honest cost table from the closed-loop rig, how to find a setting, hard guarantees, and troubleshooting including the calibrate-first relationship to auto-cal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The evidence that motivated this
00000006--31********) put numbers on it: a 0.23 Hz dither at ±0.033 m, a slow ±0.46 m lane sway, and the near-zero command crossing a wire code boundary 962 times/min (LatCtlPath_An_Actl's LSB is 0.0005 rad) — a square wave the PSCM integrates into felt motion.Why the fix is shaped this way
angle_smoothing.AngleSmoother, no I/O — unit-testable anywhere, same pattern as the auto-cal estimator): an asymmetric filter on the gain-schedule input (fast fixed attack, slow strength-scaled release — the primary fix), a low-pass on the model prediction (inside the variable-lookup slack, so free), a slew on the exit-blend ratio (boolean chatter becomes bounded ramps, not 4x steps), hysteresis on the curve-entering decision, and a 1-LSB hold on the outgoing wire value (a held frame is zero-ROC; the release step is far inside the tightest soft ROC — panda cannot be tripped).The evidence afterwards
MIN_KAPPAfloor sits at the gain-interp knee (coupling now documented at both sites), the wire hold's worst bias is two orders under the calibrator's stderr threshold, and the same hour-long drive that ran smoothing at 1.8 also locked the auto-calibration — the two features coexist measurably.What each change is
opendbc/sunnypilot/car/ford/angle_smoothing.py: the pureAngleSmoother— five elements, constants, strength semantics, transition-safe seeding.opendbc/sunnypilot/car/ford/lateral_angle_ext.py: one method call per element at the exact command-path point where it lives;smoothing_enabled/smoothing_strengthcompat properties for offline tooling.common/params_keys.h: registerFordAngleSmoothing(BOOL, default on) andFordAngleSmoothStrength(FLOAT, default 1.0 = stock).selfdrive/ui/bp/layouts/settings/bluepilot.py+mici/.../lateral_mici.py: toggle + strength stepper on both surfaces, angle-mode gated.sunnypilot/sunnylink/settings_ui.json: the two entries, same gating.opendbc/sunnypilot/car/ford/tests/test_angle_smoothing.py: the 13 pure unit tests.opendbc/sunnypilot/car/ford/tests/test_lateral_angle_ext.py: closed-loop strategy tests (OFF path equals unsmoothed math; filters reset on override paths; ROC properties hold).docs/ford-angle-smoothing.md: driver-facing guide — the numbers above, how to find a setting, hard guarantees, troubleshooting.Test it yourself
docs/ford-angle-smoothing.md.test_menu_one_is_bit_identical_passthroughholds it.python bp/angle_weave_analyze.py <dir> <route>(bp-tools) prints the band powers and sway amplitudes above.Risks / limits
🤖 Generated with Claude Code