Skip to content

fix(control): keep autonomous defaults ahead of stale commands - #798

Open
frahlg wants to merge 1 commit into
masterfrom
agent/fix-stale-driver-command-safety
Open

fix(control): keep autonomous defaults ahead of stale commands#798
frahlg wants to merge 1 commit into
masterfrom
agent/fix-stale-driver-command-safety

Conversation

@frahlg

@frahlg frahlg commented Aug 4, 2026

Copy link
Copy Markdown
Member

Summary

  • keep normal commands in their existing queue, but route autonomous defaults through a dedicated priority slot
  • block the current driver generation before accepting a default so raced normal commands are discarded without reaching hardware
  • retry an accepted default with a fresh bounded context when the watchdog caller has already timed out
  • add a deterministic legacy-driver regression test for a blocked command, a full stale queue, default recovery, and repeated -race runs

This follows the unresolved stale-queue P1 from #791. It preserves control-v2 generation selection and adds a patch changeset.

Verification

  • make verify-all with Python 3.12.13 and the verified 37-driver snapshot
  • go test -race ./internal/drivers -count=1
  • go test -race ./internal/api -count=1
  • go test -race ./internal/drivers -run '^TestRegistryDefaultBypassesStaleCommandQueue$' -count=50
  • read-only merge scratch with feat(control): optional single-battery scope for manual hold #608 head 8f19c7a: manual-hold/API/control -race tests and web test pass

Do not merge automatically; this PR needs review of the safety path.

@frahlg

frahlg commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

PAUSED / NOT READY TO MERGE. Remote head är 328e087. PR:s registrerade base är 369d120; aktuell live master är 081ea9e, så PR:n är bakom och ska rebasas före nästa push. Lokalt passerade make verify-all med Python 3.12.13 och 37 drivers, full internal/drivers -race, full internal/api -race, samt nya stale-queue-testet 50x med -race. Oberoende kvarvarande P1: defaultCh kan inte leverera default medan samma runLoop sitter fast i en legacy Command som ignorerar cancellation och aldrig returnerar; köprioritet räcker inte. P2: 2/50 statuskontroller läste före registry-statusuppdateringen; testerna måste synka på registry-status. Read-only scratch-merge mot Claude #608 head 8f19c7a var konfliktfri, utan direkt filöverlapp, och relevanta manual-hold/API/control- samt webtester passerade. Worktree har inga lokala ocommittade ändringar. Nästa steg är faktisk Lua-preemption eller säker generation-quarantine/restart, en oändlig legacy-driver-regression och status-testsynkning, sedan rebase mot 081ea9e och ny race/verify/#608-kontroll. #608 rörs inte.

frahlg added a commit that referenced this pull request Aug 5, 2026
… command (#805)

#800 established the law — a driver that cannot actuate gets its
autonomous default and stops being counted on — and wired exactly one
path into it. The reviewer named the gap and its author confirmed it:
only the storage loop filed an outcome. PV curtail discarded its error
outright and the loadpoint controller only logged one.

So the bug #800 fixed for batteries was still live for the other two.
A wallbox that answers every poll and refuses every setpoint held the
current it last accepted while the plan went on booking the charge and
the surplus reserve went on being held back from the battery for an EV
that was not drawing. An inverter that refused every cap kept exporting
into a negative price while the plan booked the saving.

Route both into the same tracker. The wiring is small; what took the
work was deciding which command on each path is the one whose refusal
means core cannot put power where it asked. Getting that wrong excludes
a healthy device, which is worse than the bug.

PV curtail: the cap counts, the release does not. A `curtail_disable`
is core letting go of the inverter, so refusing it proves nothing about
the device — the same reading #800 gives a rejected set_self_consumption
in sungrow.lua. It would also seal shut: ComputePVCurtail emits a
release the moment a driver drops offline, so a counted refusal would
let an excluded inverter hold itself out on its own exclusion.

Loadpoint: the periodic ev_set_current counts. Four other sends do not,
and each for its own reason. The 0 W standdown is core withdrawing
under a stale site meter, a transition the staleness tracker already
owns. `charge_start` goes to the bound vehicle driver, and a parked car
refuses it whenever it is asleep — counting that would take the car's
SoC out of the plan for napping, and wakeVehicleAuto already backs off
on its own. The ev_pause/ev_resume contactor cycle is documented as
free for any charger implementing those actions, so a charger that
implements neither returns an error and is behaving correctly; it also
runs on its own goroutine, which the tracker is not built to take. The
operator's force-start and refresh are not dispatch at all.

Nothing about what counts as a refusal changes: ErrObserveOnly,
ErrControlBlocked and deadline/cancel remain faults another mechanism
owns. Observe-only drivers still receive no command of any kind.

Deliberately still out of scope, as in #800: a refusal joined with
ErrControlBlocked is not counted, so such a driver stays in the MPC
fleet. #798 owns that registry state machine.

Risk direction, stated the way #800 did it: commands now reach hardware
on paths that previously sent none. An excluded PV inverter receives
its driver's declared default where it previously received nothing.
Both shipped EV chargers declare a no-op default — easee_cloud.lua and
ctek.lua both say a wallbox has no autonomous equivalent and should
hold its last current — so an excluded charger receives no new write at
all, and the exclusion is pure accounting.

Golden corpus: no record moved, as predicted. This changes command
outcome accounting, not dispatch arithmetic, and the replay never goes
through the control tick.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

@miravoss26 miravoss26 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes a P1 concurrency safety bug: autonomous "default"/watchdog commands now flow through a dedicated one-slot channel instead of sharing the normal command queue, so a stale/blocked queue can no longer starve the safety fallback path. The new TestRegistryDefaultBypassesStaleCommandQueue directly exercises the race (blocked active command + full stale queue + a default whose own caller times out) and asserts the default is accepted durably and retried to success, while confirming no stale command crosses the driver boundary.

  • Nit (info): the enqueue-time guard relies on a generation-blocked check (controlIsBlocked per the comment) outside this diff to discard a normal command that races the transition — not shown here, presumably pre-existing. Worth a quick gut-check that it behaves as claimed, though the new test empirically covers the scenario end to end.
  • No secrets, no new deps, no new network destinations.

Safe to merge from my read — you flagged this needs a human look at the safety path regardless, which I'd honor given the P1 history on #791.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants