You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Downstream of my-things-core#119 (the TieredEngine composition
primitive). Blocked on that landing first. This is the "consult the ledger
before choosing a tier" half of the original my-things-core#116 — it turns
out to need no new core surface, just a consumer-side pattern in the one
place that already has both attempt history and a retry loop: fleet_dispatch.py.
Confirmed by reading the file: _last_attempt(ledger, candidate_id)
(fleet_dispatch.py:641-674) already filters kind="dispatch" ledger entries
for one candidate and returns an Attempt with final_message
(e.data.get("final_message")) and attempt_number. MAX_ATTEMPTS
(fleet_dispatch.py:694) already gates escalation to needs_human.
Proposed change
Extend the existing attempt-reading path (generalizing _last_attempt, not
replacing it) to return the last Ndispatch entries for a candidate,
not just the latest — the data is already there, this is just not
truncating to one.
Build a skip set for TieredEngine.run(): if the last N attempts' final_message values are identical for a given tier, add that tier name
to skip on the next attempt for that candidate — the same "identical
failure N times" signal my-fleet#11's harness-bug check uses, applied to
tier selection instead of escalation.
Wire a TieredEngine into the dispatch worker in place of whatever single Engine it constructs today, with skip computed this way per candidate
before each attempt.
Explicitly out of scope
The harness-bug signature check itself and its escalation path — that's my-fleet#11, a related but separate signal (this issue skips a tier; fleet: distinguish 'the harness is broken' from 'the model keeps failing' on repeated identical failures #11 escalates the whole issue to a human). They can share the
"last-N-identical" helper if one is written generically enough, but that's
an implementation detail for whoever picks both up, not a reason to merge
the issues.
The real local-model tier itself (my-things-core#119's scope note
applies here too).
Blocked-by: my-things-core#119. Relates to my-fleet#11 (shared
"identical failure" signal, different consumer).
Context
Downstream of
my-things-core#119(theTieredEnginecompositionprimitive). Blocked on that landing first. This is the "consult the ledger
before choosing a tier" half of the original
my-things-core#116— it turnsout to need no new core surface, just a consumer-side pattern in the one
place that already has both attempt history and a retry loop:
fleet_dispatch.py.Confirmed by reading the file:
_last_attempt(ledger, candidate_id)(
fleet_dispatch.py:641-674) already filterskind="dispatch"ledger entriesfor one candidate and returns an
Attemptwithfinal_message(
e.data.get("final_message")) andattempt_number.MAX_ATTEMPTS(
fleet_dispatch.py:694) already gates escalation toneeds_human.Proposed change
_last_attempt, notreplacing it) to return the last N
dispatchentries for a candidate,not just the latest — the data is already there, this is just not
truncating to one.
skipset forTieredEngine.run(): if the last N attempts'final_messagevalues are identical for a given tier, add that tier nameto
skipon the next attempt for that candidate — the same "identicalfailure N times" signal
my-fleet#11's harness-bug check uses, applied totier selection instead of escalation.
TieredEngineinto the dispatch worker in place of whatever singleEngineit constructs today, withskipcomputed this way per candidatebefore each attempt.
Explicitly out of scope
my-fleet#11, a related but separate signal (this issue skips a tier;fleet: distinguish 'the harness is broken' from 'the model keeps failing' on repeated identical failures #11 escalates the whole issue to a human). They can share the
"last-N-identical" helper if one is written generically enough, but that's
an implementation detail for whoever picks both up, not a reason to merge
the issues.
my-things-core#119's scope noteapplies here too).
Blocked-by:
my-things-core#119. Relates tomy-fleet#11(shared"identical failure" signal, different consumer).