Skip to content

fleet_dispatch: skip tiers that already failed identically, using ledger history + TieredEngine #12

Description

@lorenzoliuzzo

Context

Downstream of my-things-core#119 (the TieredEngine composition
primitive). Blocked on that landing first. This is the "consult the ledger
before choosing a tier" half of the original my-things-core#116 — it turns
out to need no new core surface, just a consumer-side pattern in the one
place that already has both attempt history and a retry loop:
fleet_dispatch.py.

Confirmed by reading the file: _last_attempt(ledger, candidate_id)
(fleet_dispatch.py:641-674) already filters kind="dispatch" ledger entries
for one candidate and returns an Attempt with final_message
(e.data.get("final_message")) and attempt_number. MAX_ATTEMPTS
(fleet_dispatch.py:694) already gates escalation to needs_human.

Proposed change

  1. Extend the existing attempt-reading path (generalizing _last_attempt, not
    replacing it) to return the last N dispatch entries for a candidate,
    not just the latest — the data is already there, this is just not
    truncating to one.
  2. Build a skip set for TieredEngine.run(): if the last N attempts'
    final_message values are identical for a given tier, add that tier name
    to skip on the next attempt for that candidate — the same "identical
    failure N times" signal my-fleet#11's harness-bug check uses, applied to
    tier selection instead of escalation.
  3. Wire a TieredEngine into the dispatch worker in place of whatever single
    Engine it constructs today, with skip computed this way per candidate
    before each attempt.

Explicitly out of scope

  • The harness-bug signature check itself and its escalation path — that's
    my-fleet#11, a related but separate signal (this issue skips a tier;
    fleet: distinguish 'the harness is broken' from 'the model keeps failing' on repeated identical failures #11 escalates the whole issue to a human). They can share the
    "last-N-identical" helper if one is written generically enough, but that's
    an implementation detail for whoever picks both up, not a reason to merge
    the issues.
  • The real local-model tier itself (my-things-core#119's scope note
    applies here too).

Blocked-by: my-things-core#119. Relates to my-fleet#11 (shared
"identical failure" signal, different consumer).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions