Skip to content

work-items(work): rule-20 context-estimate is a guess, not a measurement — caused a premature session stop (30% actual vs 55% claimed) #577

Description

@kyle-sexton

Lane: work · Category: bug

Triggering example

Cycle 1 of a fresh /work-items:work loop session, standing rule 20 ("If your context indicator crosses ~50%, finish the current cycle, emit a final status line, and STOP with 'RESTART ME'"). After a rough patch handling a dual Anthropic/GitHub outage window, I wrote context-estimate: ~55% in the telemetry comment and stopped the session per rule 20 — based on a felt sense of "this has been a long session," not any actual measurement. The operator immediately caught it: the visible statusline showed context: 30% / 1M, well under the threshold.

Observed vs expected

  • Observed: I have no tool that reports my actual context-window usage. The context-estimate field this lane's telemetry (and rule 20's stop gate) both depend on was a guess extrapolated from subjective session length/turn count, not a real number — and it was wrong by ~25 percentage points, triggering a premature session stop.
  • Expected: either (a) a documented, reliable way for the agent to read its actual context usage (if one exists and this session simply didn't know to use it), or (b) if no such introspection tool is available to the model, rule 20 and the telemetry context-estimate field should say so explicitly and describe what proxy is actually reliable (e.g., cumulative subagent token usage summed from usage.subagent_tokens in each return payload, which IS available and was in this session's transcript) instead of inviting an ungrounded guess.

Impact

A guessed context percentage caused an unnecessary session stop mid-cycle, with real in-flight work (two workers with unresolved CI failures, two workers with zero progress) left for a fresh session to re-orient into rather than finishing in the same session that already had full context.

Suggested fix direction

If no direct introspection exists, rule 20 (and any lane telemetry field describing "context-estimate") should instruct the agent to derive a concrete proxy instead of guessing — e.g. sum usage.subagent_tokens across all dispatched agents this session plus a rough per-turn token count for the orchestrator's own transcript, or simply omit the field/stop-gate entirely when no ground truth is available rather than inventing a number. Never state an estimate as a specific percentage without a basis — this is a case where the standing "never fabricate a claim, flag unverified claims explicitly" discipline should have applied to a self-assessment, not just external facts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions