Rectify: Adjudicated Failure False Positive — PART A ONLY

[Trecek]

Summary

The dead-end guard in _compute_outcome (session.py) conflates two fundamentally different failure categories under a single "channel confirmed + failure = retriable" assumption:

1. Drain-race artifacts (transient) — session.result is empty or missing the completion marker because stdout was not fully drained before the guard ran. These are genuinely retriable: a resume will find the content.

2. Pattern-contract violations (terminal) — session.result is non-empty and contains the completion marker, but the skill's expected_output_patterns are absent. The session completed normally; the model simply did not produce the required output block. Retrying will never produce different content.

The guard promotes both categories to RETRIABLE + RESUME, causing infinite retry cycles until the budget guard terminates the run. This PR introduces ContentState, a typed enum that makes the four content evaluation outcomes first-class concepts (ABSENT, CONTRACT_VIOLATION, COMPLETE, SESSION_ERROR). The dead-end guard now checks ContentState rather than just bool, restricting drain-race promotion to ContentState.ABSENT only. This makes the bug class structurally impossible.

Architecture Impact

Process Flow Diagram

%%{init: {'flowchart': {'nodeSpacing': 40, 'rankSpacing': 50, 'curve': 'basis'}}}%%
flowchart TB
    classDef terminal fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;

    START([session exits<br/>channel confirmed])

    subgraph Recovery ["Recovery Phase (● headless.py)"]
        direction TB
        R1["_recover_from_separate_marker<br/>━━━━━━━━━━<br/>Rebuild result if marker<br/>in standalone assistant msg"]
        R2["● _recover_block_from_assistant_messages<br/>━━━━━━━━━━<br/>Rebuild result if patterns found<br/>in assistant_messages<br/>(now: CHANNEL_A + CHANNEL_B)"]
    end

    subgraph Evaluation ["★ Content Evaluation (● session.py)"]
        direction TB
        ECS["★ _evaluate_content_state<br/>━━━━━━━━━━<br/>is_error? → SESSION_ERROR<br/>empty result? → ABSENT<br/>marker absent? → ABSENT<br/>patterns fail? → CONTRACT_VIOLATION<br/>else → COMPLETE"]
    end

    subgraph Outcome ["● _compute_outcome (session.py)"]
        direction TB
        CS["● _compute_success<br/>━━━━━━━━━━<br/>CHANNEL_B: delegates to<br/>_evaluate_content_state<br/>CHANNEL_A: _check_session_content"]
        CR["_compute_retry<br/>━━━━━━━━━━<br/>API signals / termination /<br/>channel confirmation dispatch"]
        CONTRA{"contradiction guard<br/>━━━━━━━━━━<br/>success=True AND<br/>needs_retry=True?"}
        DEG{"● dead-end guard<br/>━━━━━━━━━━<br/>not success AND<br/>not needs_retry AND<br/>channel confirmed?"}
        DEGS{"★ ContentState check<br/>━━━━━━━━━━<br/>ABSENT?"}
    end

    subgraph Resolution ["Outcome Resolution"]
        direction TB
        PROMOTE["promote to RETRIABLE<br/>━━━━━━━━━━<br/>retry_reason = RESUME<br/>(drain-race rescue only)"]
        TERMINAL["remain FAILED<br/>━━━━━━━━━━<br/>CONTRACT_VIOLATION or<br/>SESSION_ERROR → terminal"]
        NS["_normalize_subtype<br/>━━━━━━━━━━<br/>adjudicated_failure /<br/>empty_result / missing_marker"]
    end

    RESULT(["SkillResult<br/>success / needs_retry / subtype"])

    START --> R1 --> R2 --> ECS
    ECS --> CS
    ECS --> CR
    CS --> CONTRA
    CR --> CONTRA
    CONTRA -->|"yes: demote success"| DEG
    CONTRA -->|"no"| DEG
    DEG -->|"yes"| DEGS
    DEG -->|"no"| NS
    DEGS -->|"ABSENT"| PROMOTE
    DEGS -->|"CONTRACT_VIOLATION<br/>or SESSION_ERROR"| TERMINAL
    PROMOTE --> NS
    TERMINAL --> NS
    NS --> RESULT

    class START,RESULT terminal;
    class R1,R2 handler;
    class ECS,DEGS newComponent;
    class CS,DEG detector;
    class CR,CONTRA phase;
    class PROMOTE stateNode;
    class TERMINAL,NS output;

Loading

Color Legend:

Color	Category	Description
Dark Blue	Terminal	Session entry and SkillResult exit
Orange	Handler	Recovery phase functions (● modified)
Green	New Component	★ _evaluate_content_state and ★ ContentState gate
Red	Detector	● _compute_success (modified), ● dead-end guard (modified)
Purple	Phase	_compute_retry, contradiction guard
Teal	State	RETRIABLE promotion (drain-race rescue path)
Dark Teal	Output	Terminal outcome and _normalize_subtype

State Lifecycle Diagram

%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
    classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
    classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;
    classDef terminal fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;

    subgraph Fields ["SKILLRESULT FIELD LIFECYCLE"]
        direction LR
        INIT["INIT_ONLY<br/>━━━━━━━━━━<br/>session_id<br/>start_ts<br/>completion_marker<br/>NEVER modify after set"]
        APPEND["APPEND_ONLY<br/>━━━━━━━━━━<br/>assistant_messages<br/>only grows during recovery"]
        MUTABLE["MUTABLE<br/>━━━━━━━━━━<br/>● needs_retry<br/>● success<br/>● retry_reason<br/>● subtype<br/>computed then may be promoted"]
        DERIVED["DERIVED<br/>━━━━━━━━━━<br/>outcome<br/>mapped from success+needs_retry<br/>(SUCCEEDED / RETRIABLE / FAILED)"]
    end

    subgraph Recovery ["● RECOVERY MUTATIONS (headless.py)"]
        direction TB
        R1["_recover_from_separate_marker<br/>━━━━━━━━━━<br/>may overwrite session.result<br/>from assistant_messages"]
        R2["● _recover_block_from_assistant_messages<br/>━━━━━━━━━━<br/>appends to session.result<br/>now: CHANNEL_A + CHANNEL_B"]
    end

    subgraph ContentGate ["★ CONTENTSTATE VALIDATION GATE (session.py)"]
        direction TB
        ECS["★ _evaluate_content_state<br/>━━━━━━━━━━<br/>reads: session.result, is_error,<br/>completion_marker, patterns<br/>writes: ContentState discriminant"]
        CS_ENUM["★ ContentState enum<br/>━━━━━━━━━━<br/>COMPLETE — all checks pass<br/>ABSENT — empty/marker missing (retriable)<br/>CONTRACT_VIOLATION — result+marker+no patterns (terminal)<br/>SESSION_ERROR — is_error=True (terminal)"]
    end

    subgraph DeadEndGuard ["● DEAD-END GUARD CONTRACT (session.py)"]
        direction TB
        PRE["● _compute_outcome entry<br/>━━━━━━━━━━<br/>success=False AND<br/>needs_retry=False AND<br/>channel_confirmed?"]
        GATE{"★ ContentState gate<br/>━━━━━━━━━━<br/>ContentState.ABSENT?"}
        PROMOTE["PROMOTE<br/>━━━━━━━━━━<br/>needs_retry = True<br/>retry_reason = RESUME<br/>(drain-race rescue)"]
        BLOCK["BLOCK promotion<br/>━━━━━━━━━━<br/>needs_retry stays False<br/>outcome = FAILED<br/>(contract violation guard)"]
    end

    subgraph SkillResult ["SKILLRESULT OUTPUT CONTRACT"]
        direction TB
        SR["SkillResult<br/>━━━━━━━━━━<br/>success / needs_retry / subtype<br/>guaranteed: CONTRACT_VIOLATION<br/>never becomes RETRIABLE"]
    end

    INIT --> Recovery
    APPEND --> Recovery
    MUTABLE --> Recovery
    R1 --> R2
    R2 --> ECS
    ECS --> CS_ENUM
    CS_ENUM --> PRE
    PRE --> GATE
    GATE -->|"ABSENT"| PROMOTE
    GATE -->|"CONTRACT_VIOLATION<br/>or SESSION_ERROR"| BLOCK
    PROMOTE --> MUTABLE
    BLOCK --> MUTABLE
    MUTABLE --> DERIVED
    DERIVED --> SR

    class INIT detector;
    class APPEND handler;
    class MUTABLE phase;
    class DERIVED gap;
    class R1,R2 handler;
    class ECS,CS_ENUM,GATE newComponent;
    class PRE detector;
    class PROMOTE stateNode;
    class BLOCK output;
    class SR terminal;

Loading

Color Legend:

Color	Category	Description
Red	INIT_ONLY / Guard Entry	Fields that must never change; dead-end guard check
Orange	APPEND_ONLY / Handlers	assistant_messages growth; recovery functions (● modified)
Purple	MUTABLE	Fields computed and potentially promoted by guards
Yellow/Amber	DERIVED	Outcome derived from field combination
Green	New Component	★ ContentState enum, ★ _evaluate_content_state, ★ gate
Teal	Promotion	needs_retry promoted for drain-race (ABSENT only)
Dark Teal	Block	Promotion blocked for terminal failures
Dark Blue	Terminal	SkillResult output contract

Implementation Plan

Plan file: temp/rectify/rectify_adjudicated-failure-false-positive_2026-03-16_000001_part_a.md

Token Usage Summary

No token data available for this pipeline run.

🤖 Generated with Claude Code via AutoSkillit

[Trecek]

AutoSkillit PR Review — Verdict: changes_requested

[Trecek]

[warning] tests: session=session first argument to _compute_outcome is under-indented relative to other keyword arguments (column 8 vs column 12). ruff will flag this and auto-fix but abort the commit, requiring re-stage and retry.