Skip to content

fix(adapters): complete dev sessions only on Stop or window death#53

Merged
pbean merged 8 commits into
bmad-code-org:mainfrom
dracic:fix/dev-idle-result-window-alive
Jul 4, 2026
Merged

fix(adapters): complete dev sessions only on Stop or window death#53
pbean merged 8 commits into
bmad-code-org:mainfrom
dracic:fix/dev-idle-result-window-alive

Conversation

@dracic

@dracic dracic commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #48

GenericAdapter.wait_for_completion could return completed from the idle-tick and grace-expiry shortcuts on terminal-artifact presence alone, while the agent window was still alive — run()'s finally: kill() then terminated the agent mid-turn and the engine post-processed half-written state (feeding the destructive auto-rollback path).

Changes:

  • Stop events and window death are now the only authoritative completion signals. Both artifact shortcuts in the idle branch are removed outright — the window is provably alive at those points (a dead window returns crashed earlier in the same tick), so the checks could never legitimately complete.
  • _final gains a keyword-only accept_result flag; the grace-expiry stall verdict passes False so an artifact under a live window cannot silently upgrade stalled to completed. Window-death paths (crashed, SessionEnd) keep the trusting default — death is authoritative.
  • Re-drive amplifier closed: strip_auto_run_result (new, in devcontract) removes the stale ## Auto Run Result section when the escalation re-arm re-opens a spec, so a re-driven spec no longer qualifies as a terminal artifact from the agent's first mid-session save. The helper is fence-aware (incl. fences indented up to 3 spaces per CommonMark) since deletion is destructive; the read-only heading scans are intentionally untouched (follow-up: [BUG] find_result_artifact / parse_auto_run_result treat a fence-quoted heading as a real terminal section #52).
  • The diagnostic trace scaffolding from the original prototype branch was deliberately not ported — it hunted an engine-death window already fixed on main.

Behavioral notes:

  • Stop-path completion (artifact + Stop, window alive) is untouched; stall-nudge and pane-activity re-arm logic identical.
  • One existing test amended to the new invariant: a completion marker landing after the nudge cap now completes via its turn-end Stop, not via the bare grace-expiry artifact.

Tests

  • New: idle-tick artifact under a live window does not complete (completes on the subsequent Stop); grace-expiry artifact under a live window stalls with result_json=None; window death with artifact still completes; strip unit tests incl. fenced/indented-fence edge cases; engine re-arm strips the stale section with the intent body untouched.
  • Full suite green on Windows (1324 passed) and Linux/WSL (1335 passed); only the two pre-existing test_module_skills_sync drift failures, unrelated.

Summary by CodeRabbit

  • Bug Fixes
    • Prevents stale terminal “Auto Run Result” sections from being treated as fresh completion when a spec is reopened or re-armed.
    • Improves fence-aware handling so “Auto Run Result” headings inside fenced code blocks are ignored for both detection and removal.
    • Enhances terminal-session completion reliability: late result.json no longer overrides an in-progress/stalled verdict while the window is alive; grace expiry and window death are honored correctly.
  • Tests
    • Expanded coverage for fenced-heading edge cases, repair/re-arm stale-section stripping, and terminal completion timing/grace/liveness scenarios.

@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Completion now waits for authoritative stop/death signals, fenced ## Auto Run Result headings are ignored unless real, and repair/re-arm paths now remove stale terminal sections before resuming a spec.

Changes

Stale result handling and completion gating

Layer / File(s) Summary
Completion gating and adapter tests
src/bmad_loop/adapters/generic.py, tests/test_generic_tmux.py
Live-window idle and grace paths no longer complete from result artifacts, _final can ignore result.json, and tmux tests cover idle, grace, death, and late-marker flows.
Fence-aware result parsing and stripping
src/bmad_loop/devcontract.py, tests/test_devcontract.py
parse_auto_run_result, find_result_artifact, and strip_auto_run_result now ignore fenced headings; tests cover fenced, indented, nested, and no-op cases.
Repair and re-arm cleanup wiring
src/bmad_loop/engine.py, src/bmad_loop/runs.py, tests/test_engine.py, tests/test_resolve.py
Repair reset and escalation re-arm strip stale auto-run sections after status reset, and tests verify the spec is cleaned while intent content remains intact.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GenericAdapter.wait_for_completion
  participant tmux window
  participant GenericAdapter._final
  participant devcontract.strip_auto_run_result
  participant Engine._reset_spec_for_repair
  participant runs.rearm_escalation

  GenericAdapter.wait_for_completion->>tmux window: Stop / window death / idle tick
  alt live window with stale result artifact
    GenericAdapter.wait_for_completion->>GenericAdapter._final: finalizes with accept_result=false
    GenericAdapter._final-->>GenericAdapter.wait_for_completion: stalled
  else dead window or authoritative Stop
    GenericAdapter.wait_for_completion->>GenericAdapter._final: finalizes with accept_result=true
    GenericAdapter._final-->>GenericAdapter.wait_for_completion: completed or crashed
  end

  Engine._reset_spec_for_repair->>devcontract.strip_auto_run_result: remove stale section
  runs.rearm_escalation->>devcontract.strip_auto_run_result: remove stale section
Loading

Possibly related issues

Poem

A rabbit hopped through spec and code,
Where stale results no longer rode.
Fences stood guard, neat and tall,
And stop or death now answer all.
Snip, hop, and wait — the path is clear,
The right end lands when it should appear.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main behavior change in dev-session completion.
Linked Issues check ✅ Passed The adapter now blocks live-window completion from artifacts, preserves Stop/window-death authority, and strips stale spec results as required by #48.
Out of Scope Changes check ✅ Passed The extra parser, reset, and test updates directly support the completion and stale-spec handling goals, with no unrelated behavior added.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

A terminal artifact observed while the agent window is still alive is
advisory: the idle-tick and grace-expiry shortcuts in wait_for_completion
could return completed mid-turn, letting run()'s finally-kill terminate a
working agent and hand the engine half-written state. Remove both
shortcuts (the window is provably alive at those points), pin the
grace-expiry stall verdict via _final(accept_result=False), and strip the
stale terminal section when a spec is re-armed for repair so a re-driven
spec cannot qualify as a result artifact from its first save.

Fixes bmad-code-org#48
@dracic dracic force-pushed the fix/dev-idle-result-window-alive branch from 940c2f9 to ca288c7 Compare July 3, 2026 18:43
pbean and others added 2 commits July 3, 2026 15:20
rearm_escalation flipped the spec back to ready-for-dev but left the
escalated attempt's `## Auto Run Result` section in place, so a resolved
re-drive still qualified as a terminal artifact from its first in-session
save — the untouched Stop path would then parse the prior attempt's stale
outcome (typically `Status: blocked`) as this session's result. Strip the
section alongside the status flip, mirroring engine._reset_spec_for_repair
on the in-loop repair path, so both re-arm paths are covered.

Refs bmad-code-org#48

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t scans

find_result_artifact qualified any fresh-mtime spec whose body contained
`## Auto Run Result` at column 0 — including a heading quoted inside a
fenced code example — so such a spec read as this session's terminal
artifact from the agent's first save. parse_auto_run_result likewise
picked a fenced copy as the "last" section and truncated detail at fenced
`## ` lines (quoted shell comments, log output). Route all three scans
(parse, find, strip) through one shared fence-aware heading/boundary
helper so a fenced example is never structure and the detectors no longer
diverge from the stripper.

Closes bmad-code-org#52

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pbean

pbean commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the excellent PR — the issue write-up (#48), the adversarial follow-up (#52), and the test coverage made validation straightforward. Everything checked out: the removed shortcuts sit strictly after the same-tick liveness return so the window is provably alive there, all four profiles emit canonical Stop events, the base adapter never arms the stall grace (so triage/non-dev sessions were never on the removed paths), and window-death still honors the artifact.

I pushed two maintainer commits onto the branch (maintainer edits are enabled):

  • 69dc4edruns.rearm_escalation (the bmad-loop-resolve re-arm) still left the stale ## Auto Run Result section behind; the strip only hooked engine._reset_spec_for_repair, which fires on the in-loop feedback-repair path, not the escalation re-drive. A resolved re-drive could therefore still parse the prior attempt's stale outcome (typically Status: blocked) on its first save + Stop. The re-arm now strips alongside the status flip, covering both re-arm paths as the PR description intended.
  • e782282 — folds in the fix for [BUG] find_result_artifact / parse_auto_run_result treat a fence-quoted heading as a real terminal section #52 while we're here: parse_auto_run_result / find_result_artifact now share the same fence-aware heading/boundary helpers as strip_auto_run_result, so a fence-quoted heading never qualifies a spec as terminal and the detectors no longer diverge from the stripper.

Full suite + trunk green locally on both commits; CI matrix re-running on the new head.

🤖 Generated with Claude Code

@pbean

pbean commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

augment review

@augmentcode

augmentcode Bot commented Jul 3, 2026

Copy link
Copy Markdown
🤖 Augment PR Summary

Summary: Makes dev-session completion authoritative only on turn-end Stop events or tmux window death, preventing premature completion when an artifact appears mid-turn.

Changes:

  • Removed idle-tick and grace-expiry “artifact shortcut” completion paths in GenericAdapter.wait_for_completion when the session window is still alive.
  • Added a final liveness re-probe on grace-expiry to ensure a just-dead window flows through the crash/death path (and can still honor a flushed artifact).
  • Introduced accept_result (kw-only) on GenericAdapter._final so live-window stalls can explicitly ignore advisory/stale artifacts.
  • Enhanced devcontract parsing to ignore ## Auto Run Result headings that are quoted inside fenced code blocks, including longer/mismatched fences.
  • Added strip_auto_run_result() and wired it into repair/re-arm flows so re-driven specs don’t retain stale terminal sections that would be misread as fresh completion.
  • Expanded tests across adapter completion timing, fence edge-cases, spec reset/rearm behaviors, and window-death vs stall outcomes.

Technical Notes: Fence tracking now considers both fence character and run-length (CommonMark-style) to avoid misclassifying nested/shorter fence lines as closes.

🤖 Was this summary useful? React with 👍 or 👎

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/bmad_loop/devcontract.py Outdated
…e rules)

The parity count in _fenced treated every ```/~~~ line as a fence toggle,
ignoring CommonMark's close rules. A shorter run inside a longer fence, or a
different fence char, flips parity early — exposing a fenced (documentation)
`## Auto Run Result` heading as a real section. On the strip path that is a
destructive misread: legitimate fenced content gets deleted on re-arm/repair.

Track the open fence's character and length and close only on a same-char run
at least as long as the opener with no trailing info string. All existing fence
tests (simple paired 3-backtick fences) are unaffected; adds parse/find/strip
tests for the nested-longer-fence and mismatched-char cases plus a
closed-then-real positive control.

Addresses the Augment review note on PR bmad-code-org#53.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pbean

pbean commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

@CodeRabbit review
augment review

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

@pbean Thanks for the detailed context and for pushing the two maintainer follow-up commits. I'll review the updated head, covering:

  • 69dc4ed: runs.rearm_escalation now stripping stale ## Auto Run Result sections for the bmad-loop-resolve escalation re-arm path.
  • e782282: parse_auto_run_result / find_result_artifact unified with the fence-aware helpers used by strip_auto_run_result.

Kicking off the review now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/bmad_loop/adapters/generic.py
The top-of-tick liveness probe and the grace-expiry stall return sit in the
same idle tick with a (small) gap between them. Because that return exits the
loop, a hard death (killed without a SessionEnd hook) landing in the gap is
never caught by a later top-of-tick probe — and accept_result=False then
discards a just-flushed terminal artifact that window death is supposed to
honor, feeding the destructive stalled -> RETRY -> rollback path.

Re-probe _window_alive once immediately before finalizing: a now-dead window
flows through the crash path (which trusts its artifact via accept_result=True),
so the outcome only ever upgrades (stall -> crash, or -> completed when the
artifact is present). decide_dev/decide_review_session treat crashed and
stalled identically, so the artifact-absent case is unchanged. A transport
error on the re-probe is not proof of death (mirrors the top-of-tick handling)
and falls through to the stall; spec.timeout_s bounds a persistent failure.

Narrowly closes the in-tick death race; the broader window-alive reconcile gap
remains issue bmad-code-org#61. Addresses the Augment review note on PR bmad-code-org#53.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pbean

pbean commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

augment review
@CodeRabbit review

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

@pbean Got it — kicking off the review again.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/bmad_loop/devcontract.py
…esult strip

A `## Auto Run Result` quoted inside a fence nested under list indentation
(4+ absolute leading spaces, past `_FENCE_LINE_RE`'s 0-3-space bound) is
co-indented with the fence, so it never matches the column-0-anchored
AUTO_RUN_HEADING_RE — nothing to strip. Locks that symmetry so any future
leading-space tolerance in the heading regex fails loudly instead of
destructively stripping quoted spec prose (reviewer #3522359669 on PR bmad-code-org#53).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pbean

pbean commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

augment review

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/bmad_loop/devcontract.py
reset_spec_status and strip_auto_run_result read the spec unconditionally, so a
re-drive/repair against a spec that no longer exists on disk crashed the entry
point. On the runs.rearm_escalation path this was reachable directly:
verify.set_frontmatter_status already no-ops on a missing file, so the following
strip_auto_run_result was the first line to raise; on engine._reset_spec_for_repair
the sibling reset_spec_status raised one line earlier. Guard both with the same
`if not spec_path.is_file(): return False` used by verify.set_frontmatter_status /
read_frontmatter / synthesize_result.

Scope is deliberate: only an absent spec is guarded. A present-but-unreadable spec
or a failing write still raises — silently skipping the strip after the caller has
flipped the frontmatter status would leave the re-opened spec carrying its stale
`## Auto Run Result` section, the exact state that makes the re-driven session's
first save read as a terminal result (the bug this PR fixes).

Addresses Augment review comment 3522384764 on PR bmad-code-org#53.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pbean

pbean commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

augment review

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/bmad_loop/devcontract.py
A bare `##` line is a valid empty CommonMark ATX heading, so
`_next_heading_start` (`^##\s`) bounding a section there is correct, not a
premature truncation. The `Status:` gate is parsed above the boundary and is
untouched; terminal detection keys on `## Auto Run Result`, never a bare `##`,
so a surviving empty-heading region can't read as a result artifact. On the
destructive strip path this is the safe direction (truncate early -> delete
less). Tightening `\s` to a space/tab delimiter would drop empty-heading
recognition on both LF and CRLF and run the strip PAST such a heading, deleting
more. Locks the intended behavior for parse + strip (reviewer #3522512350 on
PR bmad-code-org#53).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pbean

pbean commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

augment review

@pbean pbean merged commit 6562529 into bmad-code-org:main Jul 4, 2026
9 checks passed
@dracic dracic deleted the fix/dev-idle-result-window-alive branch July 4, 2026 08:22
@pbean pbean mentioned this pull request Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Dev session reported completed from the idle-poll shortcut while the agent window is still alive

2 participants