Drive a single persistent claude per worker via bidirectional stream-json (closes #455)

[FidoCanCode]

Introduces a persistent ClaudeSession that keeps a single stream-json subprocess alive across a worker's setup, task, nudge, thread, and CI phases, switching between Opus for planning/triage and Sonnet for implementation, and bounds context growth by restarting on issue boundaries. Remaining work fixes a session teardown bug so cleanup only runs on orderly shutdown (with a crash-recovery test and an outer try/finally around stop() to always unregister the child), resolves a lost-interrupt race from clearing _cancel on every lock acquisition, sends a stream-json control_request interrupt in ClaudeSession.interrupt before follow-up content, and removes the durable preempt queue since it has no production path.

Fixes #455.

---

Work queue

Completed (40)

• Blocking: clearing _cancel on every lock acquisition creates a lost-interrupt
• Woof, you are absolutely right — caught me red-pawed! That unconditional stop()
• PR comment: fix session teardown — only on orderly shutdown, add crash-recovery test
• wrap stop() body in outer try/finally to always unregister child
• send stream-json control_request interrupt in ClaudeSession.interrupt before sending follow-up content
• High: this persists a durable preempt queue, but I cant find a production path t
• PR comment: remove unnecessary default model setting
• replace ClaudeStreamError(-1) magic number with a named constant or sentinel
• replace repeated truncation slicing with a helper that wraps log calls
• use context manager for lock acquire/release in interrupt
• reuse State's thread-safe JSON base class with modify method for the preempt intent queue
• surface errors instead of swallowing them silently
• Reraise the exception
• reraise exceptions in restart cleanup instead of swallowing
• reraise exceptions in restart() instead of swallowing them
• We can't spawn a new one if the old one running when timeout happens?
• We have to kill and safe restart. Kind of our mantra.
• Yes
• Bound context growth: restart ClaudeSession on issue boundary
• surface current ClaudeSession owner thread in kennel status line
• clarify ClaudeSession persists across worker death but not kennel/home restarts
• persist preempt intent to durable queue so interrupts survive re-preemption without losing messages
• preserve ClaudeSession across worker death so webhook-driven interrupts keep the subprocess alive and hand it back when the worker reclaims the lock
• Route execute_task and nudge loop through the persistent ClaudeSession
• Route handle_threads and handle_ci claude_run calls through the persistent session
• Remove setup_session_id state and dead --resume plumbing now superseded by session
• Switch models per phase: Opus for planning/triage, Sonnet for implementation
• make interrupt cancel the in-flight turn so the lock holder exits
• make interrupt cancel the in-flight turn so the lock holder exits
• extract idle timeout poll interval and event line log-truncation length into named constants
• log exceptions in stop() cleanup instead of swallowing silently
• remove JSONDecodeError handler and let the exception propagate
• Add threading lock and context manager to ClaudeSession for send/receive serialization
• Expose lock-break path so preempt/interrupt can bypass the session lock
• Implement preempt via control_request interrupt with queued follow-up turn
• Add ClaudeSession: bidirectional stream-json subprocess with send/events/stop
• ClaudeSession: idle-timeout reader and _active_children registration for shutdown
• ClaudeSession.switch_model via /model slash command with tests
• ClaudeSession.consume_until_result: drain events until type=result boundary
• make interrupt kill the in-flight turn, then acquire the lock before sending

[rhencke]

It would be cool if kennel status could show (uptime, events/min ↓, commands/min ↑) avg last 10min moving window?

[FidoCanCode]

Ooh that's a fun idea — rolling 10min window of uptime, events/min, and commands/min would make kennel status way more useful for spotting when something's off. tail wag

But it's out of scope for this PR — I want to keep this one focused on the persistent claude stream work. I've opened #457 to track it: #457

[rhencke]

It would be nice to surface when compaction is running in kennel status like COMPACTING

[FidoCanCode]

Ooh that's a great idea — surfacing COMPACTING in kennel status would be really handy for knowing when to expect a pause. tail wag But it's a bit outside the scope of this PR, which is just about the persistent claude stream plumbing. I've filed #458 to track it so it doesn't get lost!

[rhencke]

You're going to need to determine how to interrupt Claude with a streaming event to preempt for things like ci failures first or rescopes

[FidoCanCode]

Woof, good catch — want to make sure I sniff in the right direction here. A few ways I could go:

1. Send a control_request with interrupt over the stream-json stdin (the SDK's built-in cancel), then queue a fresh user turn describing the CI failure / rescope.
2. Close/reopen the session entirely on preempt (heavier, loses context).
3. Inject a high-priority user message mid-turn without interrupting, and trust Claude to pivot.

My gut says (1) — it's the cleanest and matches what stream-json is designed for. But do you have a preference, or a specific failure mode you're worried about (e.g. tool calls in flight, partial edits)? Want to make sure I chase the right ball. 🐾

[rhencke]

Woof, good catch — want to make sure I sniff in the right direction here. A few ways I could go:

1. Send a control_request with interrupt over the stream-json stdin (the SDK's built-in cancel), then queue a fresh user turn describing the CI failure / rescope.
2. Close/reopen the session entirely on preempt (heavier, loses context).
3. Inject a high-priority user message mid-turn without interrupting, and trust Claude to pivot.

My gut says (1) — it's the cleanest and matches what stream-json is designed for. But do you have a preference, or a specific failure mode you're worried about (e.g. tool calls in flight, partial edits)? Want to make sure I chase the right ball. 🐾

1

[FidoCanCode]

Woof, going with option (1) then! tail wag

I'll send a control_request with interrupt over stream-json stdin, then queue a fresh user turn describing the CI failure / rescope. Clean, matches what stream-json is built for, and keeps the session warm so we don't lose context.

For tool calls in flight — the interrupt will cancel the current turn, and the follow-up user message will give Claude the new situation to react to. Partial edits stay on disk; the next turn can sniff around and reconcile if needed.

Off to fetch! 🐾

[FidoCanCode]

Woof! Just a heads up — the task from your comment got a little polish on its title ("Implement" is now capitalized). Same scope, same plan. tail wag Back to sniffing out the implementation!

[FidoCanCode]

Woof! Just a heads up — the task from your comment got updated to match new requirements. Back to sniffing! tail wag

[rhencke]

Claude sessions should persist across worked death but not home/kennel death