[Bug] v0.8.0 supervisor restarts make every TUI fail to reattach with “Unknown active session”

[alvinycheung]

Summary

The shared daemon supervisor disappears several times per day. Every open TUI then shows:

Daemon connection lost; reconnecting…
Error: Daemon reconnection failed: Unknown active session: <id>

This affects all active agents on the machine at the same time. The session workers often remain alive, but the replacement supervisor does not restore all active-session mappings. The TUIs keep retrying their old IDs and cannot reattach.

I first noticed this after Prime Agent performed its first self-update. I cannot prove that the update caused the initial supervisor failure. The local logs do show that the issue continued after a successful v0.8.0 update handoff.

Environment

• Prime Agent 0.8.0, global npm install
• Node.js v24.16.0
• macOS 26.6 on Apple Silicon
• Default daemon socket and default ~/.prime/agent state
• High multi-agent use. prime-agent status --json reported 28 sessions after the latest recovery.
• A beta daemon also exists, but it uses a separate state directory and socket under ~/.prime/agent-beta.

Frequency and impact

I see the user-facing failure at least three times per day. It interrupts every open Prime Agent TUI, including agents that were still working.

The current daemon log spans several package versions. It contains 40 supervisor generations and 41 worker messages that say they launched a replacement supervisor. This is not one isolated stale session.

Latest incident

The v0.8.0 update restart record started at 2026-08-21T21:34:14Z and completed with:

{"total":14,"restored":14,"resumed":2,"failed":0}

From that handoff through 2026-08-22T02:25:46Z, the log recorded 10 different supervisor generations.

At the latest replacement, the new supervisor tried to adopt existing workers:

[2026-08-22T02:25:33.050Z] Could not adopt worker <worker>: Error: Timed out connecting to daemon session worker: Error: Timed out waiting for daemon worker hello
[2026-08-22T02:25:45.743Z] Recovered worker <worker> without replaying uncertain operations: ...
[2026-08-22T02:25:46.437Z] Prime Agent daemon supervisor <generation> listening on .../daemon.sock
[2026-08-22T02:25:51.403Z] Supervisor command attach failed: Error: Unknown active session: <session>

For that startup:

• 18 worker adoptions timed out and 2 persisted workers were already dead.
• 7 workers logged as recovered.
• The supervisor then logged 153 Unknown active session attach failures: exactly 9 active IDs retried 17 times each.
• The session IDs shown in the TUI were mapped to worker descriptors that had just changed to lifecycle: "failed" with lastError: "Waiting for a client with fresh runtime context".
• Many corresponding worker PIDs were still alive.

The prior supervisor's last log line was a normal worker-listening event five minutes earlier. There was no graceful shutdown, update-restart notice, SIGTERM, or fatal record before it disappeared.

Racing replacement attempts also produce many instances of:

Daemon supervisor startup failed: Error: Lock file is already being held

What seems to be happening

The secondary failure is clear from the logs: supervisor recovery fails to adopt several live workers. Startup gives each worker connection about two seconds. After the recovery retries fail, resident workers without fresh client context become failed.

Each failed descriptor still contains its durable rootActiveSessionId. However, findWorker() only searches the worker's populated session summaries. It does not fall back to the descriptor's root ID or saved session file. Each TUI then retries its previous ID and gets Unknown active session.

The first supervisor failure is less certain. Two existing reports match the evidence:

1. Discussion Daemon lock sites omit onCompromised, so a stolen lockfile throws from an fs callback and kills the supervisor #1556 describes the daemon socket lease being stolen after an event-loop stall. In v0.8.0, acquireDaemonSocketPathLease still uses a 5-second stale window and 1-second refresh without an onCompromised handler. proper-lockfile can throw from its refresh callback after another process takes the lease. That could explain why the old supervisor ends without a final log line.
2. Issue Busy worker starves control-plane RPCs (handshake/attach/heartbeats time out); clients then misclassify the daemon as dead and race to autostart new daemons #1200 describes slow control-plane responses causing clients to treat a busy daemon as dead and race to start replacements. The repeated lock collisions and many concurrent TUIs are consistent with that path.

I have not captured an ECOMPROMISED error, so #1556 is a likely cause, not a confirmed cause. The detached supervisor launch discards stderr, which may explain why the decisive uncaught exception is absent.

Related reports:

• Daemon lock sites omit onCompromised, so a stolen lockfile throws from an fs callback and kills the supervisor #1556: daemon lock sites omit onCompromised
• Busy worker starves control-plane RPCs (handshake/attach/heartbeats time out); clients then misclassify the daemon as dead and race to autostart new daemons #1200: control-plane timeout and daemon autostart stampede
• [Bug] Session daemon loses the worker mid-session: repeated 'Unknown active session' / 'Connection error', agent continues anyway #1417: active session becomes unknown after daemon loses its worker

Recovery result

The internal retry_worker path works, but v0.8.0 does not expose it through the public command router. This command fails:

prime-agent daemon retry <root-active-session-id>
Error: Unknown command: daemon retry

I invoked the installed internal retry handler against failed descriptors whose worker PID and socket were still alive:

• 16 of 16 workers changed from failed to ready.
• All 16 answered a fresh state request.
• All 16 appeared in prime-agent list --json again.
• The two active IDs from the screenshot recovered.
• Two older descriptors had dead PIDs and no sockets, so I did not retry them.

This recovered the sessions without a global daemon shutdown. It also confirms that the durable descriptor contains enough data to repair the active-session mapping. A public retry command or automatic use of this path during reconnect would reduce the impact.

If internal retry fails, reopening the saved transcript is safer than deleting daemon lock files. prime-agent shutdown --force is a last resort because it stops all sessions.

Expected behavior

• One slow or failed worker should not disconnect every agent on the machine.
• A replacement supervisor should keep retrying live worker adoption or provide a safe resume path.
• TUI reconnect should not keep retrying an active-session ID that the replacement supervisor discarded.
• The supervisor should log its own fatal exit cause in a durable log.

Questions

1. Does the v0.8.0 supervisor write uncaught exceptions from the socket lease refresh to any log other than ~/.prime/agent/logs/agent.jsonl?
2. Could worker adoption preserve the old active-session mapping while it retries a slow worker?
3. Should the socket lease use an onCompromised handler and a longer or adaptive stale window?
4. Is there a supported diagnostic command or bundle I should capture immediately after the next occurrence?

I can provide a larger sanitized log slice if useful. I have omitted session paths, prompts, tokens, and worker authentication data.

[goghcrow]

I observed another instance of this failure and found two concrete races in the current reconnect path.

1. A healthy replacement supervisor can be shut down as stale

The sequence was:

original supervisor exits
worker launches a replacement supervisor
client connects to its socket while startup/worker adoption is still in progress
probeDaemonVersion() receives no recognizable hello within 2 seconds
the client records the daemon as stale
the replacement finishes recovery and starts listening
shutdownStaleDaemonIfNotBusy() lists 72 idle sessions and shuts it down

The relevant client log was:

daemon-launch: running daemon on <socket> sent no recognizable hello; treating as stale
...
daemon-launch: replacing stale daemon on <socket> (idle): 72 loaded session(s) will reload

The supervisor log shows that the replacement announced readiness immediately before the shutdown decision:

Session worker <worker> listening
Prime Agent daemon supervisor <generation> listening

The command journal confirms that a reconnecting daemon client then sent the shutdown command. The replacement itself was healthy by that point.

The race appears to be in src/cli/daemon-launch.ts:

• probeDaemonVersion() classifies “connected, but hello was not received in time” as stale.
• shutdownStaleDaemonIfNotBusy() subsequently reconnects and may act on a now-ready replacement without revalidating the version/identity that caused the stale decision.

A minimal safety rule would be: no hello means starting/unready, not stale. Only an explicit incompatible hello should authorize automatic replacement.

2. Generic reconnect reuses a supervisor-scoped activeSessionId

After the next supervisor started, the TUI repeatedly failed with:

Unknown active session: <old-active-session-id>

DaemonAgentConnection.reconnect() reconnects the transport and calls attach() with the previous activeSessionId. That ID is ephemeral and may not remain valid across supervisor replacement.

The same class already has the safer pattern in restoreConnectionAfterUpdate(): remember the durable sessionFile/sessionId, query list, locate the restored session, update activeSessionId, and then attach. Generic reconnect should use the same stable-identity recovery path.

In this incident, the replacement had already restored and listed the sessions, so no new daemon command or protocol change was required. The minimal fix appears to be:

1. Never mutate or shut down a reachable no-hello daemon based only on a hello timeout.
2. Share the existing stable-session lookup between update reconnect and generic reconnect.
3. Update activeSessionId before attaching.

I also verified that both code paths are still present on the current upstream main.

I can prepare focused regression tests for the delayed-hello race and the changed-activeSessionId reconnect case if maintainers confirm this direction.

[coleleavitt]

I can vouch this is also an error as well, please open a PR if not already done