fix(pi): distinguish stale locks when arming watcher

[kunchenguid]

Intent

Fix the Pi primary watcher arm refusal so an owned session lock still arms, a live other holder keeps the existing read-only refusal, and an absent or dead holder gets a distinct message saying no live session holds the lock and directing the operator to run bin/fm-session-start.sh to reclaim it and then call fm_watch_arm_pi to re-arm. Add focused coverage for owned, live-other, absent, and dead-holder states, and keep the relevant Pi, lock, type, turn-end, and lint checks green. Do not broaden this safety fix into a session-start work-under-way nudge because reliable work-state detection would require wider extension changes; record that optional idea as a PR follow-up instead.

What Changed

• Updated Pi watcher arming so owned session locks still arm, live other holders stay read-only, and absent or dead locks report the fm-session-start.sh recovery path before re-arming.
• Added focused Pi extension coverage for owned, live-other, absent, and dead-holder lock states.
• Documented the stale-lock recovery step in the Pi supervision protocol and stabilized watcher extension async waits.

Risk Assessment

✅ Low: The change is narrowly scoped to distinguishing Pi watcher arm lock states and adds focused coverage for the required owned, live-other, absent, and dead-holder cases without broadening session-start behavior.

Testing

The prompt-provided baseline full-suite was already green; I then captured reviewer-visible Pi tool output for absent/dead/live-other/owned lock states and ran the focused Pi watcher, watcher-lock, turn-end guard, and repeated Pi watcher rearm-flake checks successfully, with lint/type static checks skipped only because this prompt forbids them.

Evidence: Pi lock-state fm_watch_arm_pi transcript

absent-lock: ok=false message=watcher: not armed - no live session holds the lock; run bin/fm-session-start.sh to reclaim it, then call fm_watch_arm_pi to re-arm dead-holder-lock: ok=false message=watcher: not armed - no live session holds the lock; run bin/fm-session-start.sh to reclaim it, then call fm_watch_arm_pi to re-arm live-other-lock: ok=false message=watcher: read-only - session lock is held by another firstmate session owned-lock: ok=true message=watcher: started Pi extension arm child 1 arm-log: arm args=--restart ...

absent-lock: ok=false message=watcher: not armed - no live session holds the lock; run bin/fm-session-start.sh to reclaim it, then call fm_watch_arm_pi to re-arm
dead-holder-lock: ok=false message=watcher: not armed - no live session holds the lock; run bin/fm-session-start.sh to reclaim it, then call fm_watch_arm_pi to re-arm
live-other-lock: ok=false message=watcher: read-only - session lock is held by another firstmate session
owned-lock: ok=true message=watcher: started Pi extension arm child 1
arm-log: arm args=--restart home=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/home root=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/repo

Evidence: Focused behavior test transcript

tmux 3.6a
== tests/fm-pi-watch-extension.test.sh ==
ok - Pi primary watcher extension is tracked, self-hashing, and self-locating
ok - Pi secondmate launch wiring includes both tracked primary extensions
ok - Pi extension reports external healthy watcher output
ok - Pi custom tool returns text content and structured details
ok - Pi watcher arm distinguishes all session lock ownership states
ok - Pi process-exit cleanup listener has a bounded lifecycle
ok - Pi process-exit cleanup stops the attached arm child
ok - OpenCode primary watcher plugin has the verified TUI wake wiring
ok - OpenCode plugins have an explicit ESM boundary even under a typeless parent package
ok - OpenCode watcher plugin uses the effective FM_HOME state
ok - OpenCode watcher plugin sources the effective config
ok - OpenCode watcher plugin requires session lock ownership
ok - OpenCode watcher coordinator respects primary scope
ok - OpenCode watcher plugin rearms after a watcher wake
ok - OpenCode watcher plugin coordinates with the turn-end guard
ok - OpenCode healthy arm output does not suppress the turn-end guard
== tests/fm-watcher-lock.test.sh ==
ok - simultaneous watcher starts leave exactly one live process
ok - fm_pid_identity is locale-invariant across LC_ALL/LC_TIME
ok - killed watcher stale lock is reclaimed
ok - live watcher lock with stale heartbeat is actionable
ok - guard banner leads when down with pending wakes (re-arm-after-drain) and stays silent when fresh
ok - concurrent fm_lock_try_acquire yields exactly one winner
ok - dead-pid stale lock is reclaimed by a single acquirer
ok - concurrent stale-lock steal yields exactly one winner
ok - live steal mutex is not reclaimed
ok - live-held lock is not stolen
ok - empty mid-acquire lock keeps a minimum grace
ok - late original claimant cannot claim a recreated lock
ok - paused mid-acquire claimant backs off to active stealer
ok - watch restart refuses to signal a reused pid
ok - watch restart reports a healthy peer without attaching to it
ok - watcher self-evicts when the lock pid no longer names it
ok - arm attaches to a live fresh watcher and exits only when that cycle ends
ok - arm starts+confirms a fresh watcher on a clean lock and self-heals a dead-pid lock (never healthy off a dead pid)
ok - arm cleans child watcher and temp output on HUP
ok - arm propagates an immediate watcher wake before confirmation
ok - arm attaches to a peer watcher after child stands down and exits when peer dies
ok - arm reports FAILED and exits non-zero when no fresh watcher can be confirmed
== tests/fm-turnend-guard.test.sh ==
ok - fm_supervision_unhealthy: false with no state/*.meta at all
ok - fm_supervision_unhealthy: true with in-flight task and no beacon ever
ok - fm_supervision_unhealthy: true with in-flight task and a beacon far outside the grace window
ok - fm_supervision_unhealthy: false with in-flight task and a fresh beacon
ok - fm_supervision_status: FM_SUP_QUEUE_PENDING tracks state/.wake-queue
ok - fm-turnend-guard: silent no-op with nothing in flight
ok - fm-turnend-guard: blocks when a fresh beacon has no live watcher lock
ok - fm-turnend-guard: blocks on a dead watcher lock even when the beacon is fresh
ok - fm-turnend-guard: silent no-op with a live watcher lock and fresh beacon
ok - fm-turnend-guard: blocks on a live watcher lock with an ancient beacon
ok - fm-turnend-guard: blocks with the exact required reason in the primary when unhealthy
ok - fm-turnend-guard: blocks from active FM_HOME state, not only repo-root state
ok - fm-turnend-guard: X-mode repair reason sources the cadence config
ok - fm-turnend-guard: ignores stale repo-root state when FM_HOME is set
ok - fm-turnend-guard: uses FM_STATE_OVERRIDE ahead of FM_HOME/state
ok - fm-turnend-guard: stop_hook_active=true always allows the stop (never blocks twice in one turn)
ok - fm-turnend-guard: blocks a blind turn end in a secondmate's own home (.fm-secondmate-home no longer excludes it)
ok - fm-turnend-guard: idle-by-default - silent in a secondmate home with nothing in flight
ok - fm-turnend-guard: stop_hook_active=true allows the stop in a secondmate home (never blocks twice in one turn)
ok - fm-turnend-guard: secondmate deferred-death recovery - silent while watched, forces re-arm once the watcher exits
ok - fm-turnend-guard: inert in a secondmate's own child worktree (linked git worktree) even when unhealthy
ok - fm-turnend-guard: blocks a blind turn end in a treehouse-leased LINKED secondmate home (marker force-include)
ok - fm-turnend-guard: an invalid (empty) marker cannot spoof inclusion; linked worktree stays exempt
ok - fm-turnend-guard: a non-ASCII marker cannot spoof inclusion; linked worktree stays exempt
ok - fm-turnend-guard: inert in a crewmate/scout task worktree (linked git worktree) even when unhealthy
ok - fm-turnend-guard: fails open (never blocks) when jq is missing
ok - fm-turnend-guard: silent no-op on empty stdin
ok - fm-turnend-guard: runs well under the generous timing margin (0s)
ok - fm-turnend-guard-grok: forces one same-session resume when the shared predicate blocks
ok - fm-turnend-guard-grok: loop guard prevents a nested resume loop
ok - .claude/settings.json: Stop hook uses CLAUDE_PROJECT_DIR-anchored command
ok - .codex/hooks.json: Stop hook invokes the shared primary guard
ok - .codex/hooks.json: Stop hook uses hook process root when payload cwd is outside
ok - .codex/hooks.json: Stop hook ignores nested git root guard scripts
ok - .opencode primary plugin: session.idle forces one follow-up through the shared guard
ok - .opencode primary plugin: guard path is anchored to worktree, not directory
ok - .pi primary extension: agent_settled forces one follow-up through the shared guard
ok - .pi primary extension: no-tool and multi-tool runs each inject exactly one guard follow-up
ok - .pi primary extension: delivery failure resets the logical-run latch
ok - .grok primary hook: Stop hook invokes the grok adapter
== repeat tests/fm-pi-watch-extension.test.sh for prior rearm flake ==
-- repeat 1 --
ok - Pi primary watcher extension is tracked, self-hashing, and self-locating
ok - Pi secondmate launch wiring includes both tracked primary extensions
ok - Pi extension reports external healthy watcher output
ok - Pi custom tool returns text content and structured details
ok - Pi watcher arm distinguishes all session lock ownership states
ok - Pi process-exit cleanup listener has a bounded lifecycle
ok - Pi process-exit cleanup stops the attached arm child
ok - OpenCode primary watcher plugin has the verified TUI wake wiring
ok - OpenCode plugins have an explicit ESM boundary even under a typeless parent package
ok - OpenCode watcher plugin uses the effective FM_HOME state
ok - OpenCode watcher plugin sources the effective config
ok - OpenCode watcher plugin requires session lock ownership
ok - OpenCode watcher coordinator respects primary scope
ok - OpenCode watcher plugin rearms after a watcher wake
ok - OpenCode watcher plugin coordinates with the turn-end guard
ok - OpenCode healthy arm output does not suppress the turn-end guard
-- repeat 2 --
ok - Pi primary watcher extension is tracked, self-hashing, and self-locating
ok - Pi secondmate launch wiring includes both tracked primary extensions
ok - Pi extension reports external healthy watcher output
ok - Pi custom tool returns text content and structured details
ok - Pi watcher arm distinguishes all session lock ownership states
ok - Pi process-exit cleanup listener has a bounded lifecycle
ok - Pi process-exit cleanup stops the attached arm child
ok - OpenCode primary watcher plugin has the verified TUI wake wiring
ok - OpenCode plugins have an explicit ESM boundary even under a typeless parent package
ok - OpenCode watcher plugin uses the effective FM_HOME state
ok - OpenCode watcher plugin sources the effective config
ok - OpenCode watcher plugin requires session lock ownership
ok - OpenCode watcher coordinator respects primary scope
ok - OpenCode watcher plugin rearms after a watcher wake
ok - OpenCode watcher plugin coordinates with the turn-end guard
ok - OpenCode healthy arm output does not suppress the turn-end guard
-- repeat 3 --
ok - Pi primary watcher extension is tracked, self-hashing, and self-locating
ok - Pi secondmate launch wiring includes both tracked primary extensions
ok - Pi extension reports external healthy watcher output
ok - Pi custom tool returns text content and structured details
ok - Pi watcher arm distinguishes all session lock ownership states
ok - Pi process-exit cleanup listener has a bounded lifecycle
ok - Pi process-exit cleanup stops the attached arm child
ok - OpenCode primary watcher plugin has the verified TUI wake wiring
ok - OpenCode plugins have an explicit ESM boundary even under a typeless parent package
ok - OpenCode watcher plugin uses the effective FM_HOME state
ok - OpenCode watcher plugin sources the effective config
ok - OpenCode watcher plugin requires session lock ownership
ok - OpenCode watcher coordinator respects primary scope
ok - OpenCode watcher plugin rearms after a watcher wake
ok - OpenCode watcher plugin coordinates with the turn-end guard
ok - OpenCode healthy arm output does not suppress the turn-end guard

Evidence: Owned-lock watcher arm invocation

arm args=--restart home=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/home root=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/repo

arm args=--restart home=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/home root=/var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-e2e/repo

- Outcome: ⚠️ 1 warning across 2 runs (56m41s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

⚠️ **Test** - 1 warning

• 🚨 tests failed with exit code 1
• command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"

🔧 Fix: Stabilize watcher extension async waits
1 warning still open:

• ⚠️ I did not run the intent's lint or TypeScript typecheck portions because this testing prompt explicitly forbids linters and static analysis tools. Behavioral Pi, lock, turn-end, and rearm-flake coverage passed, but green lint/type evidence for this target commit is intentionally missing under that constraint.
• command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
• Prompt-provided baseline already completed before this round: command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
• Manual evidence check: imported .pi/extensions/fm-primary-pi-watch.ts in a fixture and called fm_watch_arm_pi for absent lock, dead holder, live other holder, and owned lock, writing /var/folders/5x/4nqprlbx0518k3ybcb1sz6gr0000gn/T/no-mistakes-evidence/01KXRV1BXD7SNEK16C5999DBC4/pi-lock-arm-transcript.txt
• bash tests/fm-pi-watch-extension.test.sh
• bash tests/fm-watcher-lock.test.sh
• bash tests/fm-turnend-guard.test.sh
• for i in 1 2 3; do bash tests/fm-pi-watch-extension.test.sh; done
• git status --short

⚠️ **Document** - 1 warning

• ⚠️ Author intent requires recording the optional session-start work-under-way nudge as a PR follow-up, but no open PR exists for branch fm/fm-piwatch-arm-stale-lock-msg; I did not create or modify external GitHub state under this documentation-only scope.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.