Skip to content

fix(tmux): resolve a target before probing it so liveness stops answering for another window - #80

Merged
Freudator86 merged 3 commits into
mainfrom
fm/fm-liveness-probe-target-fallback
Aug 6, 2026
Merged

fix(tmux): resolve a target before probing it so liveness stops answering for another window#80
Freudator86 merged 3 commits into
mainfrom
fm/fm-liveness-probe-target-fallback

Conversation

@Freudator86

Copy link
Copy Markdown
Owner

Intent

Fix a fleet-wide defect in firstmate's shared tracked code: the tmux liveness probes answer for the WRONG window, so both presence and agent-alive verdicts can describe a target that does not exist.

Root cause, reported by Tugboat 2026-07-28 and reproduced on tmux 3.4 on 2026-08-05: 'tmux display-message -p -t ' does NOT refuse a target that does not resolve. It answers for a different window (the target session's current window, or the active client's) or expands the format to empty, and returns 0 in every one of those cases. So fm_backend_target_exists reported a non-existent window as PRESENT, and fm_backend_agent_alive computed its confident alive/dead verdict from whatever the supervising pane happened to be running. The same invented window name read ALIVE on a host whose fallback pane ran claude and DEAD on a host whose fallback pane ran bash - neither is a reading of the target. This matters because bin/fm-bootstrap.sh's secondmate-liveness sweep gates a respawn on 'dead' ONLY, deliberately, so that gate could be satisfied or starved for reasons unrelated to the target: a dead secondmate never respawned on one host, or a live one at risk of duplication on another.

Deliberate decisions made while doing this work, so they are not mistakes:

  1. The fix is a single shared gate, fm_tmux_resolve_pane in bin/fm-tmux-lib.sh, built on 'tmux list-panes -t' because that command REFUSES every invented target shape (sess:no-such-window, sess:@9999, sess:win.99, %9999, @9999, bare unknown names) and resolves every real one - measured, not assumed. It deliberately does NOT parse target shapes itself.
  2. Callers read the RESOLVED pane id rather than the original target. That is intentional: the read becomes exact rather than subject to tmux's own prefix matching. It does change which target string reaches tmux, which is why several test stubs that branched on the window NAME had to be updated to branch on the pane id instead.
  3. An unresolvable target reports 'unknown', NOT 'dead'. This is deliberate and load-bearing: 'dead' means 'this pane exists and confidently holds no agent', which is a reading of the target; whether the endpoint exists at all is fm_backend_target_exists's separate question, and a gone endpoint routes to the recovery path. The liveness sweep turns 'unknown' into a reported skip and never respawns on it, so this can never duplicate a live supervisor.
  4. The gate is deliberately STRICT about a bare window name and refuses it. Cross-session bare-name lookup is firstmate's selector semantic, owned upstream by fm_backend_tmux_resolve_bare_selector, which converts a bare name to session:window before any probe runs. Guessing which session a bare name meant is exactly what this gate exists to stop a probe from doing. Verified every production caller already passes the session-qualified target recorded in state/.meta's window=.
  5. Scope was widened on purpose beyond the two probes named in the report, because the same unguarded read appeared elsewhere: fm_backend_tmux_current_path, fm_backend_tmux_send_key's preflight, fm-crew-state.sh's pane_readable, fm_tmux_composer_state, and bin/fm-context-reset.sh's own-pane read. The last one was included because its 'could not be resolved' refusal could not fire, in the one script that types a reset into whatever the target names.
  6. An enforcement test was added on purpose. The hazard was ALREADY documented in a comment beside bin/fm-spawn.sh's worktree poll while two other call sites kept the unguarded form - a comment is not enforcement. tests/fm-tmux-target-resolve.test.sh now forbids any bin/ script reading 'display-message -p -t' against an unresolved caller-supplied target, and self-checks that the rule still detects a known offender, because a rule that cannot fire is the same defect class being fixed.
  7. Two adjacent tmux sites were deliberately NOT changed and are recorded rather than silently permitted (docs/tmux-backend.md 'Adjacent sites not changed'): fm_backend_tmux_container_ensure's bare '-p' read with no -t, which resolves against the caller's own current window, and bin/fm-supervise-daemon.sh's away-mode status-line flash, which displays a message rather than producing a verdict. Neither was measured to answer wrongly, and the repo's own convention is that backend-verification docs record empirical facts, not assumptions.
  8. The other four backends were read for the same pattern and deliberately left unchanged because they are not affected: herdr, zellij, orca and cmux each address by id through an API that errors on an unknown id, and herdr additionally round-trips the echoed pane_id. That survey is recorded in bin/fm-backend.sh's header.
  9. The acceptance criterion required a control on a host whose active pane runs an agent - the exact case that produced the false ALIVE. tests/fm-backend-tmux-smoke.test.sh drives BOTH fallback shapes on a private tmux socket, using a real binary copied to the name 'claude' so tmux's own pane_current_command reports 'claude'. That copied binary is the only stand-in; the fallback behavior itself is genuine tmux. The control was verified to FAIL against the pre-fix code with exactly Tugboat's symptom.
  10. Roughly 25 test files have a one-line fake-tmux stub addition (a 'list-panes' case). That breadth is expected, not accidental: the probes now require a tmux subcommand the stubs did not model. Six suites needed more than the mechanical line because their stubs modelled dead or absent endpoints conditionally, and those conditions were mirrored rather than flattened. tests/fm-backend.test.sh's old-vs-new conformance filter and tests/fm-tangle-guard.test.sh's window-id assertion were updated for the same reason - the preflight and worktree poll legitimately changed shape.

Constraints followed: this is firstmate's shared tracked material, so .agents/skills/firstmate-coding-guidelines/SKILL.md applies - one sentence per line in tracked Markdown, plain dash never an em dash, no agent co-author, shellcheck-clean bin scripts via bin/fm-lint.sh, colocated tests named .test.sh extending the existing runner. AGENTS.md was deliberately NOT touched: by that skill's knowledge-placement decision tree this is mechanism and reference detail, so it belongs in docs/ and script headers, not in the always-loaded surface whose token cost every session pays.

Validation already run locally: bin/fm-lint.sh clean, and bin/fm-test-run.sh --all green at 127 scripts, 0 failed, 19 gate-skips, matching the pre-change baseline's 19.

What Changed

  • Added fm_tmux_resolve_pane in bin/fm-tmux-lib.sh, a shared gate built on tmux list-panes -t because that command refuses invented target shapes while display-message -p -t silently answers for a different window and still exits 0. Probes now read the resolved pane id instead of the caller's target string, and an unresolvable target reports unknown rather than dead, so the secondmate-liveness sweep skips it instead of respawning.
  • Routed every unguarded read through the gate: fm_backend_target_exists, fm_backend_agent_alive, fm_backend_tmux_current_path and fm_backend_tmux_send_key's preflight in bin/backends/tmux.sh, plus fm_tmux_composer_state, fm-crew-state.sh's pane_readable, and bin/fm-context-reset.sh's own-pane read. The other four backends address by id through APIs that error on unknown ids and were left unchanged; that survey and the two deliberately unchanged tmux sites are recorded in bin/fm-backend.sh's header and docs/tmux-backend.md.
  • Added tests/fm-tmux-target-resolve.test.sh, which forbids any bin/ script from reading display-message -p -t against an unresolved target and self-checks that the rule still detects a known offender, and tests/fm-backend-tmux-smoke.test.sh, which drives both fallback shapes against a real tmux server on a private socket and was verified to fail against the pre-fix code. Roughly 25 existing suites gained a list-panes case in their fake-tmux stub; six modelled dead or absent endpoints conditionally and had those conditions mirrored onto the pane id.

Risk Assessment

✅ Low: All three round-1 findings are correctly fixed and independently verified against real tmux, the resolver now returns the pane the target names while preserving every required refusal property, and the enforcement rule and smoke control are strengthened with regression coverage that would fail the prior defect.

Testing

Ran the two new suites (fm-tmux-target-resolve, fm-backend-tmux-smoke) plus thirteen affected existing suites - all green, no failures or gate skips - then produced the evidence the unit tests alone cannot: an end-to-end transcript driving the real bin/fm-bootstrap.sh secondmate-liveness sweep against a real tmux 3.4 server on a private socket, base commit versus fixed commit, on both host shapes from the incident report. The base code reads a non-existent secondmate window as alive/PRESENT on a claude-running host and dead/PRESENT on a bare-shell host and either never respawns or respawns against nothing; the fixed code reads unknown/ABSENT on both and the sweep reports a skip without respawning, while live endpoints still read alive and dead correctly. I additionally ran the new enforcement rule against the pre-fix bin/, where it flags all seven unguarded display-message -p -t call sites and none on the fixed tree. This change is shell/CLI only with no rendered UI surface, so CLI transcripts are the end-user-visible artifact; no screenshot applies. Transient artifacts (base checkout, private tmux sockets) were removed and the worktree is clean.

Evidence: End-to-end: bootstrap secondmate-liveness sweep, base vs fixed, on both host shapes (real tmux 3.4)
=================================================================
HOST: session 'fleet' current window is running: claude
CODE: BASE  7778ca3 (before the fix)
-----------------------------------------------------------------
state/sm1.meta records the secondmate's endpoint as:
    window=fleet:fm-sm1
live windows on this tmux server:
    fleet:firstmate  (claude)
    -> fleet:fm-sm1 does NOT exist

$ tmux display-message -p -t fleet:fm-sm1 '#{pane_current_command}' ; echo rc=$?
    claude
    rc=0  <- tmux answers, and succeeds, for a window that is not there

$ fm_backend_agent_alive tmux fleet:fm-sm1
    alive

$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    PRESENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)

    windows before sweep: firstmate 
    windows after  sweep: firstmate 
    -> the sweep did not respawn anything

=================================================================
HOST: session 'fleet' current window is running: claude
CODE: FIXED ca002c3 (after the fix)
-----------------------------------------------------------------
state/sm1.meta records the secondmate's endpoint as:
    window=fleet:fm-sm1
live windows on this tmux server:
    fleet:firstmate  (claude)
    -> fleet:fm-sm1 does NOT exist

$ tmux display-message -p -t fleet:fm-sm1 '#{pane_current_command}' ; echo rc=$?
    claude
    rc=0  <- tmux answers, and succeeds, for a window that is not there

$ fm_backend_agent_alive tmux fleet:fm-sm1
    unknown

$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    ABSENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)
    SECONDMATE_LIVENESS: secondmate sm1: skipped: liveness probe inconclusive (backend=tmux)

    windows before sweep: firstmate 
    windows after  sweep: firstmate 
    -> the sweep did not respawn anything

=================================================================
HOST: session 'fleet' current window is running: shell
CODE: BASE  7778ca3 (before the fix)
-----------------------------------------------------------------
state/sm1.meta records the secondmate's endpoint as:
    window=fleet:fm-sm1
live windows on this tmux server:
    fleet:firstmate  (sh)
    -> fleet:fm-sm1 does NOT exist

$ tmux display-message -p -t fleet:fm-sm1 '#{pane_current_command}' ; echo rc=$?
    sh
    rc=0  <- tmux answers, and succeeds, for a window that is not there

$ fm_backend_agent_alive tmux fleet:fm-sm1
    dead

$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    PRESENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)
    SECONDMATE_LIVENESS: secondmate sm1: respawn failed: error: no-mistakes gate agent must not drive the fleet (NO_MISTAKES_GATE set)

    windows before sweep: firstmate 
    windows after  sweep: firstmate 
    -> the sweep did not respawn anything

=================================================================
HOST: session 'fleet' current window is running: shell
CODE: FIXED ca002c3 (after the fix)
-----------------------------------------------------------------
state/sm1.meta records the secondmate's endpoint as:
    window=fleet:fm-sm1
live windows on this tmux server:
    fleet:firstmate  (sh)
    -> fleet:fm-sm1 does NOT exist

$ tmux display-message -p -t fleet:fm-sm1 '#{pane_current_command}' ; echo rc=$?
    sh
    rc=0  <- tmux answers, and succeeds, for a window that is not there

$ fm_backend_agent_alive tmux fleet:fm-sm1
    unknown

$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    ABSENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)
    SECONDMATE_LIVENESS: secondmate sm1: skipped: liveness probe inconclusive (backend=tmux)

    windows before sweep: firstmate 
    windows after  sweep: firstmate 
    -> the sweep did not respawn anything

=================================================================
HOST: fleet:fm-sm1 EXISTS and is running: claude
CODE: FIXED ca002c3 (after the fix)
-----------------------------------------------------------------
    fleet:firstmate  (sh)
    fleet:fm-sm1  (claude)

$ fm_backend_agent_alive tmux fleet:fm-sm1
    alive
$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    PRESENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)

=================================================================
HOST: fleet:fm-sm1 EXISTS and is running: shell
CODE: FIXED ca002c3 (after the fix)
-----------------------------------------------------------------
    fleet:firstmate  (sh)
    fleet:fm-sm1  (sh)

$ fm_backend_agent_alive tmux fleet:fm-sm1
    dead
$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT
    PRESENT

$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)
    SECONDMATE_LIVENESS: secondmate sm1: respawn failed: error: no-mistakes gate agent must not drive the fleet (NO_MISTAKES_GATE set)

=================================================================

NOTE on "respawn failed: ... NO_MISTAKES_GATE set":
This transcript was produced from inside a no-mistakes gate agent, whose own
guard refuses to let a gate agent drive a fleet. That refusal comes from
bin/fm-spawn.sh AFTER the sweep has already decided to respawn, so the line is
proof the sweep REACHED the respawn path - which is the point being shown in
both cases where it appears:
  - BASE + bare-shell host: the sweep reached respawn for a window that does not
    exist, purely because display-message answered for the supervising pane.
    In production this is the duplicate-supervisor hazard.
  - FIXED + a real fm-sm1 running a bare shell: the sweep correctly reached
    respawn for a genuinely dead endpoint it actually read.

SUMMARY
                                     agent_alive   target_exists   sweep action
  BASE,  fm-sm1 absent, host=claude    alive         PRESENT        none (dead secondmate never respawned)
  BASE,  fm-sm1 absent, host=sh        dead          PRESENT        respawn attempted on a window that is not there
  FIXED, fm-sm1 absent, host=claude    unknown       ABSENT         reported skip, no respawn
  FIXED, fm-sm1 absent, host=sh        unknown       ABSENT         reported skip, no respawn
  FIXED, fm-sm1 live, running claude   alive         PRESENT        none (correct: leave a live secondmate alone)
  FIXED, fm-sm1 live, running sh       dead          PRESENT        respawn (correct: a real dead endpoint)
Evidence: Reproducer script for the end-to-end transcript above
#!/usr/bin/env bash
# End-to-end evidence for fm-liveness-probe-target-fallback.
#
# Drives the REAL product path - bin/fm-bootstrap.sh's secondmate-liveness
# sweep - against a REAL tmux server on a private socket, for a secondmate whose
# recorded window= no longer exists. Runs the same scenario against the base
# commit and the fixed commit, on two hosts that differ only in what the
# session's own current window is running.
set -u

FIXED_ROOT=$1     # checkout of the fixed commit
BASE_ROOT=$2      # checkout of the base commit
REAL_TMUX=$(command -v tmux)
BASE_PATH=/usr/bin:/bin:/usr/sbin:/sbin
WORK=$(mktemp -d "${TMPDIR:-/tmp}/fm-liveness-e2e.XXXXXX")
SOCKET="fm-liveness-e2e-$$"

cleanup() { "$REAL_TMUX" -L "$SOCKET" kill-server >/dev/null 2>&1 || true; rm -rf "$WORK"; }
trap cleanup EXIT

# tmux shim: every bare `tmux` call from the product code lands on the private
# socket, so this never touches the host's real sessions.
SHIM="$WORK/shim"; mkdir -p "$SHIM"
cat > "$SHIM/tmux" <<SH
#!/usr/bin/env bash
exec "$REAL_TMUX" -L "$SOCKET" "\$@"
SH
chmod +x "$SHIM/tmux"

# Minimal toolchain stubs so bootstrap's unrelated read-only diagnostics stay
# quiet (mirrors tests/fm-secondmate-liveness.test.sh's make_toolchain).
STUBS="$WORK/stubs"; mkdir -p "$STUBS"
for t in node gh gh-axi chrome-devtools-axi lavish-axi quota-axi; do
  printf '#!/usr/bin/env bash\nexit 0\n' > "$STUBS/$t"; chmod +x "$STUBS/$t"
done
cat > "$STUBS/treehouse" <<'SH'
#!/usr/bin/env bash
[ "${1:-}" = get ] && [ "${2:-}" = --help ] && printf '%s\n' 'Usage: treehouse get [--lease]'
exit 0
SH
cat > "$STUBS/no-mistakes" <<'SH'
#!/usr/bin/env bash
[ "${1:-}" = --version ] && printf '%s\n' 'no-mistakes version v1.31.2 (fake)'
exit 0
SH
cat > "$STUBS/tasks-axi" <<'SH'
#!/usr/bin/env bash
case "${1:-} ${2:-}" in
  "--version ") printf '%s\n' '0.1.1' ;;
  "update --help") printf '%s\n' 'usage: tasks-axi update <id> [flags]' '  --archive-body' ;;
  "mv --help") printf '%s\n' 'usage: tasks-axi mv <id> [<id>...] --to <path-or-dir>' ;;
esac
exit 0
SH
chmod +x "$STUBS"/*

# A real binary named `claude`, so tmux's own #{pane_current_command} reports
# `claude` exactly as a real agent pane does.
AGENTBIN="$WORK/agentbin"; mkdir -p "$AGENTBIN"
cp "$(command -v sleep)" "$AGENTBIN/claude"

# A firstmate HOME holding one secondmate meta whose window= names a window
# that does not exist (the secondmate's endpoint is gone).
new_home() {
  local h=$1
  mkdir -p "$h/state" "$h/config" "$h/sm1/bin" "$h/sm1/data" "$h/sm1/state" \
           "$h/sm1/config" "$h/sm1/projects"
  touch "$h/state/.last-watcher-beat"
  printf 'codex\n' > "$h/config/crew-harness"
  printf 'sm1\n' > "$h/sm1/.fm-secondmate-home"
  printf '# Firstmate\n' > "$h/sm1/AGENTS.md"
  printf 'charter\n' > "$h/sm1/data/charter.md"
  {
    printf 'window=fleet:fm-sm1\n'
    printf 'kind=secondmate\n'
    printf 'harness=claude\n'
    printf 'home=%s\n' "$h/sm1"
  } > "$h/state/sm1.meta"
}

start_server() {  # <current-window-command-name>
  local shape=$1 cmd i
  case "$shape" in
    claude) cmd="$AGENTBIN/claude 300" ;;
    shell)  cmd="$(command -v sh)" ;;
  esac
  # kill-server is asynchronous: a new-session issued immediately after it can
  # land on the dying server ("server exited unexpectedly"). Wait it out, then
  # retry until the session is genuinely up.
  "$REAL_TMUX" -L "$SOCKET" kill-server >/dev/null 2>&1 || true
  for i in $(seq 1 50); do
    "$REAL_TMUX" -L "$SOCKET" has-session -t fleet >/dev/null 2>&1 || break
    sleep 0.1
  done
  for i in $(seq 1 50); do
    "$REAL_TMUX" -L "$SOCKET" new-session -d -s fleet -n firstmate -x 200 -y 50 "$cmd" \
      >/dev/null 2>&1 && break
    sleep 0.1
  done
  sleep 0.4
  "$REAL_TMUX" -L "$SOCKET" has-session -t fleet >/dev/null 2>&1 \
    || { echo "FATAL: could not start the private tmux server for shape=$shape"; exit 1; }
  # Assert the host really is the shape this case claims to be, or the control
  # below proves nothing.
  local got
  got=$("$REAL_TMUX" -L "$SOCKET" display-message -p -t fleet:firstmate '#{pane_current_command}')
  case "$shape:$got" in
    claude:claude|shell:sh|shell:dash|shell:bash) : ;;
    *) echo "FATAL: current window reports '$got', expected shape '$shape'"; exit 1 ;;
  esac
}

run_case() {  # <label> <root> <current-window-shape>
  local label=$1 root=$2 shape=$3 slug=$4
  local home out windows_before windows_after
  home="$WORK/home-$slug"
  rm -rf "$home"; new_home "$home"
  start_server "$shape"

  echo "================================================================="
  echo "HOST: session 'fleet' current window is running: $shape"
  echo "CODE: $label"
  echo "-----------------------------------------------------------------"
  echo "state/sm1.meta records the secondmate's endpoint as:"
  grep '^window=' "$home/state/sm1.meta" | sed 's/^/    /'
  echo "live windows on this tmux server:"
  "$REAL_TMUX" -L "$SOCKET" list-windows -t fleet -F '    #{session_name}:#{window_name}  (#{pane_current_command})'
  echo "    -> fleet:fm-sm1 does NOT exist"
  echo
  echo "\$ tmux display-message -p -t fleet:fm-sm1 '#{pane_current_command}' ; echo rc=\$?"
  "$REAL_TMUX" -L "$SOCKET" display-message -p -t fleet:fm-sm1 '#{pane_current_command}' 2>&1 | sed 's/^/    /'
  echo "    rc=$?  <- tmux answers, and succeeds, for a window that is not there"
  echo
  echo "\$ fm_backend_agent_alive tmux fleet:fm-sm1"
  PATH="$SHIM:$STUBS:$BASE_PATH" bash -c \
    '. "$1/bin/fm-backend.sh"; fm_backend_source tmux; fm_backend_agent_alive tmux fleet:fm-sm1; echo' \
    _ "$root" | sed 's/^/    /'
  echo
  echo "\$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT"
  PATH="$SHIM:$STUBS:$BASE_PATH" bash -c \
    '. "$1/bin/fm-backend.sh"; fm_backend_source tmux; fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT' \
    _ "$root" | sed 's/^/    /'
  echo
  windows_before=$("$REAL_TMUX" -L "$SOCKET" list-windows -t fleet -F '#{window_name}' | sort | tr '\n' ' ')
  echo "\$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)"
  out=$(PATH="$SHIM:$STUBS:$BASE_PATH" TMUX='' FM_BACKEND=tmux FM_HOME="$home" \
          "$root/bin/fm-bootstrap.sh" 2>&1)
  printf '%s\n' "$out" | grep -E 'SECONDMATE_LIVENESS' | sed 's/^/    /' \
    || echo "    (sweep printed nothing about sm1)"
  windows_after=$("$REAL_TMUX" -L "$SOCKET" list-windows -t fleet -F '#{window_name}' | sort | tr '\n' ' ')
  echo
  echo "    windows before sweep: $windows_before"
  echo "    windows after  sweep: $windows_after"
  if [ "$windows_before" = "$windows_after" ]; then
    echo "    -> the sweep did not respawn anything"
  else
    echo "    -> the sweep ACTED on the target (windows changed)"
  fi
  echo
}

# --- positive control: the endpoint the meta names actually EXISTS ------------
#
# "unknown for everything" would be a trivially safe answer rather than a
# correct one, so the fixed sweep is also driven against a real fm-sm1 window.
run_present_case() {  # <label> <root> <secondmate-window-command> <slug>
  local label=$1 root=$2 shape=$3 slug=$4
  local home out
  home="$WORK/home-$slug"
  rm -rf "$home"; new_home "$home"
  start_server shell
  case "$shape" in
    claude) "$REAL_TMUX" -L "$SOCKET" new-window -d -t fleet -n fm-sm1 "$AGENTBIN/claude 300" ;;
    shell)  "$REAL_TMUX" -L "$SOCKET" new-window -d -t fleet -n fm-sm1 "$(command -v sh)" ;;
  esac
  sleep 0.4

  echo "================================================================="
  echo "HOST: fleet:fm-sm1 EXISTS and is running: $shape"
  echo "CODE: $label"
  echo "-----------------------------------------------------------------"
  "$REAL_TMUX" -L "$SOCKET" list-windows -t fleet -F '    #{session_name}:#{window_name}  (#{pane_current_command})'
  echo
  echo "\$ fm_backend_agent_alive tmux fleet:fm-sm1"
  PATH="$SHIM:$STUBS:$BASE_PATH" bash -c \
    '. "$1/bin/fm-backend.sh"; fm_backend_source tmux; fm_backend_agent_alive tmux fleet:fm-sm1; echo' \
    _ "$root" | sed 's/^/    /'
  echo "\$ fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT"
  PATH="$SHIM:$STUBS:$BASE_PATH" bash -c \
    '. "$1/bin/fm-backend.sh"; fm_backend_source tmux; fm_backend_target_exists tmux fleet:fm-sm1 && echo PRESENT || echo ABSENT' \
    _ "$root" | sed 's/^/    /'
  echo
  echo "\$ bin/fm-bootstrap.sh   (session start; secondmate-liveness sweep)"
  out=$(PATH="$SHIM:$STUBS:$BASE_PATH" TMUX='' FM_BACKEND=tmux FM_HOME="$home" \
          "$root/bin/fm-bootstrap.sh" 2>&1)
  printf '%s\n' "$out" | grep -E 'SECONDMATE_LIVENESS' | sed 's/^/    /' \
    || echo "    (sweep said nothing about sm1 - a live secondmate is handled silently)"
  echo
}

run_case "BASE  7778ca3 (before the fix)" "$BASE_ROOT" claude base-claude
run_case "FIXED ca002c3 (after the fix)"  "$FIXED_ROOT" claude fixed-claude
run_case "BASE  7778ca3 (before the fix)" "$BASE_ROOT" shell base-shell
run_case "FIXED ca002c3 (after the fix)"  "$FIXED_ROOT" shell fixed-shell
run_present_case "FIXED ca002c3 (after the fix)" "$FIXED_ROOT" claude fixed-present-claude
run_present_case "FIXED ca002c3 (after the fix)" "$FIXED_ROOT" shell fixed-present-shell
echo "================================================================="
Evidence: The new enforcement rule run against the pre-fix bin/ (it flags all 7 real offenders) and the fixed bin/ (none)

The enforcement rule from tests/fm-tmux-target-resolve.test.sh, run against the PRE-FIX bin/ (base commit 7778ca3). Every line below is a real call site that read tmux display-message against an unresolved caller-supplied target. bin/backends/tmux.sh:45: tmux display-message -p -t "$1" '#{pane_id}' >/dev/null bin/backends/tmux.sh:101: tmux display-message -p -t "$1" '#{pane_current_path}' 2>/dev/null bin/backends/tmux.sh:137: tmux display-message -p -t "$1" '#{pane_current_command}' 2>/dev/null bin/fm-backend.sh:673: tmux display-message -p -t "$target" '#{pane_id}' >/dev/null 2>&1 bin/fm-context-reset.sh:209:TARGET=$(tmux display-message -p -t "$PANE" '#{session_name}:#{window_index}.#{pane_index}' 2>/dev/null || true) bin/fm-crew-state.sh:247: tmux) tmux display-message -p -t "$1" '#{pane_id}' >/dev/null 2>&1 ;; bin/fm-tmux-lib.sh:104: cy=$(tmux display-message -p -t "$target" '#{cursor_y}' 2>/dev/null) || { printf 'unknown'; return 0; } --- the same rule against the FIXED bin/ (ca002c3) --- (no offenders)

The enforcement rule from tests/fm-tmux-target-resolve.test.sh, run against
the PRE-FIX bin/ (base commit 7778ca3). Every line below is a real call site
that read tmux display-message against an unresolved caller-supplied target.

bin/backends/tmux.sh:45:  tmux display-message -p -t "$1" '#{pane_id}' >/dev/null
bin/backends/tmux.sh:101:  tmux display-message -p -t "$1" '#{pane_current_path}' 2>/dev/null
bin/backends/tmux.sh:137:  tmux display-message -p -t "$1" '#{pane_current_command}' 2>/dev/null
bin/fm-backend.sh:673:      tmux display-message -p -t "$target" '#{pane_id}' >/dev/null 2>&1
bin/fm-context-reset.sh:209:TARGET=$(tmux display-message -p -t "$PANE" '#{session_name}:#{window_index}.#{pane_index}' 2>/dev/null || true)
bin/fm-crew-state.sh:247:    tmux) tmux display-message -p -t "$1" '#{pane_id}' >/dev/null 2>&1 ;;
bin/fm-tmux-lib.sh:104:  cy=$(tmux display-message -p -t "$target" '#{cursor_y}' 2>/dev/null) || { printf 'unknown'; return 0; }

--- the same rule against the FIXED bin/ (ca002c3) ---
(no offenders)
Evidence: Verdict matrix distilled from the end-to-end transcript
agent_alive target_exists sweep action
BASE, fm-sm1 absent, host=claude alive PRESENT none (dead secondmate never respawned)
BASE, fm-sm1 absent, host=sh dead PRESENT respawn attempted on a window that is not there
FIXED, fm-sm1 absent, host=claude unknown ABSENT reported skip, no respawn
FIXED, fm-sm1 absent, host=sh unknown ABSENT reported skip, no respawn
FIXED, fm-sm1 live, running claude alive PRESENT none (correct: leave a live secondmate alone)
FIXED, fm-sm1 live, running sh dead PRESENT respawn (correct: a real dead endpoint)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 4 issues found → auto-fixed ✅
  • 🚨 bin/fm-tmux-lib.sh:116 - fm_tmux_resolve_pane resolves a pane-qualified target to the window's ACTIVE pane, not the pane the target names. Verified on tmux 3.4 with a private socket: tmux list-panes -t %0 (an inactive pane) and tmux list-panes -t sess:win.0 both list every pane of the containing window (%0 active=0, %1 active=1), because list-panes takes a target-WINDOW; the loop at lines 113-119 then prefers pane_active=1. The pre-change form was exact: display-message -p -t %0 &#39;#{pane_id}&#39; returns %0. Two reachable paths: (1) bin/fm-context-reset.sh:209 takes PANE=$TMUX_PANE (a %N pane id), lines 216-218 derive TARGET from RESOLVED_PANE, and line 244 types /clear into TARGET - in any split window where firstmate's own pane is not the active one, the reset is typed into a neighbouring pane, defeating that script's stated premise that a reset must never be typed into a pane it cannot identify. (2) bin/fm-supervise-daemon.sh resolves its supervisor target from $TMUX_PANE and calls fm_backend_composer_state -> fm_tmux_composer_state (bin/fm-tmux-lib.sh:166-169), which now reads cursor_y and capture-pane from the active pane while the injection still goes to the original %N, decoupling the composer-emptiness safety check from the pane being typed into. This is the same answer-for-the-wrong-pane class the change exists to close, narrowed from window scope to pane scope. Flagged ask-user because the repair collides with deliberate decision 1 (resolver is list-panes-only and parses no target shapes) and with tests/fm-tmux-target-resolve.test.sh:244, which asserts the resolver body contains no display-message. Suggested repair: keep list-panes as the refusal gate, then read display-message -p -t &#34;$target&#34; &#39;#{pane_id}&#39; (it cannot fall back once list-panes has proven the target resolves) and verify the returned id appears in the listing; relax that assertion to "list-panes gates before any read".
  • ⚠️ tests/fm-backend-tmux-smoke.test.sh:225 - The bare-shell half of the required two-shape control passes the literal bash as the expected fallback command, but tmux new-window -d -t $SESSION -n shell-fallback starts the host's default shell. On a zsh host (macOS default, or any user with a non-bash login shell) line 202 reads 'zsh' and the precondition at 203-204 fails on correct code, in the one suite that proves the fix against real tmux. fm_backend_tmux_agent_alive already classifies zsh/fish/dash as dead (bin/backends/tmux.sh:218), so only the literal is wrong: read the fallback window's own #{pane_current_command} after creating it and pass that as expected_cmd.
  • ⚠️ tests/fm-tmux-target-resolve.test.sh:207 - The anti-recurrence rule exempts any display-message -p -t whose argument is spelled "$pane", "$PANE", "$RESOLVED_PANE" or "$TMUX_PANE", so the guard is satisfied by variable naming rather than by actual resolution. A future pane=$1; tmux display-message -p -t &#34;$pane&#34; &#39;#{pane_current_command}&#39; - an unresolved caller-supplied target, the exact incident shape - is filtered out at line 208 and the test passes. The self-check only proves the rule fires on the $target spelling. Requiring that the exempt variable is assigned from fm_tmux_resolve_pane (or from $TMUX_PANE) in the same file would make the rule match its stated contract.
  • ℹ️ tests/fm-backend-tmux-smoke.test.sh:203 - The control hard-asserts that tmux still exhibits the display-message fallback. If a future tmux refuses the invented target, raw is empty and the suite fails even though firstmate's code is correct. The test's own comment records this as deliberate (assert rather than assume, to avoid a vacuous pass); noted as an acknowledged tradeoff, not a defect. A skip-with-reason on a non-fallback tmux would preserve both properties.

🔧 Fix: resolve a target to the pane it names, not the active one
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • bin/fm-test-run.sh tests/fm-tmux-target-resolve.test.sh tests/fm-backend-tmux-smoke.test.sh - the two new suites, including the real-tmux smoke control on a private socket (14 + 11 assertions, all pass)
  • bin/fm-test-run.sh tests/fm-backend.test.sh tests/fm-secondmate-liveness.test.sh tests/fm-context-reset.test.sh tests/fm-crew-state.test.sh tests/fm-tangle-guard.test.sh tests/fm-bootstrap.test.sh tests/fm-send-strict.test.sh - the suites whose stubs needed more than the mechanical list-panes line
  • bin/fm-test-run.sh tests/fm-fleet-snapshot-view.test.sh tests/fm-composer-ghost.test.sh tests/fm-tmux-submit-busy.test.sh tests/fm-gate-refuse.test.sh tests/fm-session-start.test.sh tests/fm-secondmate-sync.test.sh - a representative slice of the mechanically-stubbed suites
  • Manual end-to-end: bash e2e-liveness-sweep.sh &lt;fixed-checkout&gt; &lt;base-checkout&gt; - drives the real bin/fm-bootstrap.sh secondmate-liveness sweep against a real tmux 3.4 server on a private socket, for base vs fixed code, across a claude-running and a bare-shell current window, plus live-endpoint positive controls
  • Manual: ran the committed enforcement scanner (unguarded_display_message_reads lifted verbatim from tests/fm-tmux-target-resolve.test.sh) against the base commit's bin/ and the fixed bin/ to prove the rule detects the real defect, not only its synthetic probes
⚠️ **Document** - 1 info
  • ℹ️ docs/tmux-backend.md:232 - Judgment call, left unchanged: docs/tmux-backend.md:232's classifier bullet still reads "unknown - anything else, including an unreadable pane" and does not name an unresolvable target. The fact is stated authoritatively 40 lines earlier in the same document ("A target that does not resolve reports unknown, never dead"), so adding it to the bullet would duplicate the owner sentence rather than fix a wrong one. If the bullet list is meant to be the standalone contract, that sentence belongs there and the prose above should shrink to a pointer.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Coditan-XO added 3 commits August 5, 2026 14:00
…ering for another window

`tmux display-message -p -t <target>` does not refuse a target that does not
resolve. It answers for a different window - the target session's current
window, or the active client's - and returns 0 in every case. Both tmux
liveness probes were built on that exit status, so their verdicts described the
supervising pane and never touched the target at all.

Reported by Tugboat 2026-07-28 and reproduced on tmux 3.4 (2026-08-05): three
invented window names all returned a real pane id belonging to someone else.
`fm_backend_target_exists` therefore reported a non-existent window as present,
and `fm_backend_agent_alive` computed its confident alive/dead verdict from
whatever the fallback pane happened to be running - so the same invented name
read ALIVE on a host whose fallback pane ran claude and DEAD on one whose
fallback pane ran bash. That host-dependence is why a fleet-wide defect looked
like a local environment quirk. The secondmate-liveness sweep gates a respawn on
`dead` only, so that gate could be satisfied or starved for reasons unrelated to
the target: a dead secondmate never respawned on one host, a live one at risk of
duplication on another.

`tmux list-panes -t` refuses every invented shape (`sess:no-such-window`,
`sess:@9999`, `sess:win.99`, `%9999`, `@9999`, bare unknown names) and resolves
every real one, so it needs no target-shape parsing of our own.
`fm_tmux_resolve_pane` (bin/fm-tmux-lib.sh) is now the one gate every read of a
caller-supplied target passes; callers read the pane id it returns, which is
exact rather than subject to tmux's own prefix matching. An unresolvable target
reports unknown rather than dead: whether the endpoint exists at all is
`fm_backend_target_exists`'s question, and the sweep turns unknown into a
reported skip instead of a silent wrong verdict.

The hazard was already documented in a comment beside bin/fm-spawn.sh's worktree
poll while two other call sites kept the unguarded form, so the rule is now
enforced by a test rather than by a comment: no bin/ script may read
`display-message -p -t` against an unresolved caller-supplied target, and the
rule self-checks that it still detects a known offender.

Gated: fm_backend_tmux_current_command (and so agent_alive),
fm_backend_tmux_current_path, fm_backend_tmux_send_key's preflight,
fm_backend_target_exists's tmux branch, fm-crew-state.sh's pane_readable,
fm_tmux_composer_state, and fm-context-reset.sh's own-pane read - the last
because its "could not be resolved" refusal could not fire, in the one script
that types a reset into whatever the target names.

Measured, not assumed: capture-pane and send-keys refuse an unresolvable target
correctly, so no pane content was ever misread and no keystroke misdelivered.
The other four backends were read for the same pattern and are not affected -
herdr, zellij, orca, and cmux each address by id through an API that errors on
an unknown one, and herdr additionally round-trips the echoed pane_id. Two
adjacent tmux sites are recorded rather than changed, because neither was
measured to answer wrongly here: container_ensure's bare `-p` read of its own
current window, and the away-mode status-line flash.

Tests: the real-tmux control drives BOTH fallback shapes on a private socket,
including an agent-named binary in the fallback pane - the exact case that
produced the false ALIVE. It fails against the pre-fix code with that symptom.
@Freudator86
Freudator86 merged commit cb55c08 into main Aug 6, 2026
10 checks passed
@Freudator86
Freudator86 deleted the fm/fm-liveness-probe-target-fallback branch August 6, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant