feat: strengthen crew dispatch and recovery - #1986
Closed
Ospeto wants to merge 14 commits into
Closed
Conversation
* fix(bin): preserve full task contract in no-mistakes intent (kunchenguid#1447) * Preserve task contract in no-mistakes intent * no-mistakes(review): Preserve complete current task contract in no-mistakes intent * fix(bin): parse punctuated secondmate registry entries safely (kunchenguid#1452) * fix: centralize secondmate registry parsing * no-mistakes(review): Centralize secondmate registry binding validation * no-mistakes(review): Harden registry EOF and symlink validation * no-mistakes(review): Reject unreadable registries before parsing * no-mistakes(document): Document punctuation-safe secondmate registry validation * no-mistakes: apply CI fixes * feat: route brief research and specialist reviews * fix: keep review guidance within delivery authority * fix: authorize specialist review and inspect full range * test: make gamma delivery mode explicit * fix: reject malformed project mode tokens * feat: standardize crewmate specialist workflow * fix: tighten brief and mode contracts --------- Co-authored-by: Kun Chen <3233006+kunchenguid@users.noreply.github.com>
* feat(dispatch): add role-aware quota dispatch and recovery * no-mistakes(document): Document dispatch, recovery, and registry contracts * no-mistakes(test): Added missing pi harness fixture; focused dispatch test passes * test(dispatch): preserve delivery contract after rebase * fix(spawn): remove Perl timeout dependency * fix(spawn): preserve native preflight bounds * test(spawn): cover compatible timeout selection * fix(spawn): require descendant-safe timeout
* fix(bin): preserve full task contract in no-mistakes intent (#1447)
* Preserve task contract in no-mistakes intent
* no-mistakes(review): Preserve complete current task contract in no-mistakes intent
* fix(bin): parse punctuated secondmate registry entries safely (#1452)
* fix: centralize secondmate registry parsing
* no-mistakes(review): Centralize secondmate registry binding validation
* no-mistakes(review): Harden registry EOF and symlink validation
* no-mistakes(review): Reject unreadable registries before parsing
* no-mistakes(document): Document punctuation-safe secondmate registry validation
* no-mistakes: apply CI fixes
* feat(bin): add durable process-event supervision (#1483)
* feat(procevent): supervise long-polling sources into durable events
Firstmate had no way to wait on a blocking external process without holding
a conversational turn. Add a domain-neutral process-to-event runner plus a
thin adapter around the currently published `lavish-axi poll` interface:
canonical physical source identity, one machine-wide owner per source, direct
argv execution, and durable 0600 result capture before any event referencing
it is published on the existing wake queue. No second notifier, no polling
control plane, and no retry machinery.
A captured result with no durable handled acknowledgement stays eligible for
bounded re-announcement across any number of drains and restarts. Draining a
wake before acting on it and then starting a replacement session resurfaces
the same exact source and sequence, and never puts result payload text in an
event line. `fm-procevent.sh handled <source-id> <sequence>` is the only thing
that stops re-announcement: generation-keyed, private, path-safe, durable, and
atomically idempotent, so a paired external effect gated on its first-time
versus repeat report is never authorized twice.
An acknowledgement is refused unless matching captured result and adapter
records already exist, so a premature or mistyped call cannot suppress a
future result.
The source side is unchanged and still lossy: the published poll clears
feedback destructively before returning it, so a result lost in that window
is unrecoverable. This is never at-least-once, no-loss, or lossless, and the
handled acknowledgement is not a generic exactly-once effect either - a crash
between an external effect and its acknowledgement can still repeat that
effect on replay.
Integrate registered sources with watcher supervision, the guards, and
recoverable secondmate teardown across nested homes, and cover source
identity, lifecycle races, supervision, restart handling, and cleanup safety
with regressions.
* no-mistakes(review): Prevent Lavish prompt text from spoofing missing sessions
* no-mistakes(review): Serialize publication and secure handled acknowledgements
* no-mistakes(document): Document hardened process-event acknowledgement guarantees
* fix(procevent): never reclaim a source whose owned group still runs
A runner is its own process group leader and starts the blocking source in
that group, but the claim records only the leader PID and its identity. If the
leader died while the source child kept running, the missing PID was
classified stale: reconciliation released the claim and started a second
runner while the old blocking source was still consuming the same canonical
source. For the Lavish adapter that means two destructive long polls racing on
one review session, so it is not harmless process litter. It also contradicted
the documented promise that ownership is never released until the whole group
is gone.
Ownership state now distinguishes a generation that is really gone from one
whose leader crashed with its group still alive. Reconcile stops that
surviving group and releases its exact generation before starting any
replacement, and keeps the claim for a later cycle when it cannot prove the
group stopped or another home owns it. Acquisition and `start` treat the same
state as held rather than reclaimable.
Signalling that group is safe precisely because only an absent leader reaches
this state. A reused PID leaves the leader alive, so the identity comparison
still classifies it stale or uncertain and no group signal follows, which
keeps the existing PID-reuse refusal intact.
Add a public-interface regression for the exact crash cut - SIGKILL only the
leader, prove the child group survives, reconcile, and prove the old group is
gone with no second source running - plus its counterexample that a generation
with no leader and no surviving group is still reclaimed. Update the runner
help, operating documentation, skill, and verification record where they
described reclaim in terms of the leader alone.
* no-mistakes(review): Enforce runner group ownership and detect poller overlap
* no-mistakes(review): Isolate runner groups from unrelated caller processes
* no-mistakes(document): Document isolated process-event runner launch
* no-mistakes(lint): Suppress Perl literal ShellCheck false positive
* fix(bin): retire terminal process events and surface queued wakes (#1500)
* fix(bin): deliver process-event results and retire ended sources
Two defects reproduced during a real Lavish adapter session.
One human `Send & End` produced four captured results: the real feedback,
then recurring empty ended sessions. The generic runner had no way to learn
a source was finished, so every reconcile restarted a poll that returned
immediately. The runner now asks the source's own adapter -
`fm-procevent-<adapter>.sh terminal <result-file>` - and on exit 0 alone
re-proves ownership, drops the registration, and releases its own claim
under one source boundary. Terminal knowledge stays adapter-owned: for
Lavish that is an ended session, a missing session, and the final feedback
delivery the published poll marks with `session_ended`. An adapter with no
terminal command keeps its source armed exactly as before. Capture before
publication, captured-result durability, queued wake durability, bounded
re-announcement, handled deduplication, one-owner ownership, and explicit
idempotent retirement are all unchanged.
A captured result queued its `check` wake durably, but a healthy watcher
with a fresh beacon never delivered it; the result surfaced only after a
manual drain. Publication happens outside the watcher (in the runner) or
unconditionally (in reconcile), so the watcher had no newly actionable
signal to report and never reached its rewake path. It now reports a
queued-but-unsurfaced process-event record through the same actionable exit
every other wake uses, deduplicated by the same `.seen-*` marker discipline
the signal scan uses, so the record is always durable before it is
suppressed. The durable queue remains the authority and no second notifier,
poller, timer, queue, or adapter-specific wake path is added.
Regressions cover both, driven end to end: an armed Lavish source against a
stand-in for the published poll polls once, captures once, publishes one
distinct event, and retires itself; two fixture adapters prove the terminal
decision follows the adapter alone; and a real capture plus a real watcher
prove one proactive wake before any drain, with no duplicate wake while the
record stays queued or after it is acknowledged.
* no-mistakes(review): Harden process-event retirement and proactive delivery
* no-mistakes(review): Route process-event delivery through shared wake owner
* no-mistakes(document): Clarify process-event delivery and retirement documentation
* no-mistakes(lint): Fix ShellCheck control-flow warnings
* no-mistakes(lint): Fix wake output status lint warning
* perf: shard portable serial tests across CI runners (#1544)
* perf(ci): shard the portable serial behavior lane across runners
The Behavior portable serial job ran all 69 scripts of the serial
remainder on one runner. The measured serial sum on run 30725985757 was
1143762 ms (19m04s) against a 20-minute timeout, so the job intermittently
reached the cap and was cancelled with every step passing. Setup is only
about 7s, so the cost is entirely test wall time.
Split the lane into four separate-runner shards. Each shard is still
strictly serial, and separate runners mean no two of these stateful
scripts ever share a machine, so the split needs no concurrency isolation
proof. Assignment is longest-processing-time bin packing over measured
per-script duration hints, balancing every shard to 285941 ms (~4m46s) of
expected work, and the timeout tightens from 20 to 15 minutes.
bin/fm-test-run.sh owns the shard count and refuses a lane whose "ofN"
disagrees with it, while ci.yml derives the same count from
strategy.job-total rather than a literal, so changing it in either file
alone fails the lane loudly instead of leaving part of the required suite
unrun. --check-coverage additionally proves the shards are non-empty,
disjoint, and exactly equal to the serial lane. No test is weakened,
skipped, or removed.
Also replace the wall-clock sleeps in the --jobs scheduler test fixture
with an explicit signal handshake between the fixtures. The old
0.5s-versus-0.05s race failed on a loaded machine; the handshake passes
under sustained CPU saturation.
* no-mistakes(review): Correct portable serial shard balance evidence
* no-mistakes(document): Document portable serial shard evidence accurately
* fix(bin): correct session lock and attached watcher supervision (#1545)
* fix(bin): identify harness sessions by path and report delivered wakes
Two supervision faults, both reported by a contributor and both open on the
default branch.
Fault 1: the Stop auto-arm never claims the home. fm_harness_ancestry_pid()
matched only the basename of `ps -o comm=`, and Claude Code's native installer
names the per-session executable by its version (.../share/claude/versions/
2.1.220), so that basename identifies nothing. Three real failure shapes follow:
a version-named session is missed entirely and the hook exits 0 with the epoch
never written (unconditional on Linux, where procps reports the kernel exec name
and ignores argv[0]); a claude-named daemon that directly parents sessions wins
the outermost-contiguous-claude rule ahead of the session itself; and a session
that is both version-named and daemon-parented has its live lock reclaimed as
stale and rewritten to the shared daemon pid, corrupting the home's ownership
record.
Harness identity now also reads whole components of the executable path and of
argv[0], which is what both platforms still carry. Matching whole components
only keeps that widening safe: bin/fm-claude-stop-autoarm.sh and ~/.claude/hooks
scripts have no "claude" component. Ownership is then decided against the
session's whole contiguous harness ancestry rather than one chosen pid, which is
the honest form of the question the library already documents ("does the current
process descend from that same harness?"). That subsumes the outermost-pid rule
for Claude's nested bg-spare worker chain instead of reverting it, and lets a
daemon-parented session recognize its own lock. Lock acquisition still writes the
outermost pid of the run, the only pid that lives as long as the session.
Fault 2: an attached arm reports a delivered cycle as FAILED. The watcher prints
its one reason line to its own stdout, so only the arm that forked it can read
that line; an arm that attached observes nothing but a released lock and called a
completely successful cycle "cycle ended without an actionable reason". No
supervision event was lost - the durable queue held it - but every harness
protocol reads that line as "supervision is down" and directs a manual re-arm.
The arm now resolves an unobservable close against the durable wake queue, which
records every wake before the watcher prints it and whose sequence counter never
rewinds, not even across a drain. A cycle the queue proves delivered a wake
reports that wake and exits 0; a cycle whose records a handling turn already
drained reports the delivery without inventing a reason line; only a cycle that
delivered nothing is still the typed nonzero failure. Fixing it in the arm covers
codex, opencode, pi, grok and kimi, not just the Claude Stop path.
Regressions: tests/fm-session-lock-ancestry.test.sh pins both platforms' ps
semantics behind a deterministic process table and runs the real Stop auto-arm in
version-named, daemon-parented, and combined real process trees, each orphaned so
the walk cannot escape the fixture. tests/fm-watch-arm.test.sh drives a real
watcher and a real attached arm through a real wake. Every fault case fails on
the previous code.
* no-mistakes(review): Bind watcher delivery records to process identity
* no-mistakes(review): Return validated watcher identity atomically
* no-mistakes(review): Track watcher successors by PID and identity
* no-mistakes(document): Consolidate watcher arm-cycle documentation ownership
* fix(bin): harden Claude supervision auto-arm recovery (#1495)
* fix(supervision): harden Claude auto-arm failure handling
* no-mistakes(review): Guarantee automatic retry after Claude auto-arm failures
* no-mistakes(review): Gate attended fail-open on verified supervision failure
* no-mistakes(document): Document Claude auto-arm retry and guard scope
* no-mistakes: apply CI fixes
* fix(supervision): make Claude fail-open progression monotonic
* no-mistakes(review): Preserve auto-arm failure episodes until verified watcher recovery
* no-mistakes(review): Linearize auto-arm failure progression across existing locks
* no-mistakes(review): Linearize positive recovery across shared failure episode lock
* no-mistakes(review): Scope Claude recovery contention to Claude guard mode
* no-mistakes(document): Align supervision auto-arm documentation
* no-mistakes(review): Preserve actionable wakes despite healthy successors
* no-mistakes(document): Refresh supervision auto-arm documentation
* feat(bin): require an explicit per-task delivery contract (#1563)
* feat(bin): require an explicit ship delivery mode in fm-brief
A ship brief's definition of done was shaped by a silent per-project registry
lookup, so an adjusted brief and the task's recorded delivery could disagree and
no one had to decide anything per task.
fm-brief now requires --mode on ship scaffolds, validates it against the closed
set, refuses the conditional no-mistakes-prod-only registry policy as a task
mode, and records the choice as a fixed machine-readable "Delivery contract:
mode=<mode>" line that fm-spawn can check. --mode is refused on scout and
secondmate scaffolds, and --yolo is refused outright because the worker never
owns approval decisions.
* feat(bin): require an explicit ship delivery contract at spawn and promotion
fm-spawn resolved every ship and scout task's mode and yolo from the project
registry, so the delivery posture was never a per-task decision and could
contradict the brief the worker was about to follow.
fm-spawn now requires --mode and --yolo on ship spawns, validates both against
their closed sets, and reads the brief's recorded delivery contract line and
refuses a mismatch before any endpoint exists; a brief scaffolded before that
line existed warns once and launches on the flag. A batch carries one shared
contract that each pair still checks against its own brief. Scout and secondmate
spawns refuse the flags, and a scout now records no mode or yolo at all, which
teardown and the snapshot already tolerate. When the explicit mode carries less
rigor than the project's standing posture, a deviation notice is printed and the
spawn continues, so the registry stays advisory rather than an enforced default.
fm-promote requires the same two flags, because a scout carries no posture to
inherit, and writes them into the task record with the kind flip.
fm-project-mode keeps its one registry parser for the mechanical consumers that
have no task in hand, accepts the conditional no-mistakes-prod-only annotation
and maps it to its most rigorous leg for them, and grows --raw so the deviation
notice can tell a conditional policy apart from a flat mode.
* docs: record the explicit per-task delivery contract
AGENTS.md section 7 now owns how each ship task's mode and yolo are resolved at
intake, including the surface classification for a no-mistakes-prod-only project
and the unregistered-project fallback, and the project-management skill defines
that conditional policy as a registration-time posture with its defaults and
initialization consequences. The registry blurb, script table, and architecture
section follow: the registry records the captain's standing posture, and task
delivery is decided per task and passed explicitly.
* test: pass ship delivery flags per call site in the Herdr launcher e2e
The shared spawn helper also launches a secondmate, which refuses the flags, so
the contract belongs at each ship call site rather than inside the helper.
* test: pass the ship delivery contract in the secondmate suites
Both suites scaffold or spawn an ordinary ship task as the control case for a
secondmate assertion, so each needs the explicit contract the ship path now
requires.
* feat(bin): support remote secondmate homes (#1576)
* Add generic remote secondmate transport
* Add routed remote secondmate replies
* Add remote outbox backlog handoff
* Integrate remote secondmate lifecycle
* no-mistakes(review): Fix remote snapshot and handoff races
* no-mistakes(review): Serialize remote home provisioning transactions
* no-mistakes(review): Harden remote lifecycle transaction boundaries
* no-mistakes(review): Serialize remote lifecycle mutations and fail closed
* no-mistakes(review): Close remote lifecycle and file race windows
* no-mistakes(review): Serialize remote reply retirement and inheritance
* no-mistakes(review): Harden remote transfer integrity and recovery
* no-mistakes(review): Serialize remote respawn with registry retirement
* no-mistakes(document): Document remote bootstrap convergence accurately
* no-mistakes(document): Clarify skipped remote secondmate mutations
* no-mistakes(lint): Resolve remote script ShellCheck warnings
* no-mistakes: apply CI fixes
* feat: add per-task trace context propagation (#995)
* feat(spawn): propagate a native W3C traceparent to spawned agents
Add a default-off capability that resolves one W3C traceparent for a task,
injects it into the agent's pane shell as the TRACEPARENT environment
variable immediately before launch, and records the identical value as
traceparent= in state/<id>.meta, so an external observer that explicitly
reads that env value or meta field can correlate a worker, a Secondmate, and
their nested children into one trace with no collector, storage, UI, or
vendor coupling.
TRACEPARENT as an environment variable is a firstmate convention carrying a
W3C-formatted value: W3C Trace Context standardizes the header, not an env
var, and OpenTelemetry SDKs do not read it automatically, so a downstream
must consume it deliberately; this feature parents no SDK span by itself.
Identity is per task, not per spawn: the carrier is minted with random ids on
the first spawn, adopted as a child (fresh span, same trace) for a nested
spawn whose parent already holds one, and reused verbatim from the meta on
relaunch, so a task keeps one stable logical identity across restarts. A
malformed or all-zero inherited value is treated as absent and roots a fresh
trace. A new root is sampled (01) - a sampling decision a downstream
parent-based sampler honors, not a guarantee that any collector stores a
span, and firstmate emits no spans; a child preserves the inherited flag.
Trust boundary: a firstmate-minted root is random and reads no prompt, path,
task prose, credential, or arbitrary environment key. An inherited
TRACEPARENT is opaque caller-controlled data - up to 24 bytes of id passed
through after syntax validation - so whoever set it controls those bytes, a
bounded fixed-width channel rather than a general content or secret channel.
The feature adds no OTEL_* variable, no tracestate, and no arbitrary
environment injection; it runs no configurable or arbitrary command, only the
fixed local od and tr (resolved from PATH) to read a few bytes of entropy - a
small local pipeline with no network or watchdog and no hard latency
guarantee. Any entropy or validation failure that returns omits the carrier
without aborting the spawn. A default-off spawn leaves the generated meta and
launch environment unchanged.
Enablement is default-off (config/trace-context, or FM_TRACE_CONTEXT where a
non-empty value overrides and unset or empty defers to the file) and is
propagated into secondmate homes, taking effect at each agent's next launch:
a Secondmate launched or relaunched after enablement carries the primary
trace into its nested workers, while an already-running Secondmate roots new
traces for its own workers until relaunched. Injection reuses the existing
GOTMPDIR channel, so all spawn backends and harnesses and the ship, scout,
and secondmate paths are covered.
Covered by a pure-library suite and a spawn-path integration test (fake tmux
plus a real worktree, hermetic against ambient FM_TRACE_CONTEXT) proving the
recorded and injected carriers are identical and sent before launch, that
default-off writes and injects neither, that a relaunch reuses the recorded
carrier, and that an explicit FM_TRACE_CONTEXT overrides the file both ways;
plus a source-owner inheritance test proving trace-context propagates and
absence-mirrors through propagate_inheritable_config.
Documentation follows the repository documentation-audiences contract:
docs/trace-context.md is maintainer-architecture rationale, the configuration
schema lives in docs/configuration.md, and the repeatable test evidence is
separated into docs/verification/trace-context.md (maintainer-verification),
registered in docs/documentation-audiences.json.
* fix(spawn): propagate the effective trace-context decision to secondmates
FM_TRACE_CONTEXT overrode trace context only in the process that read it. A
newly launched secondmate decided enablement from the inherited
config/trace-context file alone, so the override did not cross the
primary-to-secondmate boundary: FM_TRACE_CONTEXT=off with the file present left
the secondmate's nested workers traced (a broken kill switch), and
FM_TRACE_CONTEXT=on with the file absent left them untraced despite the
inherited carrier.
Deliver the primary's effective decision to a newly launched secondmate as a
normalized on/off FM_TRACE_CONTEXT in the launch prefix, so a FM_TRACE_CONTEXT
override governs the nested primary -> secondmate -> worker chain both ways, not
just the copied file. The value is bounded to the literal on/off and does not
broaden environment injection; the already-running secondmate boundary is
unchanged.
Add a genuine two-level spawn regression that drives fm-spawn twice with the
exact environment the primary injects into the secondmate and proves both
divergent directions end to end. Correct the documentation that implied
secondmate coverage on every backend, since orca and cmux reject secondmate
spawns, and refresh the verification evidence for the new assertion count.
* no-mistakes(review): Clarify Secondmate trace-context launch snapshots
* no-mistakes(document): Correct trace-context documentation ownership and relaunch semantics
* fix(spawn): resolve the trace-context decision once for carrier and snapshot
The effective trace-context decision was read twice per spawn: once inside
fm_trace_context_resolve for the recorded carrier, and again for the secondmate
FM_TRACE_CONTEXT launch snapshot. A config-file change between the two reads
could pair a carrier with the opposite enable state - an injected carrier with
an off snapshot, or no carrier with an on snapshot.
Freeze the effective on/off decision once, drive the carrier resolution under
that frozen FM_TRACE_CONTEXT so it cannot independently re-read the file, and
reuse the same frozen decision for the secondmate launch snapshot. Add a
spawn-path regression that drives the file-decided path and proves the recorded
carrier and the delivered snapshot always agree, and refresh the verification
evidence for the new assertion count.
* no-mistakes(review): Preserve legacy Secondmate trace boundary
* no-mistakes(document): Correct trace-context verification comparison base
* no-mistakes(review): Captain, prevent failed trace delivery metadata claims
* no-mistakes(review): Captain, align trace-context tests and verification evidence
* no-mistakes(document): Correct trace-context verification evidence
* no-mistakes(lint): Suppress intentional ShellCheck literal-dollar warnings
* no-mistakes(review): Captain: freeze trace context at session start
* no-mistakes(test): Captain: stabilize scheduler test and document Kimi trace coverage
* no-mistakes(document): Document trace-context safety boundaries
* fix(trace): fail off on stale session snapshots
Publish each home session decision atomically through a same-directory temporary file and bind it to the current session lock. A replacement failure can no longer leave an earlier on decision active in a later session; missing, stale, malformed, or unpublishable state defaults safely to off.
Add regressions for read-only replacement and failed publication, update spawn and session-start fixtures for the lock-bound format, and refresh the architecture and verification records.
* no-mistakes(review): Fix trace spawn failure independence and duplicate safety
* no-mistakes(document): Refresh trace-context documentation and verification
* no-mistakes(review): Clear partial backend input after failed trace submission
* no-mistakes(review): Stop unsafe trace delivery before launch append
* no-mistakes(document): Document unsafe trace delivery handling
* fix(trace): bound each trace to one routed task, never the routing agent
A persistent Secondmate holds its launch-time TRACEPARENT in the process
environment for its whole life, and routed requests never replace it, so
resolving new-task carriers from the ambient environment chained every
routed task into one ever-growing trace per Secondmate with distinct
parent ids. Resolve now reuses the task's recorded carrier or mints a
fresh sampled root, never reading ambient TRACEPARENT, so each routed
task is its own trace boundary while relaunch, recovery, and
scout-to-ship promotion keep one stable per-task identity.
The spawn regression models the reviewed scenario exactly: two unrelated
tasks spawned sequentially through one persistent Secondmate environment
record and inject distinct trace ids, adopt nothing from the Secondmate's
carrier, and a relaunch of the first task reuses its original carrier
verbatim.
* docs(trace): define the per-task trace boundary
The design contract is one task per trace: a persistent Secondmate is
routing infrastructure with its own agent identity, never a shared trace
root for the unrelated tasks routed through it. Root/recovery semantics
replace the removed child-inheritance path, the sampling and safety
sections drop inherited-carrier language because ambient TRACEPARENT is
never read, and the verification page records the refreshed suite
inventories including the two-task Secondmate boundary regression.
* test(trace): adopt the explicit per-task delivery contract in spawn fixtures
Rebasing onto current main brings the explicit per-task delivery contract:
ship spawns now require --mode and --yolo instead of resolving them from the
project registry. The trace spawn fixtures pass the same explicit contract
canonical spawn tests use, preserving the per-task trace boundary coverage
unchanged, and the verification page records the refreshed comparison base.
* fix(bin): harden tmux agent liveness across harnesses (#1577)
* fix(bin): classify tmux agent liveness independent of process titles
`fm_backend_tmux_agent_state` attributed a pane solely from
`#{pane_current_command}`, which is a process TITLE a harness can rewrite,
not a structural fact. Claude Code 2.1.220 reports its version string there,
so a live Claude endpoint classified `ambiguous`: the session-start secondmate
liveness sweep could no longer see it, and any consumer that gates on a
positive classification refuses outright.
Read a second, independent name source: the kernel `comm` of every process in
the pane tty's foreground process group. Either source naming a verified
harness yields `alive`, because a false `dead` is the one verdict that can
start a duplicate agent on a live worktree. Scoping to the foreground process
group rather than the pane's descendants keeps a harness-named background
process from faking an agent, and covers multi-process launchers (the Pi
Launcher path) without a special case.
Verified on 2026-08-03 against all seven adapters running for real on tmux
3.6a / macOS 26.5.2 arm64: claude 2.1.220, codex-cli 0.146.0, opencode
1.18.11, pi 0.82.0, pi-signed 0.82.0, grok 0.2.118, kimi 0.31.1 all classify
`alive`, each attributed by a source independent of its title.
Two tests, because they fail for different reasons:
- tests/fm-tmux-agent-liveness.test.sh pins the logic with real processes and
no harness, so it runs everywhere CI runs tmux. It drives the two name
sources apart on purpose and asserts the divergence, so no case can go
quietly vacuous.
- tests/fm-harness-liveness-drift-live-e2e.test.sh relaunches every installed
harness and fails naming the harness and version when one stops being
attributed by a title-independent source.
AGENTS.md section 4 carries the resulting standing rule, and
firstmate-coding-guidelines owns how to satisfy it.
* no-mistakes: apply CI fixes
* docs: move the harness-dependent-check policy out of AGENTS.md
The standing rule was stated in AGENTS.md section 4 with the mechanics in
firstmate-coding-guidelines, which split one contract across two owners and
charged every session for a rule that only fires when firstmate's own
harness-dependent code is being changed.
firstmate-coding-guidelines is now the single owner of both the rule and how
to satisfy it: real-harness proof required, that proof authorized to spend
tokens, structural signals preferred over vendor-rendered surfaces, and a
guard that fails loudly naming the harness and version where a surface signal
is unavoidable. No inline stub is left behind, because AGENTS.md already
carries the load trigger for that skill in sections 7 and 13, so it is read
before any change to firstmate's shared tracked material.
Also records the cross-platform lesson the pipeline caught in the portable
regression, and corrects that file's header: the divergence assertion lives
on the version-string case, which diverges on both supported platforms,
rather than on every case.
* no-mistakes(review): Harden tmux liveness identity and drift validation
* no-mistakes(document): Clarify cross-platform tmux liveness documentation
* feat(bin): propagate trace context to remote secondmates (#1609)
* feat(bin): trace remote secondmate routes and unify the inherit allowlist
Per-task W3C trace context (#995) resolved and injected its carrier only at
the local spawn path. A remote secondmate is routed through
spawn_remote_secondmate, which returns long before that site and wrote its own
metadata block, so a remote secondmate stayed silently untraced even with the
capability enabled.
The parent home still owns that task's identity, because it holds the metadata
an observer reads. It now resolves the carrier against the task's own meta
under its own frozen decision - reused verbatim on relaunch, freshly rooted
otherwise, never adopting the parent process's ambient TRACEPARENT - and hands
it to the configured host through a new fm-spawn --traceparent argument,
accepted only for a secondmate launch and only as a strict W3C value. The
remote host exports it at the same unconditional pre-launch site and reports
back the carrier its endpoint actually holds, which the parent records, so an
already-alive endpoint reports the identity its agent really received rather
than one the parent merely intended. Disabled remains byte-identical and off.
The remote inherit path also carried its own hardcoded copy of the inheritable
config set, already drifted from FM_INHERITABLE_CONFIG by trace-context. Both
remote ends now derive from that one declaration, so a future item cannot be
sent by one side and refused by the other, and session-scoped enablement items
are skipped on live convergence exactly as the local path skips them.
Also fixes a latent stderr leak: an absent session lock printed a raw redirect
failure, which the new remote resolve site made visible.
Adds tests/fm-remote-secondmate-trace-context.test.sh, driving the real
parent -> fm-on -> remote entrypoint -> control -> remote fm-spawn chain over
the deterministic SSH boundary and reading the carrier back from the remote
pane's own log.
* no-mistakes(document): Clarify remote trace and allowlist contracts
* feat(bin): preflight remote runtime tool paths (#1623)
* feat(bin): widen the remote runtime PATH and add a remote doctor preflight
The fixed remote entrypoint hard-coded a four-directory PATH, so a remote
account whose tools live under nix or a per-user profile could not run basic
Firstmate work without a login shell. The entrypoint now composes its child
PATH from the code root's bin, the account's ~/.local/bin, the common
package-manager directories that actually exist on the host, and the portable
system tail, deduplicated and in a fixed order, still under env -i with the
same variable allowlist and no shell command string.
fm-remote-doctor.sh reports that exact PATH by inheriting it from its own
entrypoint launch rather than recomposing it, so the ordering keeps one owner.
It is read-only, reports where each required and optional tool resolved, and
exits non-zero naming every required tool that did not. Remote seeding runs it
as a preflight before anything is created on the host and restores the registry
when it fails.
* no-mistakes(review): Harden remote git authorization and missing-tool diagnostics
* no-mistakes(document): Document remote PATH doctor and safe shims
* no-mistakes(lint): Fix ShellCheck findings in remote path tests
* no-mistakes(lint): Suppress exported fixture's false-positive ShellCheck warning
* feat: gate remote second mates on Herdr readiness (#1639)
* feat(bin): gate remote second mates on herdr readiness
A remote second mate now always runs on the Herdr backend, whose server
belongs to the host's GUI login session and therefore outlives the SSH
connections that supervise it. fm-spawn's remote route forces that backend
and the host-local control script refuses any other, so the requirement
cannot be dropped from either side.
fm-remote-doctor.sh becomes the single owner of what "ready" means. It keeps
its PATH and tool reporting from #1623 and adds the Herdr, Aqua LaunchAgent,
GUI-session, server-reachability, and entrypoint-symlink checks, tagging each
gap fixable: or human: with the exact operator step. --fix closes only the
automatable gaps - writing and loading the Aqua-scoped dev.firstmate.herdr
launch agent, starting the server where no launch agent applies, and
recreating the entrypoint symlink - then re-derives every check from the host,
so a human gap is never presented as fixed. It never creates a login session,
writes an auto-login password, or touches FileVault.
Remote seed, remote spawn, and the startup liveness relaunch all run the same
check, repair, re-check sequence through one shared library and fail closed
with the doctor's own gap text. Recovery inherits the gate because it respawns
through the same route.
Tests drive the real doctor against a controlled account fixture with a
private HOME, a state-backed launchctl, and a fake herdr, and prove the
dangerous actions are never attempted. The remote lifecycle suites gain a
stateful Herdr CLI fixture and answer the readiness gate at the SSH boundary,
so they never inspect or repair the runner's own account.
* no-mistakes(review): Validate launch-agent contract and confirm Herdr startup
* no-mistakes(review): Validate loaded launch-agent contract before readiness
* no-mistakes(review): Refuse legacy remote backends without altering routes
* no-mistakes(review): Clarify conditional remote readiness repair sequence
* no-mistakes(review): Repair remote readiness before liveness probing
* no-mistakes(review): Preserve unknown seeds and reject legacy liveness
* no-mistakes(document): docs: clarify remote Herdr backend ownership
* fix: isolate remote secondmates in shared Herdr session (#1659)
* Pin remote secondmates to fm-remote
* no-mistakes(review): Fail closed on legacy remote Herdr endpoints
* no-mistakes(review): Isolate fm-remote launch agent from interactive default
* no-mistakes(document): Document shared remote Herdr retirement safety
* feat: route remote commands through an Aqua job worker (#1660)
* feat: run remote commands through Aqua job worker
* no-mistakes(review): Enforce remote job deadlines and safe worker shutdown
* no-mistakes(review): Refresh stale workers and harden dependency-free supervision
* no-mistakes(review): Harden worker ownership recovery and shutdown quarantine
* no-mistakes(review): Fix doctor bootstrap, harness repair, and output draining
* no-mistakes(review): Probe doctor tools through authenticated worker bootstrap
* no-mistakes(review): Refresh stale workers before doctor tool probes
* no-mistakes(review): Recover stopped quarantines and extend job deadlines
* no-mistakes(review): Separate queue and execution timeout windows
* no-mistakes(review): Supervise Linux worker crashes and bind root identity
* no-mistakes(review): Resolve authorized Nix profile bin links
* no-mistakes(review): Clarify Nix path resolution documentation
* no-mistakes(review): Harden PATH safety and nvm selection
* no-mistakes(review): Honor nvm system defaults and refresh doctor digest
* no-mistakes(review): Keep workers ready during active jobs
* no-mistakes(review): Bound pre-execution validation by job timeout
* no-mistakes(document): Clarify remote worker documentation
* no-mistakes(lint): Fix remote worker ShellCheck diagnostics
* no-mistakes: apply CI fixes
* no-mistakes: apply CI fixes
* no-mistakes: apply CI fixes
* fix: clarify remote doctor bootstrap path (#1691)
* fix(bin): bound remote SSH dead-peer detection (#1699)
* fix(remote): arm SSH dead-peer detection in fm-on.sh
A vanished remote host mid-poll (a reboot, a dropped link) left ssh
blocked indefinitely on a half-open TCP connection, because fm-on.sh's
ssh invocation had no ServerAliveInterval/ServerAliveCountMax. This
wedged the remote-reply ferry: fm-procevent.sh's runner blocked inside
the ssh child and never reached its own no-result -> claim-release ->
reconcile re-arm self-healing path, which otherwise already handles a
nonzero exit with empty output correctly. Recovery required a manual
retire and re-arm.
Arm ServerAliveInterval=15 and ServerAliveCountMax=3 by default
(bounded ~45s detection window), both overridable via
FM_SSH_ALIVE_INTERVAL and FM_SSH_ALIVE_COUNT_MAX. This is a transport-
level fix in fm-on.sh, so it covers every remote command routed
through it, not just the reply ferry. The remote sshd answers
keepalive probes independently of whatever the remote command is
doing, so a legitimately long-but-alive command (a 55s poll, a clone,
the doctor) is never falsely killed - only a truly vanished peer trips
it, turning that case into a bounded, detectable ssh failure (exit
255) instead of an indefinite hang.
Extends tests/fm-on.test.sh with a behavioral regression asserting a
bounded, positive ServerAliveInterval/ServerAliveCountMax on the real
ssh argv captured through the FM_SSH_BIN process seam, plus coverage
that both are env-overridable.
* no-mistakes(document): Document SSH dead-peer detection ownership
* fix: report stale AXI tools during bootstrap (#1701)
* feat(bootstrap): gate stale axi CLIs at the floors firstmate actually uses
Add gh-axi 0.1.29 floor so bare --squash PR merges stop failing quietly on
older builds. Raise tasks-axi to FM_TASKS_AXI_MIN=0.2.2 (multi-id mv) while
keeping feature probes. Keep quota-axi at 0.1.16 after verifying schema 3
and per-model availability already ship there; runway remains optional.
* no-mistakes(document): Clarify AXI compatibility documentation ownership
* fix: prevent false watcher-down alarms in Claude sessions (#1661)
* fix(guard): stop false watcher-down alarm mid-turn under Claude auto-arm
bin/fm-guard.sh derived its watcher-health verdict from fm_watcher_healthy,
which requires a live watcher process holding the home lock. Under the Claude
Stop-hook auto-arm supervision model the watcher is armed at each turn end and
exits on its wake, so it runs only between turns. Every guarded command run
mid-turn therefore found no live watcher and printed the "WATCHER DOWN -
SUPERVISION IS OFF" banner even though supervision was healthy. Because the
episode key was derived from the beacon mtime (which the between-turns watcher
advances every poll), the full banner re-printed on essentially every command,
and the message always blamed a "fresh beacon" that was in fact fresh.
Make the pull guard's health check model-aware via a new
fm_watcher_supervision_verdict in bin/fm-wake-lib.sh:
- Under the auto-arm model a beacon fresh within FM_GUARD_GRACE is healthy even
with no live watcher process; only a beacon stale beyond grace (or absent) is
a genuine lapse and alarms.
- Under every persistent-watcher harness (codex foreground checkpoint,
opencode/pi/grok background arm, tmux, unknown) a live identity-matched
watcher with a fresh beacon is still required, unchanged.
The banner now names the true failing condition, a missing live watcher process
versus a genuinely stale beacon, instead of always blaming the beacon, and the
once-per-episode dedup keys on that condition rather than the beacon mtime so a
genuine lapse announces once and does not re-print each turn.
The turn-end guard keeps the strict fm_watcher_healthy check because it fires at
the turn boundary, where the auto-arm brings a fresh watcher up and it
cooperates with that arm. fm_watcher_healthy itself is unchanged, so the arm
layer's start/attach/replace decisions are unaffected.
Tests in tests/fm-guard-stale-banner.test.sh cover the auto-arm healthy
fresh-beacon-without-a-watcher case, the auto-arm stale-beacon alarm and its
stable episode, the true-reason banner wording, and the reason-keyed episode
surviving a beacon mtime change; existing persistent-model cases are pinned to
that model.
* no-mistakes(review): Pin secondmate supervision model to launched harness
* no-mistakes(document): Align watcher documentation with model-aware supervision health
* test: prevent fixture temporary directory leaks (#1704)
* fix(tests): stop fixture-tempdir helper from self-deleting under command substitution
fm_test_tmproot is almost always called as `TMP_ROOT=$(fm_test_tmproot prefix)`,
which forks a subshell to capture its stdout. The old implementation set its
EXIT cleanup trap inside that call, so the trap fired - and deleted the fixture
root - the instant the subshell exited, before the real caller's own EXIT trap
was ever installed. Every test using the documented call pattern leaked its
fixture root on every run; two suites had already independently discovered and
worked around this with ad-hoc mktemp calls.
Registration now goes through a $$-keyed registry file instead of in-process
state, since $$ resolves to the invoking shell's PID even inside the
subshell. The real cleanup trap is armed once at source time (always the real
caller, never a subshell) for EXIT, INT, and TERM. A best-effort orphan sweep
on next source reaps marked fixture roots old enough to be from a killed prior
run.
Simplifies the two existing ad-hoc workarounds (fm-procevent.test.sh,
wake-helpers.sh) back onto the shared helper now that it works correctly.
* no-mistakes(review): Preserve live fixtures during orphan reaping
* no-mistakes(review): Harden fixture ownership against PID reuse
* no-mistakes(review): Secure cleanup registry against path precreation
* no-mistakes(review): Make fixture registration transactional
* no-mistakes(document): Documentation already matches fixture cleanup behavior
* no-mistakes: apply CI fixes
* no-mistakes: apply CI fixes
* feat(herdr): enable presentation spaces by default (#1708)
* feat(herdr): default presentation spaces on with an explicit opt-out
Herdr's disposable one-task presentation workspace was opt-in through the
presence of local config/herdr-presentation-spaces. It is now on by default,
and a home opts out by writing "off" into that same file.
Values are read with the whole-file whitespace-stripped convention the other
scalar config items already use, plus case folding. An absent file, an empty
file, and "on" all resolve on; only "off" opts out; an unrecognized value warns
and keeps the default rather than failing a spawn over a purely visual setting.
The empty file is exactly the historical opt-in form, so every home that had
already enabled the projection stays enabled with no migration step, and no
previously enabled home can be turned off by the flip.
Because absence now means on at both ends, secondmate inheritance needs no
item-specific convergence: mirroring an absent primary file converges a
secondmate to the same default-on rather than turning its projection off, and
only an explicit primary opt-out propagates the opt-out.
The gate itself moves into fm_backend_herdr_presentation_enabled in the Herdr
adapter so the semantics have one owner that regressions can exercise directly.
* no-mistakes(document): Document Herdr default-on presentation safety
---------
Co-authored-by: kunchenguid <kun-1@kunchenguid.com>
* fix(bin): surface fleet-wide open decisions on every wake drain (#1711)
* fix: surface consolidated open decisions on every wake-drain
A needs-decision or blocked event buried under later, unrelated status
appends was only ever shown via the last-line wake annotation, so a
still-open captain decision could go silently missed even though
status_open_decisions (fm-classify-lib.sh) already folds the whole
status stream correctly and fleet-snapshot/bearings already reuse it.
Wire that same fold into bin/fm-wake-drain.sh: a new fleet-wide
scan_open_decisions wrapper scans every state/<id>.status, and
fm-wake-drain.sh prints a separate, bounded OPEN DECISIONS section on
every drain (including the empty-queue fast path), so session-start
and every wake-handling turn surface it for free without duplicating
the open/resolved fold itself. Heartbeat wakes drain through the same
script, so this covers that surface too.
Also tighten status_open_decisions' file guard to skip an unreadable
status file instead of leaking a bash redirection error, now that a
fleet-wide directory scan can reach files a single targeted read
would not.
* no-mistakes(review): Prevent status symlinks leaking open decisions
* fix: drop unbounded perl subprocess from status symlink guard
The review step's own symlink-safety auto-fix (O_NOFOLLOW read via a
perl subprocess) forked one perl process per status file scanned by
the new fleet-wide open-decisions scan, with no cap - inflating
fm-wake-drain.sh's total external-read cost from 8 (the existing
annotation read_cap) to 18 in the enrichment-caps regression test.
The plain [ -L "$f" ] check already rejects any status file that is
itself a symlink before any read happens, which is exactly what the
new regression test exercises and is the same defense level the
sibling scan_captain_relevant_statuses/last_status_line already rely
on elsewhere in this file (no O_NOFOLLOW). Drop the subprocess-based
nofollow read and keep the cheap builtin guard.
* no-mistakes(document): Document actionable fleet-wide open decision drains
* fix(bin): abort parked runs and reap leaked processes before teardown (#1710)
* fix(bin): abort orphaned no-mistakes runs and reap leaked processes at teardown
Teardown could remove a task's worker while its no-mistakes pipeline run was
still parked at a gate, leaving an orphaned run holding a fleet slot
indefinitely (observed 2026-08-03: runs parked 7h39m and parked at a
post-CI approval gate). It could also leave backgrounded/disowned
descendant processes rooted under the worktree or tasktmp surviving
reparented to init (observed: two `go test` binaries pinning CPU for
hours with no live task meta to attribute them to).
Add two coupled pre-teardown steps, both scoped to this task's exact
branch/head or worktree/tasktmp so they can never touch another task's
run or processes:
- conclude_task_no_mistakes_run aborts a run parked at a gate via
`no-mistakes axi abort`, cd'd into the exact worktree so the daemon
resolves the run itself rather than teardown naming a --run id.
- reap_task_worktree_processes sweeps for processes whose cwd is under
the worktree or tasktmp (via `lsof -a -d cwd`) and TERM/KILLs them.
Both run before any worktree return, branch delete, or backend kill,
and are idempotent on a retried teardown. The branch+head attribution
logic is factored out of bin/fm-crew-state.sh into the new shared
bin/fm-nm-run-lib.sh so both scripts use the same ownership contract.
* no-mistakes(review): Fail closed on incomplete teardown cleanup
* no-mistakes(review): Bind teardown cleanup to verified run and process identities
* no-mistakes(review): Require confirmed aborts and convergent identity-safe process reaping
* no-mistakes(review): Handle process exits during teardown identity checks
* no-mistakes(review): Restore teardown library in hermetic gotmp fixtures
* no-mistakes(document): Document teardown run attribution and timeout
* no-mistakes(lint): Rename shell variable conflicting with done keyword
* no-mistakes: apply CI fixes
* fix: resolve fm-remote-entrypoint.sh SCRIPT_DIR through a PATH symlink (#1709)
The script installs as a symlink under ~/.local/bin. Taking dirname of
the symlink itself (instead of its real target) pointed SCRIPT_DIR at
~/.local/bin, breaking sourcing of the sibling fm-remote-job-lib.sh.
Resolve the real path first, preferring python3's os.path.realpath,
then realpath, falling back to the raw BASH_SOURCE on hosts with
neither.
* fix(pi): gate Calm built-in overrides by activation state (#1724)
* fix(pi): stop Calm claiming a built-in tool name another extension owns
fm-calm.ts claimed bash/read/edit/write/grep/find/ls unconditionally at
extension load, regardless of whether Calm was on. Pi resolves two
extensions registering the same built-in name by first-registered-wins
with no merge and no unregister call, and Calm's project-local
.pi/extensions/ position beats any global or CLI-configured extension,
so a user who never even enabled Calm could have their own bash/read/etc
override silently replaced.
Captain-approved plan implemented:
- Registration is now gated on config/calm already being "on" at load
time. A Calm-off session or reload registers nothing, so a non-Calm
user never contests a name. This stays synchronous during the
factory's own load, not deferred to session_start: /reload (and
ctx.newSession/fork/switchSession) render the restored transcript from
a pre-session_start snapshot of the tool registry, so a deferred claim
would miss that render - confirmed by tests/fm-calm-pi-extension
.test.sh's hidden-block-geometry E2E when trialed.
- The first time Calm turns on in a session that started off
(activateBuiltInsIfNeeded, from the /calm command handler), Calm calls
pi.getAllTools() - safe only once every extension has finished loading,
unlike the load-time path above - to see whether a different extension
already owns a name, and skips claiming only that one, leaving it and
its owning extension fully intact and callable.
- A contested name found this way prints a prominent ctx.ui.notify()
warning naming the tool, plus a console diagnostic.
- reportBuiltInLosses() remains the backstop for the one case neither of
the above can reach: a session that starts or reloads with Calm already
on, where the registry snapshot is taken before Calm gets any chance to
check ownership. A symlink-safe realpath comparison avoids misreporting
Calm's own registration as foreign when its path crosses a symlink
(macOS /tmp, /var).
Confirmed, bounded trade-off: the very first time a session that started
Calm-off turns Calm on, tool-call rows already on screen from before that
toggle do not retroactively collapse, because Pi never lets an extension
re-point an already-rendered row at a definition registered later. Every
session after that first toggle starts with the preference already on and
takes the synchronous load-time path, so the guarantee is intact from
then on. docs/calm.md and the file's own header document this in full.
tests/fm-calm-pi-extension.test.sh gains test_builtin_gate_load_time
(config/calm off registers nothing, on registers all 7 synchronously at
load) and test_calm_activation_collision_and_regression_bound (first
activation claims every uncontested built-in, leaves a foreign bash tool
fully intact and callable, warns and logs the contested name, and locks
in the documented pre-activation bound against real ToolExecutionComponent
rendering). test_rendering_and_session_lifecycle and the live interactive
E2E are updated for the new gate-at-load and first-activation-bound
contract.
* no-mistakes(document): Document Calm tool collision boundaries
* no-mistakes: apply CI fixes
* fix(bin): persist secondmate parent bindings for cleanup (#1727)
* fix(bin): give secondmate homes a durable parent binding record
Finished-worker cleanup on a remote second mate refused forever with
"cannot resolve the primary home ... durable parent binding". The
remote launch hands the child the remote code checkout as its parent
home (fm-spawn.sh's sole writer of FM_PUBLIC_FOLLOWUP_PRIMARY_HOME
receives FM_HOME=$FM_ROOT from fm-remote-secondmate-control.sh's
host-local launch), and that path can never carry the parent's real
records, so the guard refused unconditionally once relay looked active
anywhere on that host.
fm-home-seed.sh and fm-remote-home-provision.sh now write a durable
.fm-secondmate-parent record next to the .fm-secondmate-home identity
marker, naming the home's route to its parent as local (with the real
parent path) or remote (with the parent's SSH alias for diagnostics
only). fm-teardown.sh's cleanup gate reads it: a remote parent is out
of scope for the delegated-promise check (the whole promised-public-
reply subsystem is same-filesystem by construction, so a remote parent
can never hold one), while a token committed directly to the child's
own .env file - never the process environment - still refuses, so an
unrelated export in the remote host's login shell can no longer mask
in. For a local secondmate, the durable parent_home now also backs up
the launch-time env var, closing a silent fail-open where a restart
that dropped the launch prefix made the guard treat a genuinely active
parent relay as off.
Regression coverage drives the real remote route (SSH boundary + Herdr
fixture) and real fm-home-seed.sh seeding rather than hand-crafted
markers.
* no-mistakes(review): Captain: fail closed on unsafe durable parent records
* no-mistakes(review): Captain: enforce durable parent binding commit protocol
* no-mistakes(review): Captain: publish local parent binding before identity
* no-mistakes(review): Captain: refuse conflicting local parent bindings
* no-mistakes(review): Captain: reject non-regular secondmate seed leaves
* no-mistakes(review): Captain: enforce unique durable parent bindings
* no-mistakes(review): Captain: reject route-incompatible durable parent fields
* no-mistakes(document): Document durable secondmate parent bindings
* no-mistakes(lint): Fix secondmate parent parser ShellCheck warnings
* no-mistakes: apply CI fixes
* feat(bin): enforce latest AXI-family tool floors (#1733)
* feat(bin): gate lavish-axi at its session_ended floor in bootstrap
bin/fm-procevent-lavish.sh decides that a human "Send & End" review is
terminal by reading session_ended from the poll response's leading session
block. That field first shipped in lavish-axi 0.1.35, so an older installed
build silently leaves every ended review source armed forever and captures
an empty ended result on each later cycle. The same release is what makes a
plain reopen refuse a session the human deliberately ended.
Add LAVISH_AXI_MIN=0.1.35 to the existing axi-family floor structure in
bin/fm-bootstrap.sh, reusing tool_version_at_least and the same MISSING
diagnostic gh-axi already emits, so an incompatible build is reported as an
upgrade request before any review surface is armed. Later lavish-axi
releases only add artifact-authoring surface the adapter never reads, so
the floor is the feature-introduction point rather than latest.
Fixtures that stubbed lavish-axi as a bare exit-0 tool would now be read as
unparseable builds, so tests/lib.sh gains fm_fake_version_tool and every
bootstrap-running suite uses it for lavish-axi.
* no-mistakes(review): Clarify lavish-axi version floor rationale
* no-mistakes: apply CI fixes
* feat(bin): set axi-family floors to current latest under the bump policy
The axi-family bootstrap floors are the CURRENT LATEST published version of
each tool, captain-bumped periodically to move the whole fleet onto the
newest axi tools. They are not the minimum feature-introduced version. The
earlier lavish-axi work set a feature-minimum floor, which is the opposite
of this policy, so replace it along with the older feature-minimum rationale
carried by tasks-axi and quota-axi.
State the policy explicitly in bin/fm-bootstrap.sh's header, which owns it,
and in each per-tool floor owner, so no future change argues a floor back
down to the earliest release that happens to satisfy some behavior. Remove
the lavish-axi session_ended and upstream-PR citation, the tasks-axi
multi-ID-mv minimum argument, and the quota-axi credential-source argument
as floor rationale; the tasks-axi feature probes remain as a separate
defense-in-depth concern.
Floors: lavish-axi 0.1.45 (was 0.1.35), tasks-axi 0.2.4 (was 0.2.2),
quota-axi 0.1.17 (was 0.1.16), gh-axi 0.1.29 unchanged and already latest.
Each was verified against the tool's current published version.
The mechanism is unchanged: the same shared version helper and the same
MISSING diagnostic path. The below-fires and at-or-above-silent regression
rows move to the new floors, keeping each boundary genuine by pinning the
patch immediately below each floor rather than a version that was only
below the old one. Fleet fixtures move to the new floors so a bootstrap-
running suite is not reported as an out-of-date build.
Three operator-facing backlog handoff and receipt errors named "0.2.2+"
while the enforced floor moved, so they now point at the floor's owner
instead of duplicating a version number that drifts.
* no-mistakes(review): Centralize AXI floor policy beside constants
* no-mistakes(review): Clarify bootstrap boundary test comment
* no-mistakes(document): Centralize AXI floor policy rationale
* fix(bin): bound open decision scans with incremental cursors (#1737)
* fix(bin): bound OPEN DECISIONS scan cost with a per-status-file cursor
The fleet-wide OPEN DECISIONS scan added in #1711 re-reads and refolds
every task's entire lifetime status log on every drain, so its cost
grows unbounded with total log size. Add status_open_decisions_incremental
and scan_open_decisions_incremental to fm-classify-lib.sh: they persist a
per-status-file byte cursor plus the folded open-decision set, and fold
only newly appended bytes on each call, reusing status_open_decisions'
exact fold-line rule (extracted into _fm_decision_fold_line) so the two
strategies can never disagree on what is open. A missing or invalidated
cursor (new task, truncated/rewritten/shrunk log) falls back to a full
re-fold. bin/fm-wake-drain.sh now calls the incremental wrapper instead
of the whole-file scan.
* fix(bin): add O(1) rotation detection and read-failure guarding to the cursor fold
Add the two pieces the incremental open-decisions cursor was missing,
scoped to this repo's actual status-file usage (create-once, append-only,
never replaced or rewritten in place):
- An O(1) device+inode identity check (one stat call) alongside the
existing size-shrink check, so a status file replaced/rotated/recreated
at the same path is detected and falls back to a full re-fold, even
when the replacement is the same size. A same-inode, same-size,
in-place byte edit is a deliberately accepted gap: no code path in
this repo ever does that to a status file.
- Checked reads: a stat/wc/tail failure is a genuine I/O error, not
"the file is empty" - it now reports the already-trusted persisted
open set unchanged instead of risking a silent invalidation.
Both stay O(1) plus new bytes per call, matching the cursor's bounded-
cost design; no content hashing or pending-fragment machinery.
* no-mistakes(review): Preserve cursor state across failed incremental reads
* no-mistakes(review): Refold status when cursor cache reads fail
* no-mistakes(document): Document cursor-backed open-decision scanning
* no-mistakes: apply CI fixes
* fix(bin): prevent remote polls from blocking session startup (#1754)
* fix(bin): preempt remote reply long-polls for queued short jobs
Session start on a home with live remote second mates could stall silently
for many minutes: the single serial remote job worker ran each armed
fm-remote-delta-read.sh reply poll to its full 55s window while bootstrap's
short sync, inherit, state, and route commands sat queued behind it, and
non-FIFO queue pickup let re-armed polls keep winning the lane. Measured
end to end, a trivial short job took 31s behind one 30s poll window.
The worker now preempts a running preemptible job (the read-only, cursor-
anchored delta read is the only member of that class) as soon as a
non-preemptible job is queued, publishing exit 75 with emptied output -
byte-identical to the poll's own elapsed-window-with-no-data result - so
the parent runner takes its existing no-result path and the watcher re-arms
from the same cursor with nothing lost. The delta read translates SIGTERM
into that same exit after removing its staging directory. Sibling polls
never preempt each other, so two armed monitors cannot churn. The same
measured scenario now completes in 1s.
* no-mistakes(document): Clarify remote poll preemption documentation
* docs: present X mode as the X and Discord public surface (#1778)
Discord mentions already ride the same pairing-token opt-in, relay poll,
and platform-aware reply path as X mentions, but the docs still read as
X-only, so a stranger could not self-serve the Discord path.
Add the numbered turn-on steps to the X mode configuration reference,
pointing at the myfirstmate dashboard for account creation, bot install,
and token issuance rather than duplicating operator setup here, and drop
the X-only framing from the README bullet, the documentation index, and
the architecture overview.
* fix(bin): run session start deterministically from hooks (#1781)
* feat(bin): run session start deterministically on hook-capable harnesses
Session start relied on a native nudge that only asked the agent to run
bin/fm-session-start.sh, and an agent can defer that. Observed 2026-08-01:
an /ahoy-first session followed the recap path and did not take the helm
until a later request forced it.
Claude, Codex, and Pi now RUN the digest in their session-open hook through
the new bin/fm-sessionstart-run.sh, so the full ordered digest is in model
context before the first turn. That wrapper is the single owner of what a
session-open source means: startup and Pi's "new" take the helm, clear and
compact re-emit, resume/reload/fork delegate to the nudge, and an unreadable
source takes the helm because doing that redundantly is idempotent while
skipping it is the bug. Grok and OpenCode keep the nudge as the floor, since
neither can carry hook stdout into a model turn.
Because the hook now blocks session initialization, fm-session-start.sh
bounds itself first. Its steps are not all individually bounded - bootstrap's
gh auth probe, tool version probes, the backlog listing and per-task endpoint
reads are unbounded - so the whole digest runs as one bounded child (default
120s). Whatever it emitted before the bound survives, and the parent adds a
loud STARTUP TRUNCATED banner naming the stage that stalled and every stage
that never ran, still exiting 0.
--reemit skips only the sweeps startup already reconciled. It still re-verifies
lock ownership and still drains queued wakes, which arrived after startup and
are the turn's work. fm-bootstrap.sh gains FM_BOOTSTRAP_LOCKED so a re-emit
keeps repair ownership instead of deferring to a lock holder that is itself.
Also adds bin/fm-timeout-lib.sh as the single owner of bounded execution,
replacing three near-identical copies, and gives the ahoy skill a helm check
so a nudge-tier harness cannot recap before taking the helm.
Verified live on 2026-08-05 against Claude 2.1.222,…
…mate-upstream-sync # Conflicts: # bin/fm-spawn.sh # tests/fm-spawn-dispatch-profile.test.sh
…file test assertions across upstream sync
Ospeto
force-pushed
the
fm/firstmate-upstream-sync-final
branch
from
August 9, 2026 02:44
b5608a7 to
d56401b
Compare
Author
|
Closing upstream PR: intended target is fork Ospeto/firstmate:main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Integrate upstream kunchenguid/firstmate main at exact audited commit 833a9a2 into fork Ospeto/firstmate from exact base head 5b49c0b, preserving every fork feature and shipping the integration through a pull request.
What Changed
Risk Assessment
✅ Low: The timeout now encloses both checker execution and output draining, the tmux resume path remains safely repaired, and both the exact fork base and audited upstream commit are preserved ancestors of the target.
Testing
Git-history checks proved the exact two-parent audited integration; five focused automated suites and manual CLI/brief checks demonstrated preserved fork behavior end to end, all passed, reviewer-visible transcripts were captured, and the worktree remained clean. Push, PR, and remote CI validation remain owned by later outer-pipeline phases.
Evidence: Exact merge ancestry and conflict-surface transcript
Evidence: Carried fork CLI behavior transcript
Evidence: Spawn, dispatch, preflight, fallback, and resume E2E transcript
Evidence: Generated no-mistakes task brief
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
.agents/skills/harness-adapters/SKILL.md- branch carries 6 commit(s) that exist on your local main branch but were never pushed to origin/main; rebasing would bundle this unrelated work (45 file(s)) into the PR:Push main to origin, or rebase your branch onto origin/main, before gating.
🔧 **Review** - 2 issues found → auto-fixed (2) ✅
bin/fm-spawn.sh:822- A normal tmux agent exit leaves its window alive with only a shell. This check classifies that endpoint asdeadand permits resume, but the laterfm_backend_tmux_create_taskcall refuses because the samefm-<id>window still exists. Thus--resumefails in its primary recovery scenario; the current test omitswindow=and misses this sequence. After the ownership check, reuse or remove the confirmed-dead endpoint before creating its replacement.bin/fm-spawn.sh:1100-FM_ANTIGRAVITY_PREFLIGHT_OUTPUT_BYTESis validated but never applied: all checker output is written to the temporary file until the timeout. A noisy checker can therefore consume unbounded disk for 30 seconds, contradicting the documented bounded-output invariant. Since the output is never read, discard it or cap storage while continuing to drain the process.🔧 Fix: Fix resume cleanup and bound preflight output
1 error still open:
bin/fm-spawn.sh:1103- The new drain sits outsidetimeout. If the checker exits after spawning a background child that inherits stdout,timeoutreturns butcatwaits for that descendant to close the pipe, so the 30-second preflight can hang indefinitely. The existing forked-checker fixture redirects the child’s stdout and does not exercise this sequence. Put the producer and drain under the same process-group timeout, or simply discard checker output since it is never consumed.🔧 Fix: Bound preflight drain within checker timeout
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
git show -s --format=... 7aecd0e09f97652c6e3490ad276c1a8c7d7d2983andgit merge-base --is-ancestorfor upstream833a9a25…, fork base5b49c0bd…, and target8b5fbb03…git show --remerge-diff --stat 7aecd0e09f97652c6e3490ad276c1a8c7d7d2983bash tests/fm-spawn-dispatch-profile.test.shbash tests/fm-brief.test.shbash tests/fm-task-delivery.test.shbash tests/fm-bootstrap.test.shbash tests/fm-secondmate-lifecycle-e2e.test.shManual isolatedfm-project-mode.shchecks for direct-PR, conditional +yolo, raw policy, and malformed-policy fallbackManualfm-brief.sh audited-sync firstmate --mode no-mistakesgeneration and inspectionbash tests/fm-spawn-dispatch-profile.test.sh | tee .../spawn-dispatch-e2e.txtFinalgit status --shortand evidence-file verification✅ **Document** - passed
✅ No issues found.
🔧 **Lint** - 1 issue found → auto-fixed ✅
🔧 Fix: Suppress intentional child-shell expansion lint warning
✅ Re-checked - no issues remain.
✅ **Push** - passed
✅ No issues found.