feat(no-mistakes): Add run liveness detection tools - #8
Conversation
- Extract active_steps section before parsing to avoid mixing with steps table - Parse last_activity timestamp from active_steps for real-time progress tracking - Report actual elapsed time since last activity instead of cumulative step duration
…tent timeout config, and liveness detection logic
…d dead code, unified duplicate utilities
… index and environment variables
|
Warning Review limit reached
Next review available in: 32 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdded single-run and all-run no-mistakes liveness checkers. The tools query run and step data, detect stale activity, resolve task ownership, return status-based exit codes, and document configurable timeouts and supervision workflows. ChangesNo-mistakes liveness
Estimated code review effort: 4 (Complex) | ~45 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant Operator
participant fm-no-mistakes-liveness.sh
participant no-mistakes
participant StateDirectories
Operator->>fm-no-mistakes-liveness.sh: request all-run or task-specific check
fm-no-mistakes-liveness.sh->>no-mistakes: query running runs and step data
no-mistakes-->>fm-no-mistakes-liveness.sh: return run status records
fm-no-mistakes-liveness.sh->>StateDirectories: resolve branch and task ownership
StateDirectories-->>fm-no-mistakes-liveness.sh: return task identifier
fm-no-mistakes-liveness.sh-->>Operator: report live or hung status and exit code
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@bin/fm-nm-run-is-live.sh`:
- Around line 23-99: Remove the duplicate check_run_liveness definition in
bin/fm-nm-run-is-live.sh lines 23-99 and invoke the shared check_run_liveness
from bin/fm-no-mistakes-liveness.sh, mapping its result to exit code 2 for query
failures if needed. In bin/fm-no-mistakes-liveness.sh lines 271-305, replace the
fallback’s hardcoded 1800 threshold with STALE_THRESHOLD and include running
alongside fixing as hung-eligible, so the shared implementation applies
FM_NM_LIVENESS_STALE consistently.
In `@bin/fm-no-mistakes-liveness.sh`:
- Around line 336-356: Update find_task_for_branch to return both the resolved
task identifier and worktree, using a stable delimiter such as a tab, and parse
both values at the call site. Replace the local task_id/meta_file lookup in the
run-ID resolution block with the returned worktree, then run axi status directly
there so secondmate and project tasks reuse the already-resolved path.
- Around line 106-117: Update the secondmate-home scan in the surrounding shell
block to stop skipping lines beginning with "- ", while continuing to ignore
comment lines and blank lines so the documented "- [id](path) — description"
entries reach the existing regex extraction. Remove the ineffective 2>/dev/null
redirection from the [ -f "$FM_HOME/data/secondmates.md" ] test.
- Around line 67-70: Extend the sed range in the usage() function so --help
includes the complete Usage and Exit codes documentation, covering the lines
after the current line 11 cutoff while preserving the existing comment-format
stripping and exit behavior.
- Around line 369-374: Update the output formatting in the liveness reporting
block so an unknown elapsed value prints “last-activity: unknown” without
appending the seconds suffix, while numeric values retain the “s” suffix. Also
revise the header comment’s documented status values to match the actual values
emitted by the liveness logic, including live, hung, and unknown.
- Around line 73-85: Update nm_run so the fallback branch executes no-mistakes
directly without a timeout when HAVE_TIMEOUT is unavailable, instead of running
true. Remove the || true suffix from every timeout and perl invocation so each
command preserves its exit status for callers such as check_all_runs and command
substitutions.
- Around line 197-212: Update parse_active_steps_time() to recognize day-scale
and larger active_steps durations, converting them to seconds so
check_run_liveness() correctly identifies long-stalled runs as stale. Support
composite formats such as “1h30m ago:” when present, or return a sentinel that
check_run_liveness() explicitly treats as stale/hung instead of live.
In `@docs/no-mistakes-run-liveness.md`:
- Around line 44-47: Update the exit-code documentation to describe hung runs as
inactive for greater than the configured stale threshold, with 300 seconds as
the default, rather than always stating five minutes.
- Around line 26-28: Update the fenced code block containing the run status
output example to declare the `text` language, changing the opening fence while
preserving the example content and closing fence.
- Around line 30-33: Update the supervision example and protocol text in
docs/no-mistakes-run-liveness.md to distinguish exit code 1 (hung) from exit
code 2 (usage or precondition failure), escalating only the intended status or
explicitly documenting manual intervention for status 2. Also define the outcome
mapping for status: unknown, including whether it is fail-closed for
daemon-affecting actions or follows a separate handling rule.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9deb5985-b8e4-4aa0-a500-26ea71109bfe
📒 Files selected for processing (5)
bin/fm-nm-run-is-live.shbin/fm-no-mistakes-liveness.shdocs/configuration.mddocs/no-mistakes-run-liveness.mddocs/scripts.md
| check_run_liveness() { # <run-id> | ||
| local run_id=$1 | ||
| local run_out outcome awaiting_msg elapsed | ||
|
|
||
| run_out=$(nm_run axi status --run "$run_id") || return 2 | ||
|
|
||
| [ -n "$run_out" ] || return 2 | ||
|
|
||
| outcome=$(strip_quotes "$(nm_field "$run_out" outcome)") | ||
|
|
||
| # Terminal outcomes are not hung | ||
| if [ -n "$outcome" ]; then | ||
| return 0 | ||
| fi | ||
|
|
||
| # Check if awaiting_agent (parked at gate) - not hung, deliberately waiting | ||
| awaiting_msg=$(printf '%s\n' "$run_out" | grep -E '^[[:space:]]*awaiting_agent:' | head -1) | ||
| if [ -n "$awaiting_msg" ]; then | ||
| return 0 | ||
| fi | ||
|
|
||
| # Check active_steps table first (more accurate than steps table) | ||
| local active_steps_section active_step_line last_activity_field | ||
| active_steps_section=$(printf '%s\n' "$run_out" | sed -n '/^[[:space:]]*active_steps\[/,/^[^[:space:]]/p') | ||
|
|
||
| if [ -n "$active_steps_section" ]; then | ||
| active_step_line=$(printf '%s\n' "$active_steps_section" | grep -E '^[[:space:]]*[^,]+,(running|fixing),' | head -1 | sed 's/^[[:space:]]*//') | ||
|
|
||
| if [ -n "$active_step_line" ]; then | ||
| # Found an active running/fixing step. Extract last_activity. | ||
| # Format: step,status,active_for,"time ago: message","pid",round | ||
| if [[ "$active_step_line" =~ ^[^,]*,[^,]*,[^,]*,\"([^\"]*) ]]; then | ||
| last_activity_field="${BASH_REMATCH[1]}" | ||
| if [ -n "$last_activity_field" ]; then | ||
| elapsed=$(parse_active_steps_time "$last_activity_field") | ||
| if [ "$elapsed" -gt "$STALE_THRESHOLD" ]; then | ||
| return 1 # Hung | ||
| fi | ||
| return 0 # Live | ||
| fi | ||
| fi | ||
| fi | ||
| fi | ||
|
|
||
| # Fallback: check steps table if no active_steps found | ||
| local step_line step_status duration | ||
| step_line=$(printf '%s\n' "$run_out" | grep -E '^[[:space:]]*[^,]+,[[:space:]]*"?(running|fixing)"?[[:space:]]*,' | head -1 | sed 's/^ *//') | ||
|
|
||
| if [ -z "$step_line" ]; then | ||
| # No running/fixing step - not hung | ||
| return 0 | ||
| fi | ||
|
|
||
| # Extract step status and duration | ||
| step_status="${step_line#*,}" | ||
| step_status="${step_status%,*}" | ||
| step_status=$(strip_quotes "$(trim "$step_status")") | ||
|
|
||
| # Skip findings count (third field) and get duration | ||
| local rest="${step_line#*,}" | ||
| rest="${rest#*,}" | ||
| rest="${rest#*,}" | ||
| duration=$(trim "$rest") | ||
|
|
||
| case "$step_status" in | ||
| running|fixing) | ||
| if [ -n "$duration" ] && [[ "$duration" =~ ^[0-9]+$ ]]; then | ||
| elapsed=$((duration / 1000)) | ||
| if [ "$elapsed" -gt "$STALE_THRESHOLD" ]; then | ||
| return 1 # Hung | ||
| fi | ||
| fi | ||
| ;; | ||
| esac | ||
|
|
||
| return 0 # Live | ||
| } |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift
Duplicated check_run_liveness with diverging thresholds. bin/fm-nm-run-is-live.sh sources bin/fm-no-mistakes-liveness.sh and then redefines check_run_liveness. The two copies of the steps-table fallback disagree, so the same run can be reported live by one script and hung by the other. docs/configuration.md line 406 documents FM_NM_LIVENESS_STALE as the hung threshold, and the base copy ignores it.
bin/fm-nm-run-is-live.sh#L23-L99: delete this redefinition and call the shared function frombin/fm-no-mistakes-liveness.sh. If the facade needs exit code 2 for a query failure, map the shared function's result instead of reimplementing the decision path.bin/fm-no-mistakes-liveness.sh#L271-L305: replace the hardcoded1800withSTALE_THRESHOLD, and treatrunningas hung-eligible in the fallback, so the single shared implementation honorsFM_NM_LIVENESS_STALEfor both step statuses.
📍 Affects 2 files
bin/fm-nm-run-is-live.sh#L23-L99(this comment)bin/fm-no-mistakes-liveness.sh#L271-L305
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-nm-run-is-live.sh` around lines 23 - 99, Remove the duplicate
check_run_liveness definition in bin/fm-nm-run-is-live.sh lines 23-99 and invoke
the shared check_run_liveness from bin/fm-no-mistakes-liveness.sh, mapping its
result to exit code 2 for query failures if needed. In
bin/fm-no-mistakes-liveness.sh lines 271-305, replace the fallback’s hardcoded
1800 threshold with STALE_THRESHOLD and include running alongside fixing as
hung-eligible, so the shared implementation applies FM_NM_LIVENESS_STALE
consistently.
| usage() { | ||
| sed -n '2,11p' "$0" | sed 's/^# //' | ||
| exit 2 | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
--help omits the usage syntax and exit codes.
sed -n '2,11p' stops at line 11. The Usage: block (lines 13-16) and the Exit codes: block (lines 18-21) are never printed. Extend the range.
🐛 Proposed fix
usage() {
- sed -n '2,11p' "$0" | sed 's/^# //'
+ sed -n '2,24p' "$0" | sed 's/^#$//; s/^# //'
exit 2
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| usage() { | |
| sed -n '2,11p' "$0" | sed 's/^# //' | |
| exit 2 | |
| } | |
| usage() { | |
| sed -n '2,24p' "$0" | sed 's/^#$//; s/^# //' | |
| exit 2 | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 67 - 70, Extend the sed range in
the usage() function so --help includes the complete Usage and Exit codes
documentation, covering the lines after the current line 11 cutoff while
preserving the existing comment-format stripping and exit behavior.
| HAVE_TIMEOUT=none | ||
| if command -v timeout >/dev/null 2>&1; then HAVE_TIMEOUT=timeout | ||
| elif command -v gtimeout >/dev/null 2>&1; then HAVE_TIMEOUT=gtimeout | ||
| elif command -v perl >/dev/null 2>&1; then HAVE_TIMEOUT=perl | ||
| fi | ||
| nm_run() { # <args...> | ||
| case "$HAVE_TIMEOUT" in | ||
| timeout) timeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | ||
| gtimeout) gtimeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | ||
| perl) perl -e 'my $t = shift; my $pid = fork; die "fork failed" unless defined $pid; if (!$pid) { setpgrp(0, 0); exec @ARGV } local $SIG{ALRM} = sub { kill "TERM", -$pid; select undef, undef, undef, 0.2; kill "KILL", -$pid; exit 124 }; alarm $t; waitpid $pid, 0; exit($? >> 8)' "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | ||
| *) true ;; | ||
| esac | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
No-timeout hosts never run no-mistakes, and || true hides all failures.
Two defects in this helper:
- If neither
timeout,gtimeout, norperlexists, the*)branch runstrue.no-mistakesis never called.check_all_runsthen printsno runs foundand exits 0, so a hung run is reported as healthy. || truediscards the exit status. This makesruns_out=$(nm_run runs --limit 50) || die ...(Line 314) andrun_out=$(nm_run axi status --run "$run_id") || return 2inbin/fm-nm-run-is-live.sh(Line 27) dead code. Callers cannot distinguish a CLI failure from empty output.
Run no-mistakes unbounded when no timeout wrapper exists, and preserve the exit status.
🐛 Proposed fix
nm_run() { # <args...>
case "$HAVE_TIMEOUT" in
- timeout) timeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;;
- gtimeout) gtimeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;;
- perl) perl -e '...' "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;;
- *) true ;;
+ timeout) timeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;;
+ gtimeout) gtimeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;;
+ perl) perl -e '...' "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;;
+ *) no-mistakes "$@" 2>/dev/null ;;
esac
}Keep the existing perl one-liner body unchanged; only the || true suffix is removed.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| HAVE_TIMEOUT=none | |
| if command -v timeout >/dev/null 2>&1; then HAVE_TIMEOUT=timeout | |
| elif command -v gtimeout >/dev/null 2>&1; then HAVE_TIMEOUT=gtimeout | |
| elif command -v perl >/dev/null 2>&1; then HAVE_TIMEOUT=perl | |
| fi | |
| nm_run() { # <args...> | |
| case "$HAVE_TIMEOUT" in | |
| timeout) timeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | |
| gtimeout) gtimeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | |
| perl) perl -e 'my $t = shift; my $pid = fork; die "fork failed" unless defined $pid; if (!$pid) { setpgrp(0, 0); exec @ARGV } local $SIG{ALRM} = sub { kill "TERM", -$pid; select undef, undef, undef, 0.2; kill "KILL", -$pid; exit 124 }; alarm $t; waitpid $pid, 0; exit($? >> 8)' "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null || true ;; | |
| *) true ;; | |
| esac | |
| } | |
| HAVE_TIMEOUT=none | |
| if command -v timeout >/dev/null 2>&1; then HAVE_TIMEOUT=timeout | |
| elif command -v gtimeout >/dev/null 2>&1; then HAVE_TIMEOUT=gtimeout | |
| elif command -v perl >/dev/null 2>&1; then HAVE_TIMEOUT=perl | |
| fi | |
| nm_run() { # <args...> | |
| case "$HAVE_TIMEOUT" in | |
| timeout) timeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;; | |
| gtimeout) gtimeout "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;; | |
| perl) perl -e 'my $t = shift; my $pid = fork; die "fork failed" unless defined $pid; if (!$pid) { setpgrp(0, 0); exec `@ARGV` } local $SIG{ALRM} = sub { kill "TERM", -$pid; select undef, undef, undef, 0.2; kill "KILL", -$pid; exit 124 }; alarm $t; waitpid $pid, 0; exit($? >> 8)' "$NM_TIMEOUT" no-mistakes "$@" 2>/dev/null ;; | |
| *) no-mistakes "$@" 2>/dev/null ;; | |
| esac | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 73 - 85, Update nm_run so the
fallback branch executes no-mistakes directly without a timeout when
HAVE_TIMEOUT is unavailable, instead of running true. Remove the || true suffix
from every timeout and perl invocation so each command preserves its exit status
for callers such as check_all_runs and command substitutions.
| if [ -f "$FM_HOME/data/secondmates.md" ] 2>/dev/null; then | ||
| while IFS= read -r line; do | ||
| [ -n "$line" ] || continue | ||
| case "$line" in '#'*|'- '*) continue ;; esac | ||
| # Parse secondmate entry: "- [id](path) — description" | ||
| # Simple extraction: look for [...] and (...) patterns | ||
| if [[ "$line" =~ \[([^\]]+)\]\(([^\)]+)\) ]]; then | ||
| local sm_path="${BASH_REMATCH[2]}" | ||
| [ -d "$sm_path/state" ] && homes_to_check+=("$sm_path/state") | ||
| fi | ||
| done < "$FM_HOME/data/secondmates.md" | ||
| fi |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
The secondmate-home scan is unreachable.
Line 109 skips every line that starts with - . The documented entry format at line 110 is - [id](path) — description, so all secondmate entries are skipped and the regex at line 112 never runs. Secondmate-owned runs always report owned-by: unknown.
Remove the '- '* pattern from the skip list. The 2>/dev/null on the [ -f ... ] test at line 106 has no effect and can go.
🐛 Proposed fix
- if [ -f "$FM_HOME/data/secondmates.md" ] 2>/dev/null; then
+ if [ -f "$FM_HOME/data/secondmates.md" ]; then
while IFS= read -r line; do
[ -n "$line" ] || continue
- case "$line" in '#'*|'- '*) continue ;; esac
+ case "$line" in '#'*) continue ;; esac📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if [ -f "$FM_HOME/data/secondmates.md" ] 2>/dev/null; then | |
| while IFS= read -r line; do | |
| [ -n "$line" ] || continue | |
| case "$line" in '#'*|'- '*) continue ;; esac | |
| # Parse secondmate entry: "- [id](path) — description" | |
| # Simple extraction: look for [...] and (...) patterns | |
| if [[ "$line" =~ \[([^\]]+)\]\(([^\)]+)\) ]]; then | |
| local sm_path="${BASH_REMATCH[2]}" | |
| [ -d "$sm_path/state" ] && homes_to_check+=("$sm_path/state") | |
| fi | |
| done < "$FM_HOME/data/secondmates.md" | |
| fi | |
| if [ -f "$FM_HOME/data/secondmates.md" ]; then | |
| while IFS= read -r line; do | |
| [ -n "$line" ] || continue | |
| case "$line" in '#'*) continue ;; esac | |
| # Parse secondmate entry: "- [id](path) — description" | |
| # Simple extraction: look for [...] and (...) patterns | |
| if [[ "$line" =~ \[([^\]]+)\]\(([^\)]+)\) ]]; then | |
| local sm_path="${BASH_REMATCH[2]}" | |
| [ -d "$sm_path/state" ] && homes_to_check+=("$sm_path/state") | |
| fi | |
| done < "$FM_HOME/data/secondmates.md" | |
| fi |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 106 - 117, Update the
secondmate-home scan in the surrounding shell block to stop skipping lines
beginning with "- ", while continuing to ignore comment lines and blank lines so
the documented "- [id](path) — description" entries reach the existing regex
extraction. Remove the ineffective 2>/dev/null redirection from the [ -f
"$FM_HOME/data/secondmates.md" ] test.
| parse_active_steps_time() { # <active_steps_line> | ||
| local line=$1 | ||
| # Look for patterns like "2s ago:" or "30m ago:" | ||
| if [[ "$line" =~ ([0-9]+)(s|m|h)\ ago: ]]; then | ||
| local value="${BASH_REMATCH[1]}" | ||
| local unit="${BASH_REMATCH[2]}" | ||
| case "$unit" in | ||
| s) printf '%d' "$value" ;; | ||
| m) printf '%d' "$((value * 60))" ;; | ||
| h) printf '%d' "$((value * 3600))" ;; | ||
| *) printf '0' ;; | ||
| esac | ||
| else | ||
| printf '0' | ||
| fi | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find other parsers of no-mistakes relative timestamps to confirm the supported unit set.
set -u
rg -n --glob '*.sh' -e 'ago' -e '\[0-9\]\+\)\(s\|m\|h' -e 'active_steps' bin | head -60
fd -e md . docs --exec rg -n -l 'active_steps' {} \;Repository: trillium/firstmate
Length of output: 2841
🏁 Script executed:
#!/bin/bash
set -u
echo "== repository files of interest =="
git ls-files bin docs | sed -n '1,120p'
echo
echo "== fm-no-mistakes-liveness.sh relevant sections =="
cat -n bin/fm-no-mistakes-liveness.sh | sed -n '1,340p'
echo
echo "== fm-nm-run-is-live.sh relevant sections =="
cat -n bin/fm-nm-run-is-live.sh | sed -n '1,120p'
echo
echo "== search no-mistakes CLI/docs for active_steps examples =="
rg -n -C 3 'active_steps|no-mistakes|mistakes|active_for|ago:' . --glob '!*.log' | sed -n '1,220p'Repository: trillium/firstmate
Length of output: 40342
🏁 Script executed:
#!/bin/bash
set -u
echo "== no-mistakes version declaration =="
rg -n "no-mistakes|v\d+\.\d+\.\d+|1\.31\.2|31\.2" . --glob '!*.log' | sed -n '1,160p'
echo
echo "== parse_active_steps_time usage in tests =="
rg -n "parse_active_steps_time|active_steps|last_activity|ago:" bin tests --glob '*.sh' --glob '*.md' | sed -n '1,220p'
echo
echo "== local bash regex behavior for active_for fields =="
bash -c '
parse_active_steps_time() {
local line=$1
if [[ "$line" =~ ([0-9]+)(s|m|h)\ ago: ]]; then
local value="${BASH_REMATCH[1]}"
local unit="${BASH_REMATCH[2]}"
case "$unit" in
s) printf "%d" "$value" ;;
m) printf "%d" "$((value * 60))" ;;
h) printf "%d" "$((value * 3600))" ;;
*) printf "%d" "UNREACHABLE" ;;
esac
else
printf "NO_MATCH"
fi
}
for s in "2s ago: log: message" "30m ago: log: message" "2h ago: log: message" "2d ago: log: message" "1h30m ago: log: message"; do
printf "%s => %s\n" "$s" "$(parse_active_steps_time "$s")"
done
'Repository: trillium/firstmate
Length of output: 25248
🌐 Web query:
no-mistakes axi status active_steps active_for time ago output examples
💡 Result:
In the no-mistakes framework, the command no-mistakes axi status provides real-time observability into an active validation pipeline run [1][2][3]. When a step is actively running or fixing, the status output may include an active_steps table [2][4]. This table typically contains the following fields: - active_for: The active duration of the step [1][2]. - last_activity: The latest meaningful activity from log lines or native subprocess lifecycle events [1][2]. If no activity has been detected for longer than the configured step_quiet_warning, this field is prefixed with "quiet" [1][2]. - agent_pid: The native agent PID if a subprocess agent is currently running [1][2]. - round: The current execution or fix round (e.g., "round 1", "auto-fix 1/3", or "fix 2") [1][2]. If a run is parked at an approval or fix-review gate, the status output will instead display awaiting_agent: parked [1][2]. This field indicates that the run is waiting for you to execute no-mistakes axi respond [1][2]. The no-mistakes axi command family uses TOON (a machine-readable format) for output [3][5][6]. You should use no-mistakes axi status to check progress without interrupting long-running steps, rather than canceling or re-issuing the command [2][3].
Citations:
- 1: https://kunchenguid.github.io/no-mistakes/concepts/gate-model/
- 2: https://claudeskills.info/skills/kunchenguid/no-mistakes/no-mistakes/
- 3: https://explainx.ai/skills/kunchenguid/no-mistakes/no-mistakes
- 4: https://tessl.io/registry/skills/github/kunchenguid/no-mistakes/no-mistakes
- 5: https://kunchenguid.github.io/no-mistakes/start-here/quick-start/
- 6: https://kunchenguid.github.io/no-mistakes/start-here/introduction/
🌐 Web query:
site:github.com/kunchenguid/no-mistakes "active_for" no-mistakes ago
💡 Result:
The term active_for does not exist as a standard keyword, command, or documented parameter in the no-mistakes software project [1][2]. Based on the available documentation and codebase for no-mistakes (a tool for gating and automating code pipelines), the term "active" is frequently used in technical contexts such as: * Active runs: The system tracks "active runs" of a pipeline [3][4]. The CLI provides commands like axi run to attach to or start these runs [1][5], and there are safeguards (e.g., guard destructive daemon lifecycle commands against active runs) to prevent interrupting them [4]. * Active vs. Terminal: Documentation distinguishes between active (in-progress) and terminal (finished) pipeline runs [3][6]. It is possible that "active_for" is a misinterpretation of a phrase like "active for [branch name]" or "active run for [ID]" found in logs or error messages produced by the daemon [1]. If you encountered this term in a specific log or error, it likely refers to a reference to an active pipeline run associated with a specific entity (such as a git branch) [1][3].
Citations:
- 1: bug:
axi runissuesrerun(notstart_run) — cannot start a new run on a branch with no history kunchenguid/no-mistakes#351 - 2: https://github.com/kunchenguid/no-mistakes/
- 3: fix(cli): reattach active AXI runs by submitted head kunchenguid/no-mistakes#514
- 4: kunchenguid/no-mistakes@493fc69
- 5: fix(daemon): recover runless gate branches kunchenguid/no-mistakes#409
- 6: feat(branchsync): recover custody for terminal pipeline-owned runs kunchenguid/no-mistakes#502
Handle larger active_steps time units before treating stale runs as live.
parse_active_steps_time() only matches s, m, and h, but day-scale activity such as 2d ago: falls through to 0. check_run_liveness() then reports the run as live, so long-stalled runs escape detection. Add d and wider units, or return a sentinel that forces the caller to treat the parse failure as stale/hung. Also handle composite durations such as 1h30m ago: if they can appear.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 197 - 212, Update
parse_active_steps_time() to recognize day-scale and larger active_steps
durations, converting them to seconds so check_run_liveness() correctly
identifies long-stalled runs as stale. Support composite formats such as “1h30m
ago:” when present, or return a sentinel that check_run_liveness() explicitly
treats as stale/hung instead of live.
| # Try to find a task with this branch and get its run ID | ||
| task=$(find_task_for_branch "$branch") | ||
| if [ "$task" != "unknown" ]; then | ||
| # Found a task - get its run ID from axi status | ||
| local task_id="${task##*/}" | ||
| local meta_file="$STATE/$task_id.meta" | ||
| if [ ! -f "$meta_file" ] && [ -d "$FM_HOME/state" ]; then | ||
| meta_file="$FM_HOME/state/$task_id.meta" | ||
| fi | ||
| if [ -f "$meta_file" ]; then | ||
| local worktree | ||
| worktree=$(grep "^worktree=" "$meta_file" | tail -1 | cut -d= -f2-) | ||
| if [ -n "$worktree" ] && [ -d "$worktree" ]; then | ||
| local axi_out | ||
| axi_out=$(cd "$worktree" && nm_run axi status 2>/dev/null || true) | ||
| if [ -n "$axi_out" ]; then | ||
| run_id=$(strip_quotes "$(nm_field "$axi_out" id)") | ||
| fi | ||
| fi | ||
| fi | ||
| fi |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Run-ID resolution re-derives the meta path and fails for non-primary homes.
find_task_for_branch already found the matching .meta file and its worktree. This block throws that away, strips the home qualifier at line 340, then searches only $STATE and $FM_HOME/state. For a secondmate or project task the meta file lives elsewhere, so run_id stays unknown and liveness is never evaluated for that run.
Return the resolved worktree from find_task_for_branch and reuse it here. Consider a second output field, for example task_id<TAB>worktree, so both callers avoid the duplicate lookup.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 336 - 356, Update
find_task_for_branch to return both the resolved task identifier and worktree,
using a stable delimiter such as a tab, and parse both values at the call site.
Replace the local task_id/meta_file lookup in the run-ID resolution block with
the returned worktree, then run axi status directly there so secondmate and
project tasks reuse the already-resolved path.
| elapsed="unknown" | ||
| liveness="unknown" | ||
| fi | ||
|
|
||
| printf 'run: %s · branch: %s · status: %s · owned-by: %s · last-activity: %ss\n' \ | ||
| "$run_id" "$branch" "$liveness" "$task" "$elapsed" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Output prints unknowns and uses undocumented state names.
Two output defects:
- When
run_idisunknown,elapsed="unknown"and the%ssformat renderslast-activity: unknowns. - The header comment at line 24 documents
status: <running|hung|done|failed>, but this line printslive,hung, orunknown.
Emit last-activity: unknown without the s, and align the header comment with the printed values.
🐛 Proposed fix
- printf 'run: %s · branch: %s · status: %s · owned-by: %s · last-activity: %ss\n' \
- "$run_id" "$branch" "$liveness" "$task" "$elapsed"
+ local activity="${elapsed}s"
+ [ "$elapsed" = "unknown" ] && activity="unknown"
+ printf 'run: %s · branch: %s · status: %s · owned-by: %s · last-activity: %s\n' \
+ "$run_id" "$branch" "$liveness" "$task" "$activity"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@bin/fm-no-mistakes-liveness.sh` around lines 369 - 374, Update the output
formatting in the liveness reporting block so an unknown elapsed value prints
“last-activity: unknown” without appending the seconds suffix, while numeric
values retain the “s” suffix. Also revise the header comment’s documented status
values to match the actual values emitted by the liveness logic, including live,
hung, and unknown.
| ``` | ||
| run: <run-id> · branch: <branch> · status: <live|hung|unknown> · owned-by: <task-id> · last-activity: <seconds>s | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add a language to the fenced code block.
Markdownlint reports MD040 for Line 26. Use text for this output example.
Proposed fix
-```
+```text📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ``` | |
| run: <run-id> · branch: <branch> · status: <live|hung|unknown> · owned-by: <task-id> · last-activity: <seconds>s | |
| ``` |
🧰 Tools
🪛 markdownlint-cli2 (0.23.1)
[warning] 26-26: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/no-mistakes-run-liveness.md` around lines 26 - 28, Update the fenced
code block containing the run status output example to declare the `text`
language, changing the opening fence while preserving the example content and
closing fence.
Source: Linters/SAST tools
| **Exit codes:** | ||
| - 0 = success, all checked runs are live | ||
| - 1 = at least one run is hung | ||
| - 2 = usage error or precondition failure |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 8 'unknown|exit [012]|return [012]|FM_NM_LIVENESS' \
bin/fm-no-mistakes-liveness.sh bin/fm-nm-run-is-live.shRepository: trillium/firstmate
Length of output: 27593
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- docs excerpt ---\n'
sed -n '1,120p' docs/no-mistakes-run-liveness.md | cat -n
printf '\n--- supervisor references ---\n'
rg -n -C 9 'fm-nm-run-is-live\.sh|no-mistakes|daemon|hung|hanging|liveness|unknown' docs/no-mistakes-run-liveness.md
printf '\n--- all unknown/status references in docs ---\n'
rg -n -C 5 '\bunknown\b|status:\s*<|owned-by:\s*<|last-activity:' docs/no-mistakes-run-liveness.mdRepository: trillium/firstmate
Length of output: 247
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- docs excerpt ---'
sed -n '1,120p' docs/no-mistakes-run-liveness.md | cat -n
printf '%s\n' ''
printf '%s\n' '--- supervisor references ---'
rg -n -C 9 'fm-nm-run-is-live\.sh|no-mistakes|daemon|hung|hanging|liveness|unknown' docs/no-mistakes-run-liveness.md
printf '%s\n' ''
printf '%s\n' '--- all unknown/status references in docs ---'
rg -n -C 5 '\bunknown\b|status:\s*<|owned-by:\s*<|last-activity:' docs/no-mistakes-run-liveness.mdRepository: trillium/firstmate
Length of output: 12000
Preserve the usage failure path in supervision.
fm-nm-run-is-live.sh exits with 2 for usage/precondition failures, but the supervision example treats every non-zero result as hung. Branch on the exact status and escalate only exit code 1, or document that usage/precondition failures also require manual intervention.
status: unknown is a documented multi-run output state, but no outcome mapping is in the protocol text. Define whether unknown ownership is fail-closed for daemon-affecting actions or has a separate handling rule.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/no-mistakes-run-liveness.md` around lines 30 - 33, Update the
supervision example and protocol text in docs/no-mistakes-run-liveness.md to
distinguish exit code 1 (hung) from exit code 2 (usage or precondition failure),
escalating only the intended status or explicitly documenting manual
intervention for status 2. Also define the outcome mapping for status: unknown,
including whether it is fail-closed for daemon-affecting actions or follows a
separate handling rule.
| **Exit codes:** | ||
| - 0 = run is live/progressing (or not in a step that can hang) | ||
| - 1 = run is hung (step inactive for > 5 minutes) | ||
| - 2 = usage error |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Document the configured stale threshold.
Line 46 says a run is hung after five minutes, but FM_NM_LIVENESS_STALE can change this value. State “greater than the configured stale threshold, 300 seconds by default.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/no-mistakes-run-liveness.md` around lines 44 - 47, Update the exit-code
documentation to describe hung runs as inactive for greater than the configured
stale threshold, with 300 seconds as the default, rather than always stating
five minutes.
6835fd3 to
adfb815
Compare
1 participant
Intent
Add no-mistakes run liveness checking tools: two scripts (fm-no-mistakes-liveness.sh for full inventory, fm-nm-run-is-live.sh for lightweight single-run checks) that detect hung vs genuinely-progressing runs using active_steps last_activity timestamps, match runs to tasks by branch name and worktree state, handle primary and secondmate homes, and provide structured exit codes. Plus documentation explaining usage, integration points, and incident prevention protocols. Solves the 2026-07-31 incident where a live run (fold-slice4-shims) was silently killed by daemon restart because firstmate had no way to distinguish it from an old entry in no-mistakes runs.
What Changed
bin/fm-no-mistakes-liveness.sh: comprehensive inventory of run liveness and ownership across primary and secondmate homes, detecting hung vs actively progressing runs usingactive_stepslast_activity timestamps and matching runs to tasks by branch name and worktree state.bin/fm-nm-run-is-live.sh: lightweight single-run liveness check for supervision polling with identical detection logic, enabling real-time validation during daemon operations.docs/no-mistakes-run-liveness.md, and updated configuration/scripts indices) covering usage, detection algorithms, integration points, and incident prevention protocols.Risk Assessment
✅ Low: The code is clean, all previous fixes were correctly applied, and no new bugs were introduced. The two new scripts provide well-designed tools for detecting hung no-mistakes runs with proper error handling, timeout management, and task ownership resolution across multiple homes.
Testing
Validated both scripts through interface testing, parsing function verification, task ownership resolution, and documentation coverage. All core features work correctly end-to-end: active_steps timestamp detection, hung vs. live classification using 300s stale threshold, task matching by branch, structured exit codes, and comprehensive incident prevention documentation with 2026-07-31 context.
Evidence: validation-evidence.md
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 6 issues found → auto-fixed (2) ✅
bin/fm-no-mistakes-liveness.sh:370- Hardcoded absolute path /Users/trilliumsmith/code/firstmate/state will fail on other systems and repo locations. Should use FM_ROOT or other configurable basis.bin/fm-nm-run-is-live.sh:54- Unused function get_timestamp() defined but never called; remove to reduce code size.bin/fm-nm-run-is-live.sh:62- Unused function parse_log_timestamp() defined but never called; remove to reduce code size.bin/fm-nm-run-is-live.sh:33- Code duplication: trim(), strip_quotes(), and nm_field() are copied instead of sourced from fm-no-mistakes-liveness.sh or a shared helper. This creates maintenance burden and risk of divergence.bin/fm-nm-run-is-live.sh:26- Hardcoded timeout value 15 instead of using configurable FM_NM_LIVENESS_TIMEOUT variable like fm-no-mistakes-liveness.sh (line 37). Reduces flexibility for different network/load conditions.bin/fm-nm-run-is-live.sh:107- Lightweight script skips active_steps table parsing entirely, going straight to steps table (line 109). This is less accurate than fm-no-mistakes-liveness.sh. Both should use same detection logic for consistency.🔧 Fix: Fixed hardcoded path, unused functions, inconsistent timeout config, and liveness detection logic
6 issues (3 errors, 3 warnings) still open:
bin/fm-no-mistakes-liveness.sh:252- check_run_liveness returns 1 without printing elapsed time when nm_run fails, violating function contract. Causes $elapsed to be empty in callers, producing malformed output missing elapsed value.bin/fm-no-mistakes-liveness.sh:449- check_task_run calls nm_run axi status in wrong directory. Missing cd prefix means run status fetched from current dir, not task worktree. Compare to line 378 which correctly uses cd.bin/fm-nm-run-is-live.sh:84- check_run_liveness returns without printing when nm_run fails, violating contract. Same issue as fm-no-mistakes-liveness.sh line 252 with early return.bin/fm-no-mistakes-liveness.sh:94- get_timestamp() function never called. Dead code should be removed.bin/fm-no-mistakes-liveness.sh:103- parse_log_timestamp() function never called. Dead code should be removed.bin/fm-nm-run-is-live.sh:40- Helper functions (trim, strip_quotes, nm_field) duplicated instead of sourced from fm-no-mistakes-liveness.sh, creating maintenance burden and divergence risk.🔧 Fix: Fixed error handling, directory contexts, removed dead code, unified duplicate utilities
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
Script presence and executabilityScript help/usage interfacesError handling and exit codes (usage, missing tasks, unknown options)Parsing logic: parse_active_steps_time() for s/m/h formatsUtility functions: trim(), strip_quotes(), nm_field()Task ownership resolution by branch name matchingGraceful no-runs handling when daemon not initializedDocumentation completeness: sections, incident reference, usage protocolsFunction library sourcing capabilitiesSupport for primary and secondmate homes✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
Summary by CodeRabbit
New Features
Documentation