epic: improve-amplihack-test-coverage — baseline + per-crate targets

[rysweet]

Why this epic

We have been telling ourselves "let's improve test coverage" without a number. That makes any subsequent claim of "improvement" unfalsifiable. This epic establishes a concrete, reproducible coverage baseline for the amplihack Rust workspace (rysweet/amplihack-rs @ 844248d) and stakes out per-crate targets so future cycles can dispatch narrowly-scoped engineers against measurable goals.

This is a measurement and planning epic. No source code or tests are added in this cycle.

Methodology

Tool: cargo-llvm-cov 0.6.x (installed via cargo install cargo-llvm-cov --locked).

cd /path/to/amplihack-rs
cargo llvm-cov --workspace --ignore-run-fail --summary-only \
    --ignore-filename-regex 'tests?/|target/' \
    2>&1 | tee /tmp/amplihack-cov-baseline.txt

Notes on the invocation:

• --ignore-run-fail is used instead of the originally-suggested --no-fail-fast because cargo-llvm-cov refuses both flags together; with --ignore-run-fail, the two consistently-failing test binaries (-p amplihack --test no_python_probe, -p amplihack-cli --test issue_625_update_prompt_subprocess_safe) do not abort summary generation. Those failures pre-exist this measurement and are out of scope for this epic — but they should be investigated separately.
• --ignore-filename-regex 'tests?/|target/' excludes test scaffolding and generated artifacts from the per-file table.
• The --json variant was attempted but aborted under disk pressure (workspace already exceeds 30 GB of build artifacts on this box). The per-file hotspot list below was therefore derived from the text-format summary, which contains identical raw counts. Re-run with --json > /tmp/amplihack-cov-baseline.json on a host with ≥40 GB free if you need the structured form.

Repository layout note: the originating prompt referenced crates/amplihack-*/src/. The amplihack Rust workspace lives in rysweet/amplihack-rs (not in this Simard repo, which is a single crate). This epic is filed in rysweet/Simard to stay co-located with related coordination issues (#1927, #1916, #1901), all of which are tracked here.

Workspace baseline

Metric	Value
Total executable lines (instrumented)	116,923
Missed lines	23,403
Line coverage	79.98%
Total regions	184,435
Missed regions	37,105
Region coverage	79.88%
Total functions	13,531
Function coverage	79.94%

The workspace is in a much healthier place than "0% measured" suggested — but the per-crate breakdown reveals concentrated gaps worth attacking.

Per-crate baseline (sorted asc by line %)

Crate	Files	Lines	Line %	Region %	Func %
amplihack-asset-resolver	1	14	0.00%	0.00%	0.00%
amplihack-remote	12	2,609	38.06%	38.84%	40.20%
amplihack-reflection	9	510	57.65%	52.38%	59.38%
amplihack-multilspy	13	1,826	63.36%	65.62%	49.85%
amplihack (bin)	2	181	64.09%	66.22%	76.92%
amplihack-blarify	39	5,924	65.97%	65.15%	71.49%
amplihack-state	5	477	72.12%	79.52%	70.13%
amplihack-cli	254	41,892	73.11%	72.47%	72.14%
amplihack-agent-generator	12	1,743	76.99%	76.98%	77.34%
amplihack-utils	36	5,609	78.11%	79.16%	85.98%
amplihack-fleet	5	1,044	80.36%	82.08%	83.11%
amplihack-launcher	31	5,394	81.41%	82.67%	81.91%
amplihack-session	7	1,715	84.61%	83.54%	76.44%
amplihack-safety	3	541	84.66%	83.56%	95.35%
amplihack-builders	6	389	84.83%	81.52%	75.51%
amplihack-memory	37	6,700	87.42%	85.40%	86.17%
amplihack-hooks	48	6,800	88.71%	88.00%	88.28%
amplihack-domain-agents	16	2,722	90.15%	90.72%	90.68%
amplihack-orchestration	13	2,098	90.42%	90.63%	87.60%
amplihack-agent-core	60	10,443	90.43%	90.59%	90.71%
amplihack-hive	37	5,319	92.39%	92.83%	91.40%
amplihack-workflows	6	1,103	93.20%	94.65%	94.02%
amplihack-context	6	956	93.51%	93.55%	88.00%
amplihack-delegation	9	1,588	93.89%	93.54%	88.20%
amplihack-types	4	345	94.49%	94.87%	91.38%
amplihack-recovery	7	1,081	94.63%	93.94%	96.19%
amplihack-agent-eval	37	6,951	95.05%	94.53%	93.36%
amplihack-security	5	949	97.05%	95.21%	94.62%

Top-10 lowest-coverage hotspots (>50 executable lines, under crates/amplihack-*/src/)

Rank	File	Lines	Missed	Line %	Region %
1	crates/amplihack-cli/src/commands/fleet/tui_main.rs	258	258	0.00%	0.00%
2	crates/amplihack-cli/src/commands/auto_mode/session.rs	243	243	0.00%	0.00%
3	crates/amplihack-cli/src/commands/remote.rs	183	183	0.00%	0.00%
4	crates/amplihack-cli/src/commands/query_code/runners.rs	163	163	0.00%	0.00%
5	crates/amplihack-cli/src/commands/fleet/commands_scout.rs	134	134	0.00%	0.00%
6	crates/amplihack-cli/src/commands/memory/agent_kv.rs	93	93	0.00%	0.00%
7	crates/amplihack-cli/src/commands/builder.rs	81	81	0.00%	0.00%
8	crates/amplihack-cli/src/commands/reflect.rs	69	69	0.00%	0.00%
9	crates/amplihack-reflection/src/reflection.rs	55	55	0.00%	0.00%
10	crates/amplihack-cli/src/commands/multitask/launcher.rs	155	149	3.87%	6.27%

Pattern observation: 8 of the top 10 are CLI dispatch modules (crates/amplihack-cli/src/commands/...) that are reachable only by running the binary. They have zero unit tests because they are thin wiring layers, and zero integration coverage because no end-to-end harness exercises the corresponding amplihack <subcommand> invocations. The fix is not "add unit tests to each file" — it is "add a subprocess-style integration test that drives the CLI binary," which will pull most of these modules over 0% in one move. See child issue #1940.

The two non-CLI hotspots (amplihack-reflection/src/reflection.rs and amplihack-cli/src/commands/multitask/launcher.rs) are legitimately untested orchestrator code and warrant their own per-file work — see #1938.

Additional larger files worth flagging (>200 lines, <40% line coverage):

• crates/amplihack-remote/src/executor.rs — 340 lines, 12.26% line, 17.95% region (Azure VM remote-exec, currently exercised only by #[ignore]d live tests)
• crates/amplihack-remote/src/orchestrator.rs — 300 lines, 14.40% line (VM lifecycle via azlin)
• crates/amplihack-remote/src/vm_pool.rs — 239 lines, 19.03% line
• crates/amplihack-remote/src/integrator.rs — 353 lines, 24.08% line
• crates/amplihack-remote/src/packager.rs — 279 lines, 24.37% line

Together these motivate the amplihack-remote child issue #1939.

Proposed targets

Q starting baseline guard:

• No regression below the per-crate line % shown in the table above. Any PR that drops a listed crate's line coverage by more than 0.5 absolute percentage points must explicitly justify the drop in the PR body.

Quarter goal — worst-3 crates, +5% absolute line coverage:

Crate	Today	Target
amplihack-asset-resolver	0.00%	n/a — single 14-line main.rs bin; not a meaningful target, treat as a tracking exception
amplihack-remote	38.06%	≥43.06% line (see #1939)
amplihack-reflection	57.65%	≥62.65% line (see #1938)
amplihack-multilspy (4th-worst real crate)	63.36%	≥68.36% line — stretch target, no child issue this cycle

Rationale for the asset-resolver exception: it is a one-file 14-line binary that just wires amplihack-launcher → asset_resolver. There is no meaningful coverage to add without inventing busywork. Documenting the exception now prevents a future engineer from chasing it.

Coordination with existing work — do not duplicate

This epic does not duplicate, and explicitly defers to, the following open issues:

• adopt-tdd: establish TDD discipline (charter + pilot module + CI gate) #1927 — adopt-tdd: establish TDD discipline (charter + pilot module + CI gate). This epic supplies the measured baseline that adopt-tdd: establish TDD discipline (charter + pilot module + CI gate) #1927's pilot module work should be evaluated against. The TDD pilot is scoped to one module (Simard's meeting subsystem, per adopt-tdd: establish TDD discipline (charter + pilot module + CI gate) #1927's errata). The two efforts are complementary: adopt-tdd: establish TDD discipline (charter + pilot module + CI gate) #1927 enforces authoring order on new work; this epic measures the existing surface so we can pick the highest-leverage next module after the pilot completes.
• cognitive-memory-persistence (C): hermetic outside-in cross-session recall test gating every PR #1916 — cognitive-memory-persistence (C): hermetic outside-in cross-session recall test gating every PR. The cross-session recall test mandated by cognitive-memory-persistence (C): hermetic outside-in cross-session recall test gating every PR #1916 lives in the Simard crate, not in amplihack-rs. It will not directly move any line in the table above. Do not redirect cognitive-memory-persistence (C): hermetic outside-in cross-session recall test gating every PR #1916 effort toward amplihack-* coverage — finish cognitive-memory-persistence (C): hermetic outside-in cross-session recall test gating every PR #1916 on its current scope.
• amplihack-rs parity: PostToolUse workflow enforcement — behavioral contract test between Rust and Python strategies #1901 — amplihack-rs parity: PostToolUse workflow enforcement — behavioral contract test between Rust and Python strategies. The amplihack-hooks crate is already at 88.71% line / 88.00% region. amplihack-rs parity: PostToolUse workflow enforcement — behavioral contract test between Rust and Python strategies #1901 adds a contract test (Rust ↔ Python parity), which protects against drift but will only marginally bump the raw % on that crate. Do not retarget amplihack-rs parity: PostToolUse workflow enforcement — behavioral contract test between Rust and Python strategies #1901 into a coverage exercise.

Child issues filed in this cycle

• improve coverage: amplihack-remote (38.06% → ≥43.06% line) #1939 — improve coverage: amplihack-remote (38.06% → ≥43.06% line)
• improve coverage: amplihack-reflection (57.65% → ≥62.65% line) — reflection.rs orchestrator is 0% #1938 — improve coverage: amplihack-reflection (57.65% → ≥62.65% line) — reflection.rs orchestrator is 0%
• improve coverage: amplihack-cli/commands/* — 8 dispatch modules at 0% (≈1,400 dark lines) #1940 — improve coverage: amplihack-cli/commands/* — 8 dispatch modules at 0% (≈1,400 dark lines)

Reproduce

git clone https://github.com/rysweet/amplihack-rs.git
cd amplihack-rs
git checkout 844248d
cargo install cargo-llvm-cov --locked
cargo llvm-cov --workspace --ignore-run-fail --summary-only \
    --ignore-filename-regex 'tests?/|target/' \
    | tee /tmp/amplihack-cov-baseline.txt

Total wall-clock on a 16-core box with a warm cargo cache: ~20 min (instrumented build dominates). Disk: ≥30 GB free required.

Out of scope for this epic

• ❌ Adding any tests in this cycle — measurement only.
• ❌ Investigating the two pre-existing cargo test failures (no_python_probe, issue_625_update_prompt_subprocess_safe) — flagged here, but they get their own ticket.
• ❌ A CI coverage gate / per-crate threshold check — premature; revisit after the worst-3 targets are met.
• ❌ Coverage for bins/ and the amplihack integration bin — these are exercised by end-to-end harnesses that don't show up under unit-test coverage; address with integration-test instrumentation, not unit tests.

Artifacts

The raw cargo llvm-cov output (/tmp/amplihack-cov-baseline.txt, 664 KB) and the parsed per-crate / hotspot tables (/tmp/amplihack-cov-parsed.json, 12 KB) live on the engineer host that ran this measurement. They are not committed — re-run the command above to regenerate on demand. The numbers in this issue are the published baseline.