batch(a): consolidate 12 reviewed PRs into one merge-queue entry (43m48s each, serial queue) - #2534
Merged
Conversation
`check_book_examples_executable.sh` (FALSIFY-BOOK-EXAMPLE-EXECUTES-001) sources `scripts/apr_bin.sh`, which asks cargo for the target dir and asserts the binary's embedded SHA matches HEAD. That is the repo's binary-pinning protocol and it works. The gate then never used the result. `$APR_BIN` gated only the SKIP branch. Every example that DID run went through `timeout N bash -c "$code"`, and the code says `apr ...` — resolved by PATH. Measured on this box: a bare `apr` is 0.60.0 while the tree is 0.63.0, so the gate certifying the book's CLI examples was exercising a binary from three minor releases back. That is the exact failure this repo has hit four times (#2357/#2358/#2360/#2361) and the reason apr_bin.sh exists. Fix: prepend the pinned binary's directory to PATH. That covers `apr` in any position — pipelines, subshells, `$(...)` — which substituting a leading token does not. Verified: `bash -c 'apr --version'` reports 0.60.0 before and the pinned binary after. Second defect, worse, in the verdict itself. With no apr built from HEAD every CLI example skips, and the gate printed: total=244 pass=0 skip=244 fail=0 FALSIFY-BOOK-EXAMPLE-EXECUTES-001: PASS Zero executed, verdict green. It now refuses: * no apr binary -> "NOT RUN — nothing was verified", exit 1, with the build command to fix it * apr present but 0 of 244 executed -> FAIL, "the gate measured nothing" NOT wired into CI in this commit, deliberately. It needs a job that builds apr first, and when apr IS available this gate reports 5 real failures in book text. Wiring it before those are fixed turns main red. The order is: fix the 5, add a build step, then wire. This commit makes the gate honest so that sequence is possible at all — until now it could not have reported anything. Refs #2481 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bashrs flags a unicode em-dash as SC1100. Two were mine (the new NOT RUN and FAIL verdict lines); three predate this change. Fixed all nine occurrences so the file is genuinely clean rather than merely no-worse. Method note: my first with/without measurement was invalid -- I ran `git stash` AFTER committing, so both sides measured the same tree. Comparing against `git show origin/main:<file>` gave the real answer (3 pre-existing, 2 added). That stash pop also restored an unrelated 44-file stash into the worktree; reset --hard restored it and all 89 stash entries are intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…y directory
`check_package_includes.sh` is the CB-510 guard: a Cargo.toml `exclude` pattern
can strip an `include!()` target from the published crate while git still tracks
it, so the crate compiles in-tree and fails for everyone installing from
crates.io. That has shipped twice -- `models/` matching `src/models/`, and an
unanchored `"tests/"` dropping 443 files from published aprender-serve in 0.63.0.
It scanned `src/` and packaged `-p aprender`: the PRE-MONOREPO layout. After
consolidation the root `src/` holds 2 files with zero `include!()`, while 1798
live under `crates/`. So it reported, truthfully and uselessly:
OK: All 0 include!() files are included in cargo package
Zero of zero, exit 0, for every release since the consolidation -- while being
Gate 1 of `.claude/skills/pre-release/SKILL.md`.
It now enumerates publishable workspace crates from `cargo metadata`, resolves
every `include!()` against the including file's directory, and diffs against that
crate's OWN `cargo package --list`. 10 crates, 1539 include targets. A vacuity
assertion fails below 100 targets, so the empty-scan mode cannot return.
RESULT: the tree is CLEAN. Every one of the 1539 targets survives packaging.
Three bugs of my own, each caught by a mutation that refused to turn red. Worth
recording because each produced a confident wrong answer:
1. `cargo package` inside `while read ... <<< "$pkgs"` consumed the heredoc on
stdin. The scan silently dropped a crate (11 -> 10) AND compared one crate's
include targets against another crate's listing, inventing a CB-510 violation
on src/bench/backend.rs. Fixed with `< /dev/null`.
2. One forked `grep -qxF` per target -- 922 for aprender-serve alone -- treating
ANY non-zero as "not packaged". grep exits >1 on ERROR, and a forked grep can
die under load, so the guard named a different innocent file on each run.
3. Replacing that with python, I put a heredoc script AND a `<<<` data
redirection on the same call. Last redirection wins, so python received the
include list as its SCRIPT and printed nothing. The guard then PASSED a
mutation that provably dropped a file from the package -- I had written
another gate that cannot fail, inside the fix for a gate that cannot fail.
Both inputs now go through explicit files.
Mutation-verified, with the mutation itself proven to engage first
(`cargo package --list | grep -c` goes 1 -> 0 before the guard is consulted):
* exclude "src/bench/" -> RED, 27 files named
* scan a nonexistent dir -> RED on vacuity
* restored -> GREEN
Wired into the merge-blocking guard block. bashrs 0 errors; the three embedded
python fragments moved to scripts/lib/ because bashrs parses an inline heredoc as
shell (8 phantom SC1007s from python assignments, 2 SC1078s from nested quotes
across a line continuation).
Refs #2481, #2474
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…em gate anything
An exemption in `deny.toml` is a standing decision to accept a known
vulnerability. Twenty of the twenty-nine were for advisories that no longer fire
at all -- the dependency had been upgraded or dropped from the graph.
`RUSTSEC-2026-0002` is the clearest: it exempted
"lru 0.12.5: transitive via ratatui, fixed in 0.16 but ratatui pins 0.12"
while `Cargo.lock` already resolved **lru 0.16.4** -- the fixed version named in
its own rationale. The exemption described a world that had moved on, and a
reviewer reading deny.toml could not tell which of the 29 entries were
load-bearing.
`cargo deny` was already reporting every one of these, as `advisory-not-detected`
warnings. Nobody acted because they are warnings and the command exits 0.
Removed the 20; `cargo deny check advisories` exits 0 with **zero**
advisory-not-detected warnings and 9 live exemptions remaining.
Added `check_deny_exemptions_live.sh`, which turns that existing warning into a
gate. Deliberately separate from the advisory check itself: a newly-FIXED
upstream must never fail someone's build, so it fails only this guard, whose
remedy is deleting a line.
A larger finding, reported not fixed here. **These exemptions gate nothing in
CI.** `cargo deny` appears in ZERO workflows -- only `make deny` -- verified with
a positive control (9 workflows mention `cargo`, 0 mention `deny`). And per
ci.yml:8 the `security` job runs `cargo audit` with `continue-on-error`, which
cannot fail the build AND does not read deny.toml at all. So the advisory
surface today is: one tool that ignores the exemption file running in a job that
cannot fail, plus one tool that honours it running nowhere.
Wiring cargo-deny into CI needs `cargo-deny` on the guard runner and is a
sequencing decision, not something to slip into this commit -- so the guard is
wired into `make deny`, where cargo-deny is already required, and the CI gap is
filed instead.
Mutation-verified: re-adding RUSTSEC-2026-0002 -> RED naming it; removed ->
GREEN. Re-verified after the bashrs refactor, since extending a guard is not
proof the old verification still holds.
Refs #2481
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-on to the dead-exemption removal in this branch. The larger finding was
that none of it gated anything:
* `cargo deny` appeared in ZERO workflows -- only `make deny`. Positive
control: 9 workflows mention `cargo`, 0 mentioned `deny`.
* The `security` job runs `cargo audit` with `continue-on-error` (ci.yml:8),
so it cannot fail the build -- and `cargo audit` does not read deny.toml.
So the advisory surface was one tool that ignores the exemption file, running in
a job that cannot fail, plus one tool that honours it running nowhere. All 29
exemptions were documentation.
Three steps added to `guard-runner-labels`, which `gate` hard-requires
(ci.yml:563 `needs: [ci, workspace-test, mutants, guard-runner-labels]`), so
these genuinely block merge:
1. install cargo-deny if absent -- free once the runner has it, self-healing
if a runner is rebuilt from a base image without it
2. `cargo deny check advisories` -- the real gate, with deny.toml honoured
3. `check_deny_exemptions_live.sh` -- kept SEPARATE on purpose: a newly-FIXED
upstream must never fail anyone's build. It fails only this guard, whose
remedy is deleting a line.
Mutation-verified: deleting a live exemption (RUSTSEC-2024-0384) -> `cargo deny
check advisories` exits 1; restored -> 0. So the exemptions are now load-bearing
rather than decorative.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Issue #2473: `ProbarDriver` had exactly one implementation in the whole workspace, `MockDriver`. `ChromiumDriver` was named in three doc comments -- one of them a `BrowserController::<ChromiumDriver>::launch(..)` example -- against a type nobody had written. So every layer built on the trait (locators, validators, playbooks, pixel coverage) drove a mock, and the Playwright-competitor framing did not survive contact with the code. The CDP machinery was already here: browser.rs::cdp launches a real chromiumoxide browser, capabilities.rs and zero_js.rs take a real chromiumoxide::Page, and chromiumoxide has been a declared dependency behind the `browser` feature all along. What was missing was the adapter between that and the trait. This is that adapter: all 16 trait methods against live Chrome. FALSIFY-PROBAR-DRIVER-001, 7 tests, each chosen so MockDriver FAILS it -- a test that merely calls the trait passes identically against a mock and proves nothing: * JS is folded by a real engine ([1..8].reduce -> 36) and the UA is Chromium, with a control proving execute_js does not return one constant for every script * a bounding box comes back as the CSS 120x40, i.e. from Blink layout, and a selector matching nothing returns None rather than a fabricated handle * typing changes the real input's .value * the screenshot carries the PNG magic and >1000 bytes from the compositor * wait_for_selector genuinely waits for an element appended at +300ms, and TIMES OUT on one that never appears * launching with a bogus executable ERRORS rather than quietly handing back something that answers questions it cannot know Mutation: make execute_js return a canned value, the way a mock would -> 6 of 7 go RED. Two defects found while proving it, both mine, both caught by that mutation rather than by review: 1. chromiumoxide points every browser at the SHARED, FIXED profile dir /tmp/chromiumoxide-runner, and Chrome's ProcessSingleton then refuses the second instance outright ("Failed to create .../SingletonLock: File exists (17) ... Aborting now to avoid profile corruption"). Two concurrent drivers could not coexist -- on one machine, or between two developers sharing a box. A browser-automation library that cannot run two browsers at once forfeits test parallelism, which is most of the point. Each driver now gets its own profile directory, removed on drop. The suite went from passing only under --test-threads=1 to passing in parallel, and got faster doing it (2.95s -> 1.03s). 2. screenshot() fell back to the CONFIGURED viewport when the page would not report its own -- echoing config back as if it were a measurement, which is the quiet degradation this file's own doc comment condemns. It was why the screenshot test survived the mutation. Now an error. The mutation then killed 6 of 7 instead of 5. navigation_timeout is honoured rather than decorative; it was an ignored config field, i.e. a promise the driver did not keep. Green: 7 driver tests against Chrome 151, 6304 lib tests, clippy clean. NOT claimed, and not yet true: * `apr probar`'s own commands do not route through this driver yet, so the CLI is not made real by this commit -- the library is. * the tests are NOT on ci.yml's beat list. They need a Chrome-equipped runner, the way the GPU falsifiers need a CUDA one. They fail rather than skip without a browser, deliberately. * #2473's other half -- 1,741 tests across 15 files wired into no `mod` -- is untouched here. No public claim should call probar a Playwright alternative until the CLI routes through this and those tests compile. Refs #2473
…part 2) #2473 reported that 15 `*_tests.rs` files in aprender-test-lib are wired into no `mod` and no `include!`, so "1,741 tests have never compiled". The wiring half of that is exactly right. The implication is not. Those files are byte-identical copies of test modules that ALREADY RUN inline in their parent files. No coverage was ever lost. Measured, not inferred. Comparing each orphan against the body of its parent's `#[cfg(test)] mod tests { .. }`: browser_tests.rs 3414 lines 0 differing locator_tests.rs 2164 lines 0 differing docker_tests.rs 1184 lines 0 differing capabilities_tests.rs 1187 lines 0 differing validators_tests.rs 2756 lines 8 differing llm/score_tests.rs 603 lines 9 differing and `#[test]` counts match exactly per file -- 274/274 browser, 216/216 locator, 91/91 docker, 100/100 capabilities. Where the copies DO differ, the orphan is strictly the OLDER text, missing clippy fixes its inline twin received: `if let` vs `match .. _ => {}`, `.keys()` vs iterating entries, `!contains_key(..)` vs `get(..).is_none()`. That is the signature of a snapshot left behind, not of tests aimed at an API that does not exist. Mounted correctly as submodules of their parents (so `use super::*` resolves as written) all 15 compile with ZERO errors and run 1544/1544 green -- with a test-NAME set identical to the inline modules, `diff` exit 0. They are duplicates, not orphans. Provenance: all 15 arrived in one commit, 8bd4ce5 (2026-05-07), a 17,830-file APR-MONO vendoring blob. They were already orphaned on arrival -- an "extract tests to separate files" refactor where the copy was made and neither the `mod` declaration nor the deletion of the inline block ever happened. So the remedy is deletion, not wiring. Wiring them in would add 1,544 duplicate test executions and zero assurance. Proof of zero coverage delta, same command either side: before test result: ok. 6390 passed; 0 failed after test result: ok. 6390 passed; 0 failed Also drops `**/browser_tests.rs` from .pmat-gates.toml's file_health exclusions -- with the file gone that pattern now matches nothing, and a dead exclusion is a rule that looks like it is protecting something. This corrects #2473's second finding. Its FIRST finding stands and is addressed separately: ProbarDriver really did have only MockDriver. Refs #2473
The layer above the driver has the same defect #2473 found below it. `ActionExecutor` is the trait playbooks execute through, and it had NO production implementation at all -- the only two `impl ActionExecutor` in the workspace are `MockExecutor`, both inside `#[cfg(test)]` modules (executor.rs:485, runner.rs:457). A playbook could be authored, parsed, validated and "run" with nothing reaching a browser. `ChromiumExecutor` implements all 12 methods over the `ChromiumDriver` from the previous commit, so `click`, `navigate`, `wait` and `screenshot` in a playbook drive Chrome. FALSIFY-PROBAR-EXEC-001, 6 tests, each written so a mock executor fails: * click on a <button> makes ITS OWN onclick write to another element -- an executor that records the click without dispatching it cannot produce that text * get_text / get_attribute read the live DOM, and a selector matching nothing is ElementNotFound rather than empty string * evaluate must be able to say NO: `1+1===3` is false. An executor hardcoded to true passes the true case and fails this one * wait observes real state -- an element appended at +300ms is found, and a condition that is never true TIMES OUT * screenshot writes a file that starts with the PNG magic Mutation: make `evaluate` return true unconditionally, the way a mock would -> evaluate_is_decided_by_the_page goes RED, the other 5 stay green. Sync-over-async is bridged with a runtime the executor owns. Calling it from inside an async context would deadlock, so `launch` REFUSES with an error naming the problem instead of panicking several frames down inside tokio; there is a test for that path. Two things fixed while proving it: * four hardcoded `Duration::from_secs(30)` waits now come from `DriverConfig::element_timeout`. That field existed and was ignored -- the same unkept-promise defect as `navigation_timeout` in the driver. It also cut the suite from 30.6s to 3.7s, since the deliberate-timeout test was waiting out a hardcoded 30s. * selectors are pasted into JS as JSON string literals, so one containing a quote cannot break out of the expression. `WaitCondition::NetworkIdle` is approximated from the page's own Resource Timing rather than CDP network events, and is commented as an approximation rather than presented as exact. Green: 6 executor tests + 7 driver tests against Chrome 151, clippy clean. Still NOT true, and still not claimed: `apr probar`'s commands do not route through this yet. `probar test` is a cargo-test wrapper (its "placeholder" comment is stale -- discover_tests really does shell out to `cargo test --list`), and `probar record` prints its configuration and records nothing. Refs #2473
This reverts bad1103, which is not a retraction: the finding and the evidence stand exactly as committed. The deletion simply belongs in its own PR rather than buried under a driver change. #2498 now carries it standalone, and a 26,205-line deletion is far easier to review on its own than mixed into ~800 lines of new driver and executor code where the diffstat hides it. The .pmat-gates.toml exclusion for browser_tests.rs comes back with the file, since it only becomes dead once the file is gone -- it moves to #2498 with the deletion it depends on. Refs #2473, #2498
…atures Found while verifying #2473: `cargo clippy -p aprender-test-lib --all-targets` passes, and the same command with `--features browser,docker,llm,proptest,derive,compute-blocks` fails with 8 errors. This is the trap already recorded in CLAUDE.md — clippy is clean on the default feature set while the crate is broken behind a non-default one — and it had gone unnoticed because CI only ever lints the default set. Confirmed pre-existing rather than assumed: the error set was captured on the base commit and again after the #2473 deletion, and the two were byte-identical. None of the sites is in a file that PR touched. FIXED tui/brick.rs:209 manual_assert if !x.is_empty() { panic!(…) } -> assert!(x.is_empty(), …) tui/compute_block.rs:171 manual_assert same llm/report.rs:96 redundant_closure .map(|r| to_markdown_row(r)) -> .map(to_markdown_row) llm/score.rs:229 manual_clamp .round().min(74.0).max(0.0) -> .round().clamp(0.0, 74.0) docker.rs:161 should_implement_trait documented #[allow] (see below) runtime.rs:928/936/1169 undocumented_unsafe_blocks real SAFETY comments The clamp rewrite is behaviour-preserving here. `min`/`max` and `clamp` differ only on NaN, and the operand is `75.0 * good / value` on a branch where `value > good > 0.0`, so it is finite by construction. `Browser::from_str` keeps its name and its `Option` return behind a documented allow rather than becoming a `FromStr` impl: an unrecognised browser name is an ordinary "not one of the three" answer, not an error worth an `Err` type, and renaming a `pub fn` would break callers. The three unsafe blocks are in tests. Each now states the caller obligation it discharges — `u32` has no invalid bit patterns for the two `read_at` calls (one of which is deliberately out of bounds and returns Err before dereferencing), and the `Box::from_raw` pointer came from `Box::into_raw` two lines up and is reconstituted exactly once. VERIFIED cargo clippy -p aprender-test-lib --all-targets --features browser,docker,llm,proptest,derive,compute-blocks -- -D warnings exit 0 (was 101) cargo clippy -p aprender-test-lib --all-targets -- -D warnings exit 0 cargo test -p aprender-test-lib --lib --features … 6458 passed, 0 failed cargo fmt --all -- --check exit 0 NOT FIXED HERE Nothing stops this recurring: no CI job lints this crate under those features. Closing that needs a decision about which crates and which feature combinations are worth the CI minutes, which is a bigger question than these 8 errors. Refs #2473 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…et that does not exist (#2504) contracts/apr-cli-commands-v1.yaml declared, five times: enforcement: "cargo test --test apr_cli_commands <fn>" There is no `apr_cli_commands` test target. The target is `cli_commands` (crates/apr-cli/tests/cli_commands.rs), so every one of those five commands exits 101 with `error: no test target named apr_cli_commands`. Two were wrong twice over, naming `test_all_commands_help` where the function is `test_all_commands_respond_to_help` (cli_commands.rs:201). The underlying tests are REAL and DO run: `cli_commands` is invoked at ci.yml:327 inside `workspace-test`, which is inside `gate.needs`. Nothing was unguarded. What was fiction is the contract's ACCOUNT of how it is enforced -- and that is not cosmetic. A reader auditing whether FALSIFY-CLI-003 is live runs the command the contract hands them, gets an error, and cannot distinguish "the pointer is stale" from "the gate is missing". The contract destroyed exactly the discrimination it exists to provide. R1 one level up: a claim verified against nothing. WHY NOTHING CAUGHT IT These strings live under a top-level `falsification:` list. The typed `Contract` struct has no such field -- it has `falsification_tests` (crates/aprender-contracts/src/schema/types.rs:33). serde drops the unknown key silently, so `pv validate` never sees them, and the sibling guard check_contract_test_binding.sh (#2465) reads `falsification_tests[].test`, a different field. The whole block is inert YAML that reads as governance. That is also why the new guard scans YAML text rather than the typed model: the field it must police is one the typed model does not admit. Teaching the schema about `falsification:` is the better long-term home and is named as follow-up in the contract's roadmap -- it is a schema change with its own blast radius, not a prerequisite for closing the hole. WHAT LANDS scripts/check_contract_enforcement.sh resolves every cargo-shaped `enforcement:` string against `cargo metadata` -- not against a guess: 1. `--test T` names a real workspace test target 2. `-p P`, when present, owns T 3. a trailing bare filter token exists as `fn F(` in T's source Rule 3 is an exact match, deliberately stricter than cargo's substring filtering: accepting prefixes would let `test_all` "resolve" against `test_all_commands_respond_to_help`, the precise near-miss that produced this bug. contracts/apr-contract-enforcement-v1.yaml metadata.kind: pattern, set explicitly so pv's `kernel` default never silently applies. Honestly reports L2 and states why L3/Kani does not apply rather than declaring an un-backed harness. Five conditions corrected, and the `scope:` prose -- which still read "77 commands" against a 105-entry registry -- now points at the list instead of carrying a hand-maintained number. FULL SWEEP All 1771 contracts, not just the reported file. 195 `enforcement:` strings exist; 13 name a cargo invocation. Of those 13: 5 broken (all in apr-cli-commands-v1.yaml), 7 `monorepo_invariants` resolve, 1 `cli_commands` resolves. Every target AND every named test function was checked. No baseline ratchet is needed -- the tree reaches zero in this PR. The other 182 strings name CI jobs, runtime assertions or build-time checks; they have no single mechanical resolver and are declared out of scope in the contract rather than silently skipped. EVIDENCE guard on unmodified main exit 1, naming exactly the 5, "5 of 13" guard after the fix exit 0, "13 cargo enforcement strings; every target and test fn resolves" the corrected command `cargo test -p apr-cli --test cli_commands test_all_commands_respond_to_help --no-run` exits 0, where the original exited 101 cargo test -p apr-cli --test cli_commands 10 passed, 0 failed bashrs lint 0 errors Self-test: 8 resolution cases (bad target / bad fn / bad package / bad PREFIX, and four that must NOT flag) plus a vacuity arm. Vacuity matters here: a regex that stops matching is indistinguishable from a clean tree, and that failure mode has already shipped twice (#2476, #2485), so the guard refuses to pass on zero and prints the count it resolved. Both guard-logic mutations verified RED, each with the correct message: neuter resolve_one() to `return 0` -> all four must-flag cases fail disable the MIN_CMDS floor -> the vacuity arm fails Wired into `guard-runner-labels` (inside gate.needs) and `make tier3`, self-test first in both. Needs `cargo metadata` only -- no build. Closes #2504 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#2506 (SURF-7/R14). `catch_unwind` and `CatchPanicLayer` appeared nowhere in aprender-serve/src, apr-cli/src or aprender-mcp/src outside a test helper and commands/qualify.rs, and tower-http's `catch-panic` feature was not enabled at all. So a panic in any axum handler unwound out of the service: the client got a transport error with no status and no body. That is also the one error shape that escaped an invariant this crate already asserts. route_surface_2376 establishes that no error leaves this server as anything but actionable JSON -- a dropped connection never becomes a response, so it slipped past by not being an error response at all. The issue said its first two falsifiers "fail on main today -- they are findings, not future tests. Confirm that before writing the fix." They do, and I did, before touching anything: a_panicking_handler_returns_json_500... FAILED the_server_still_answers_after_a_panic FAILED the_panic_probe_route_is_actually_reached ok <- the control `CatchPanicLayer` is now mounted OUTERMOST, after cors, so a panic raised in any layer below -- the JSON sanitizer, the cancel middleware, or one added later -- still becomes a response. The panic payload goes to stderr and NOT to the client: it is a Rust-internals detail, and #2376 finding 7 already bans bodies naming things a client cannot act on. ON WHAT THE TEST DOES AND DOES NOT PROVE, because the first version proved less than it looked like it did: axum's `Router::layer` wraps only the routes present when it is called. The test adds its panicking probe AFTER create_router_with_config, so the production layer does not cover it and the test must re-apply the layer. I found this by diagnosing the still-RED result rather than assuming the fix was wrong -- re-applying the layer turned all three green, which isolated ORDER as the cause and cleared the conversion fn. That means the scaffold alone would be a test of my test. Production coverage is established by mutation instead: making the REAL /health handler panic returns left: 500, right: 200 -- a response, with a status. Before the layer, the identical mutation unwound and `oneshot(..).expect(..)` fired instead. 500-instead-of-200 IS the finding: the panic became an answer. That reasoning is recorded in the test module so the next reader does not have to re-derive why the scaffold is shaped this way. Green: 15665 aprender-serve lib tests, clippy clean under --features server. Scope: the HTTP leg only. #2506's MCP leg (a tool panic must yield a JSON-RPC error frame) and its structural invariant (no Command reachable from an MCP tool constructed outside tools/subprocess.rs) are NOT done here. Refs #2503, #2506
…ract FALSIFY-README-002 caught this in CI on #2508: adding contracts/apr-contract-enforcement-v1.yaml moved the filesystem count while the README still claimed 1771. All three copies in the README are updated - the guard checks every contract-count claim precisely because the file carried three different ones (#2485). Refs #2504
… schema
CI caught what I did not run locally. Three schema defects in the contract this
PR introduced, each reported by aprender-contracts' own lint gates:
parse falsification_tests[].id is required; `name:` is deliberately NOT
aliased (types.rs:452) because legacy contracts ship name+description
side by side and aliasing both collapses to a duplicate-field error.
parse verification_summary is a typed struct (total_obligations, l2/l3/l4
counts), not prose. The narrative moved to a comment above it and the
field now carries 4 obligations, 4 at L2, 0 Kani, 4 N/A.
lint SCHEMA-001 metadata.references must not be empty; SCHEMA-008 every
falsification test must state a prediction; SCHEMA-009 each should
state if_fails.
The predictions and if_fails are real, not filler - each names the observable
outcome and the specific way the resolver would have to be broken to produce it.
The prefix case says so explicitly: if it starts passing, the fn match has
loosened to a substring, which is how `test_all` would silently resolve against
test_all_commands_respond_to_help.
L3/Kani is recorded as l4_not_applicable rather than un-proved: the subject is
filesystem and cargo-metadata resolution over unbounded strings, not a bounded
kernel. Declaring an un-backed harness to inflate the level would be theater.
cargo test -p aprender-contracts --lib 1435 passed, 0 failed (was 3 failed)
guard self-test + tree scan exit 0 / exit 0
Refs #2504
Conflict in ci.yml's guard block only. Every guard PR appends steps to that one block, so its conflicts are structural rather than semantic and they are ADDITIVE: taking either side silently drops the other branch's gates, which is precisely how a guard stops running without anyone editing it. Both sides kept, main's first. Verified by parsing the YAML and counting the steps rather than by reading the diff.
#2378 finding 8, the only unfixed P0 in the #2373 dogfood epic. The tracker describes it as "dequantizes the whole model to F32 before the parity gate can reject the path". That is imprecise, and the real defect is sharper. `try_wgpu_generate` DOES upload Q4_K projection weights as raw Q4_K bytes. It then called `dequant_model_weights(model)` for "the rest" -- which materialized an F32 `Vec` for EVERY tensor including those, and skipped only the UPLOAD: let weights = wgpu_adapter::dequant_model_weights(model)?; for (name, data, ..) in &weights { if !q4k_names.contains(name) { fwd.upload_weight(name, data); } } The allocation had already happened; the result was built and dropped. On a Q4_K model that is the bulk of the weights, and it is the remaining half of the problem batch_wgpu.rs already documents: "Why 56 GB? dequant_model_weights() called TWICE (28 GB each)" "Previous code called it twice (56 GB peak -> 28 GB peak)" Calling it once was the first half. Not dequantizing what was never going to be uploaded is this one. `dequant_model_weights_except(model, skip)` takes the set of names already sent as raw Q4_K. `dequant_model_weights` delegates to it with an empty set, so the two other call sites keep their exact behaviour. The skip is a MACRO, not a helper fn, and that is the entire mechanism: macro arguments expand INSIDE the guard, so `dequant_tensor_public(..)?` is never evaluated for a skipped tensor. A function would evaluate its arguments first and dequantize precisely what we are avoiding. ON THE FALSIFIER, because the first one I wrote was theater: It compared f32 element COUNTS between the filtered and unfiltered results. A mutation that evaluates the dequant eagerly and merely skips the push PASSED it -- the OUTPUT is identical either way. It measured the result, not the work. That is the same class this repo keeps closing, and I only found it because I ran the mutation instead of trusting the green. The replacement makes the work observable: corrupt a Q4_K tensor so dequantizing it FAILS, then assert the unfiltered call errors (the control -- proving the corruption bites) while the filtered call succeeds. Lazy passes; eager errors. Re-running the identical mutation now turns it RED with "a tensor uploaded as raw Q4_K was dequantized anyway". Host-side, so it needs NO GPU. The defect was filed as a GPU-path issue and is fully testable without one, which is why it survived: the falsifiers that would have caught it were assumed to need hardware. NOT CLAIMED: the 13.9 GB RSS and cosine-0.884 figures in #2378 are not reproduced here. This fixes a measured code path -- the dequant of skipped tensors -- and does not re-measure the reported footprint. A cross-silicon run is still owed before that number is quoted again. Green: 15664 aprender-serve lib tests, clippy clean under --features gpu. Refs #2373, #2378
Found by accident, looking for something else. Of 32 `scripts/check_*.sh`,
five were named by no workflow:
check_contract_test_binding.sh ci=0 makefile=2
check_wasm32_core_builds.sh ci=0 makefile=1
check_guards_are_wired.sh (this one, new)
check_book_examples_executable.sh ci=0 makefile=0 <- NOTHING invoked it
check_package_includes.sh ci=0 makefile=0 <- NOTHING invoked it
Makefile-only means `make tier3`, which CI does not run. The bottom two
were reachable from no automated path at all.
`check_package_includes.sh` is the sharp one. It is the CB-510 guard --
written because a `models/` pattern matched `src/models/` and hid source
from crates.io -- and its own header instructs the reader to run it after
any `.gitignore` or `Cargo.toml` exclude change. Its sibling
`check_include_files.sh` IS wired. Nothing enforced the instruction.
WIRED: contract_test_binding and wasm32_core_builds. Both pass today
(371 test references resolved; wasm32 builds), both are cheap, and there
was never a reason for them to be dark.
NOT WIRED, with reasons recorded in a shrink-only baseline rather than
argued about later:
check_book_examples_executable.sh -- RED on main, 4 failures. Two are
genuine (`apr dataset audio-inspect --help`, `apr kernel parity ...`
both error); two need a .gguf absent from this box. Minutes to run,
one subprocess per example. Wiring a known-red long job into
gate.needs helps nobody; fix the two examples first.
check_package_includes.sh -- VACUOUS on main:
OK: All 0 include!() files are included in cargo package
exit 0 having examined nothing. Wiring it now adds a gate that
measures zero, which is the defect class it exists to prevent. #2483
is the fix; wire it when that lands.
THE META-GUARD is the actual deliverable. check_guards_are_wired.sh
asserts every check_*.sh is named by at least one workflow, held at a
shrink-only baseline whose entries carry a reason. Without it the sixth
dark guard is found the way these five were.
Its own case table (--self-test) has two rows, the second being the
control: wiring the fixture guard must CLEAR the report. Without that,
row 1 passes even if the scan reported every guard it saw.
Mutation on the real tree: unwire check_wasm32_core_builds ->
"unwired guards grew 2 -> 3 ... NEW: check_wasm32_core_builds.sh", RED.
Restoring goes green.
Vacuity arm: fewer than 20 guards scanned is a hard failure, since a glob
matching nothing reports zero unwired and looks like a pass -- which is
how five went unnoticed.
Comments are excluded from the baseline count, so a reason costs nothing.
bashrs: 0 errors.
Refs #2512
#2481 F-7. `mutants` is in `gate.needs`, so it blocks -- and it could be satisfied three different ways by a run that measured nothing. All three verified by reading ci.yml at 720-750: 1. `MISSED=${MISSED:-0}; TIMEOUT=${TIMEOUT:-0}` If the greps stop matching -- a cargo-mutants JSON shape change is all it takes -- MISSED becomes 0 and the gate passes. Being unable to measure is not the same as measuring zero. Now a hard failure that prints the head of outcomes.json so the shape change is diagnosable. 2. A missing outcomes.json was an unconditional `exit 0`. The comment gives one reason the file can be absent (no mutants in the diff) and that reason is real. The other is that cargo-mutants CRASHED before writing it, and that passed identically. 3. `MUT_EXIT=$?` was captured on line 726, echoed on 727, and never tested again. 2 and 3 are the same defect: the exit status that distinguishes "clean diff" from "the run died" was already in a variable and simply unused. The missing-file branch now consults it and refuses when it is non-zero. This is the anti-theater class the repo keeps closing, in the gate whose own error message says "This would have merged SILENTLY before (PMAT gap #1)". It still would have, by a different door. Verified: ci.yml parses, `mutants` still in gate.needs, and the modified shell fragment extracted from the docker -c payload passes `bash -n`. NOT verified end-to-end: I cannot make cargo-mutants crash on demand in CI from here, so paths 2 and 3 are reasoned from the code rather than exercised. Path 1 is exercised by construction -- an unparseable file is the same code path as a shape change. Refs #2481, #2512
…es to binaries that do not exist, and a unit test that shelled out to cargo clippy Binary-surface audit, continued. 27 crates build 29 binaries; 24 of those crates are named nowhere in ci.yml, so their ~204 integration test files never run. This is what was hiding in them. DEAD BINARY REFERENCES (169 genuine) A crate's `[lib] name` is not its binary name, so CARGO_BIN_EXE_<libname> never existed and every test using it was dead: aprender-serve -> "realizar" 32 refs lib-only, no bin at all aprender-mcp -> "apr" 5 refs lib-only, no bin at all aprender-shell -> "aprender-shell" 4 refs bin deleted 2026-04-08 aprender-orchestrate -> "batuta" 1 ref aprender-test-cli -> "probador" 1 ref (aprender-profile -> "renacer", 126 refs, is #2516 on its own branch.) Per crate: * aprender-serve: DELETED tests/integration_cli.rs (423 lines, 32 tests). No `realizar` binary exists anywhere in the workspace and none was ever deleted -- the file was imported wholesale from the standalone repo during consolidation and the binary stayed behind. Not silently passing: 32/32 hard-failed. Every behaviour it claimed is already covered in-process by tests that DO run (artifact_falsification, active_pygmy_inference, cli/tests_03, ...). 5 of the 32 were tautologies that could not fail. * aprender-shell: DELETED 7 files / 88 tests. The bin was deliberately removed on 2026-04-08 in f5db50a under contracts/apr-mono-binary-rule-v1.yaml; the tests outlived it by four months because ci.yml:317 names no aprender-shell target. 11 salvaged into src/robustness_tests.rs against the public API, deliberately under --lib because that is the only aprender-shell target CI runs. * aprender-mcp: resolves the apr binary from cargo's OWN --message-format=json compiler-artifact record -- the scripts/apr_bin.sh doctrine, ask cargo rather than guess. Immune to CARGO_TARGET_DIR redirects. Asserts exactly one distinct apr executable so an ambiguous graph fails loudly instead of being decided by luck. Builds unconditionally: the "file already exists" short-circuit IS the stale-artifact hole. Note the old code was unreachable by construction: `if candidate.is_file() { candidate } else { build_apr_binary() }` -- cargo_bin PANICS rather than returning a missing path, so the repair arm could never run. THE 202-SECOND UNIT TEST bug_hunter::tests ran hunt(Path::new(".")) -- against the REAL crate, since cargo test's cwd is the manifest dir. That fans out to `cargo clippy --all-targets` (a full nested compile), `pmat query` over the whole tree, and `git blame` per source file. hunt_ensemble does it three times. One test passed /tmp, so pmat walked the entire system temp dir. test_bh_mod_001_hunt_all_modes 202.7s test_bh_mod_001_hunt_returns_result 157.3s test_bh_mod_046_..._no_pmat 118.0s Rewritten onto a fixture with one src/lib.rs and an lcov.info carrying one deterministic trigger per mode. It deliberately has NO Cargo.toml, so the nested cargo clippy finds no manifest and exits without compiling -- the fix REMOVES the nested build rather than serialising it, so these do not need nextest's serial-build group. bug_hunter: 688 tests, 202.7s -> 0.30s Why local and CI disagreed: bug_hunter caches into <project>/.pmat/bug-hunter-cache/, so a warm dev box looked fine (34s) while a fresh CI checkout paid full price every run. TWO FAILING TESTS * oracle::local_workspace::tests::test_get_git_status_current_repo asserted on the git status of whatever directory it ran in -- passes on a clean checkout, fails in a dirty one or a detached worktree. Now builds its own repo in a per-process temp dir and asserts a known state. Mutation-verified RED. * pixel_coverage::wasm_demo::tests::h0_perf_02_fill_pass_reasonable_time was a wall-clock assertion (banned here) AND it was failing at 30.8s. Both timing bounds replaced with value oracles plus non-vacuity assertions. NEW COVERAGE aprender-ptx-debug had a hand-rolled `match args[1]` parser -- the pattern banned after the identical one in simular silently dropped --seed. Converted to clap derive; 78 tests where there were none, asserting that an unknown flag, a valueless flag, and an unparseable value are all ERRORS rather than defaults, plus Cli::command().debug_assert(). Three smoke tests for previously untested binaries (presentar, train-distill, verificar), each mutation-verified RED. One mutation did NOT turn red and was diagnosed rather than shrugged at: a redundant second branch in validate_teacher also catches the empty string, so the property was re-mutated instead. VERIFICATION lib 13,757 passed 0 failed integration 45 targets 13,929 passed 0 failed (rc=0, read directly) aprender-serve cargo check --tests rc=0 clippy --all-targets (9 crates) rc=0, 0 errors cargo fmt --all --check rc=0 Cargo.lock is deliberately NOT in this commit; main's lockfile is stale and that is #2518. Findings that are recommendations, not code, are filed as #2519: three train-* binaries report confident results without doing the work (one of them published to crates.io). Refs #2503, #2519
…e, because unknown types silently defaulted to F32
`GgufToAprQ4KConverter::ggml_tensor_byte_size_h` ended in
_ => num_elements * 4, // Default to F32
Its match covered ggml types 0,1,2,3,6,7,8,12,13,14. Everything else --
Q2_K (10), Q3_K (11), Q8_K (15), every IQ type (16-23), and BF16 (30) -- took
the F32 guess.
That value is not cosmetic. It slices raw bytes out of the GGUF file:
let byte_size = Self::ggml_tensor_byte_size_h(qtype, num_elements);
let tensor_start = gguf_model.tensor_data_start + tensor_meta.offset as usize;
if tensor_start + byte_size > gguf_data.len() { /* reject */ }
A Q2_K super-block is 84 bytes per 256 elements. The fallback claims 1024 --
12.2x too many. So converting a Q2_K GGUF either fails the bounds check on a
perfectly valid file, or copies 12x past the tensor into the next one. BF16 is
2x over, and BF16 GGUFs are common.
THE CRATE ALREADY KNEW THE RIGHT NUMBERS
gguf/metadata.rs:147 Q2_K SUPER_BLOCK_BYTES = 84
gguf/metadata.rs:177 Q3_K SUPER_BLOCK_BYTES = 110
One crate, one file, two code paths, disagreeing with itself about its own
tensor sizes. The reader and the converter must not drift again, so the fix
uses the GGUF_TYPE_* constants and QK_K rather than bare numerals.
WHY NO TEST CAUGHT IT
convert/tests_byte_size.rs never calls the function:
let byte_size = num_elements.div_ceil(32) * 34;
assert_eq!(byte_size, 32 * 34);
Every test in that file re-implements the arithmetic inline and asserts an
expression against itself. They pass against any implementation, including one
that does not exist. This is the assertions-must-exclude-an-outcome class.
FIX
Added Q2_K/Q3_K/BF16 from the crate's own constants, and made an unknown type
an ERROR rather than a guess. Refusing to size a tensor we do not understand is
the honest failure; assuming F32 is exactly the silent dtype fallback that
F-DOD-005 bans.
FALSIFIER (convert/tests_byte_size.rs, calling the real function)
q2_k_is_sized_by_its_super_block_not_as_f32 336, not 4096
q3_k_is_sized_by_its_super_block_not_as_f32
bf16_is_two_bytes_per_element_not_four
an_unknown_ggml_type_is_an_error_not_a_guess <- non-vacuity
The last one is load-bearing. Without it the other three would pass even if a
permissive `_ => num_elements * 4` survived alongside the three new arms, and
the silent fallback would return for the next unlisted type. It also asserts a
KNOWN type still succeeds, so it cannot be satisfied by a function that fails
for everything.
MUTATION: restoring the silent fallback (dropping the Q2_K/Q3_K arms and BF16)
turns all 4 RED; the fix turns them green. Verified both directions.
VERIFICATION
cargo test -p aprender-serve --lib 15,666 passed 0 failed (rc=0)
cargo clippy -p aprender-serve --all-targets 6 errors, ALL pre-existing --
identical count and locations on origin/main, 0 in the changed files.
Found while triaging tests/falsification_spec_v10_tests.rs, which is named in
the Makefile and in no workflow: 140 tests, never run by CI, 38 failing.
Refs #2503
…ILE SIZE — refuse instead of fabricating
crates/aprender-train-inspect/src/inspect.rs said so itself:
// For real implementation, would parse the actual file
// Here we return simulated data based on file size
let estimated_params = estimate_params_from_size(metadata.len(), &format);
let tensors = generate_mock_tensors(estimated_params);
It synthesised a tensor list from the file's SIZE, then ran architecture
detection over the invented shapes. Reproduced independently before changing
anything -- 5 KB of /dev/urandom renamed `.safetensors`:
Format SafeTensors
Architecture llama
Hidden Dimension 768
Layers 1
Vocab Size 256
Tensors 9
rc=0
A real one-tensor safetensors file gets the SAME nine tensors, because the answer
never depended on the contents. **This crate is published to crates.io**, so that
output reached users as an "inspection".
Worth naming what it defeated: architecture.rs carries an N-05 hardening that
derives hidden-dim from tensors rather than hardcoding 4096. It derives honestly
-- from tensors fabricated one call earlier. The hardening sat one layer above
the lie.
FIX: return an error naming what it cannot do, and pointing at the tools that
actually read the file (`apr inspect`, `apr tensors`).
rc=1 "Unsupported model format: SafeTensors: `inspect` cannot parse model
files. It previously synthesised a tensor list from the file SIZE ..."
Refusing is strictly better than fabricating. Whether this binary should exist at
all is a separate question tracked in #2519; this does not prejudge it.
Also cleaned up rather than left behind:
* the two fabrication helpers are now #[cfg(test)] -- retained only for the
unit tests that assert their arithmetic, so no production path can call them
* ArchitectureDetector import dropped (nothing detects from invented shapes)
* the metadata read is KEPT as `_metadata` with a comment: it still surfaces a
real permission/IO error. Reporting the size was never the problem; inferring
architecture from it was.
* zero warnings in this crate
FALSIFIER: crates/aprender-train-inspect/tests/falsify_no_fabricated_metadata_2519.rs
garbage_bytes_are_not_reported_as_a_model
two_different_files_do_not_get_the_same_invented_answer <- the sharp one:
two DIFFERENT files of EQUAL size must not yield identical architecture
and tensor count, which is precisely what size-derived answers do
a_missing_file_fails_for_its_own_reason <- non-vacuity
MUTATION, done properly. My first attempt hand-restored the old code and did not
COMPILE (missing field `total_params`), so it proved nothing -- same standard as
any mutation that fails to turn RED. Redone with the real original from git:
git show HEAD:...inspect.rs > inspect.rs && cargo test --test falsify_...
garbage_bytes_are_not_reported_as_a_model FAILED
two_different_files_do_not_get_the_same_invented_answer FAILED
a_missing_file_fails_for_its_own_reason ok
Two RED, and the non-vacuity companion GREEN -- which is the discrimination that
matters: the tests target the fabrication, not a function that refuses
everything.
VERIFICATION
cargo test -p aprender-train-inspect 66 + 3 passed, 0 failed
cargo clippy -p aprender-train-inspect --all-targets 0 errors
cargo fmt -p aprender-train-inspect -- --check rc=0
--no-verify per #2526.
Refs #2519
…d model facts that were never measured Completes the #2519 class. The third crate, aprender-train-inspect, is the previous commit on this branch; these two are the same defect and take the same treatment: refuse, name what cannot be done, point at tools that work. Neither crate is deleted -- that remains the owner's call in #2519. === aprender-train-bench: the most serious of the three === `temperature` with NO model, NO data and NO config exited 0 and printed a full loss/accuracy table ending: Optimal: temperature = 4.00 (loss=0.6043, accuracy=80.7%) with 3.50 and 4.50 BOTH at 0.6543 -- a parabola about the vertex its own comment names ("Temperature ~4.0 is optimal"). `simulate_training` in sweep.rs said "Simulated training - in real implementation would run actual training". inspect lied about a file; this lies about WHICH HYPERPARAMETER TO USE. Anyone tuning a real distillation on that output was actively misled. Three fabricating sites, not the two in the brief: sweep.rs Sweeper::run() errors; simulate_training -> #[cfg(test)] strategies.rs compare() errors; simulate -> #[cfg(test)] cost.rs+main.rs FOUND WHILE REPRODUCING: `cost-performance` and `recommend` accepted --results and IGNORED it, substituting an 8-row literal table. `recommend --max-cost 50` printed "Top recommendation: LoRA r=32" from numbers in the source. `recommend` had no way to supply results at all. The Pareto analysis was always genuine and only lacked real input, so cost.rs gains `load_points` (JSON array of measured runs) and `recommend` gains --results. Verified BOTH directions, which matters -- a fix that only ever fails is not a fix: $ recommend --max-cost 50 rc=1, names the missing input $ recommend --max-cost 50 --results measured.json ★ only-run (Best accuracy within constraints) rc=0 i.e. it now reports the run from the FILE, not a literal. benches/sweep_benchmarks.rs timed `run().expect("sweep must succeed")` -- it benchmarked the parabola. Now times values()/to_table() on caller-supplied data. Three unit tests asserted the fabrication and were flipped, each commented: test_sweeper_finds_optimal_temperature, test_combined_is_best, and lib.rs::test_temperature_sweep_returns_results, which asserted is_ok(). Tests that assert is_ok() on input the tool cannot handle LOCK THE DEFECT IN -- the 0.63.0 audit's finding, here in the wild. === aprender-train-shell: exactly its two defects === printf 'fetch does-not-exist/totally-fake-7b\nexit\n' | aprender-train-shell ✓ Fetched does-not-exist/totally-fake-7b Parameters: 7.0B Layers: 32 Nothing was fetched, and 7.0B/32 are string-matched out of "7b" in the ID. Deliberately NOT changed: it already warned about architecture and reported `unknown` -- that part behaved, and overstating a defect is its own error. execute_fetch errors, naming `apr pull` / `apr import hf://`. detect_architecture, estimate_params, estimate_layers, ARCH_PATTERNS -> #[cfg(test)]. `-c "fetch ..."` exits 1; the interactive REPL still exits 0, because a failed command should not kill a session. === Falsifiers, and one honest weakness === train-bench tests/falsify_no_fabricated_benchmarks_2519.rs 9 tests train-shell tests/falsify_no_fabricated_fetch_2519.rs 8 tests Both written against API present in BOTH trees so the mutation COMPILES -- a mutation that fails to build proves nothing, which is exactly what happened on the inspect commit and had to be redone. Mutation via `git show HEAD:<path> > <path>`, fix restored from copies after (cmp clean, md5 match): shell 5 RED / 3 GREEN <- the proper shape. The three that stayed green are the non-vacuity anchors: fetch-without-an-id fails for its OWN reason, real commands still succeed, role flags still parse. bench 9 RED / 0 GREEN <- WEAKER, and worth stating plainly. Its two non-vacuity tests also go red because the pre-fix `recommend` had no --results flag, so clap rejects the invocation before the assertion runs. So for bench the mutation proves the tests detect the old code, but NOT that they discriminate fabrication from any-failure. The shell pair carries that property; bench's does not. Best RED evidence: the discriminating test printed left == right == [0.9064, 0.8564, 0.8064, 0.7564] for temperature 1.0-2.5 and 5.5-7.0 in mirror order -- byte-identical, proving the answer was f(|value - 4.0|). VERIFICATION (exit codes read directly, never through a pipe) cargo test -p aprender-train-bench 63 lib + 9 falsifier passed, 0 failed cargo test -p aprender-train-shell 57 lib + 8 falsifier passed, 0 failed cargo test -p aprender-train-inspect 66 lib + 3 falsifier passed, 0 failed cargo clippy (all three, --all-targets) 0 diagnostics cargo fmt --all -- --check rc=0 No new deps; no Cargo.toml/lock change; zero reverse-deps on either lib API. STILL OPEN, flagged not fixed: * These falsifiers are DARK. ci.yml runs --lib workspace-wide and names integration targets one by one at line 327; none of the three #2519 files is there. Only one PR may edit that line without a merge-queue conflict, so all three want consolidating into a single edit. * Same class, untouched in train-shell: execute_export reports "Exported to {path}" while writing nothing; execute_memory computes activations from a hardcoded 4096x32 regardless of model. Refs #2519
…e unguarded
The three fabrication fixes on this branch shipped with falsifiers that CI never
executed. `workspace-test` runs `--lib` workspace-wide; integration targets are
named one by one on a single line (ci.yml:317, which lists 16). None of the three
was there, so the tests existed and nothing ran them -- the fix was real and the
guard was theater.
Added to that chain:
cargo test -p aprender-train-inspect --test falsify_no_fabricated_metadata_2519
cargo test -p aprender-train-bench --test falsify_no_fabricated_benchmarks_2519
cargo test -p aprender-train-shell --test falsify_no_fabricated_fetch_2519
VERIFIED EACH TARGET ACTUALLY RUNS under the exact `--test` name wired, rather
than assuming the name matched the file:
falsify_no_fabricated_metadata_2519 3 passed
falsify_no_fabricated_benchmarks_2519 9 passed
falsify_no_fabricated_fetch_2519 8 passed
WIRING IS LOAD-BEARING, checked by mutation rather than by reading it. A target
name that does not exist:
$ cargo test -p aprender-train-inspect --test falsify_typo_does_not_exist
error: no test target named `falsify_typo_does_not_exist` in ...
rc=101
so a typo breaks the chain instead of silently skipping. That mattered enough to
check: this session found several guards that scanned nothing and reported PASS.
Note the line number: 317 on main, not 327 -- an earlier report of mine said 327
and both numbers have appeared in my notes. 317 is the integration chain; the
guard-runner-labels block that #2527 edits is further down, so the two touch
different hunks of the same file.
YAML validated with yaml.safe_load; 23 `--test` invocations total.
Refs #2519, #2503
…N_EXE_...") broke the CI build
The falsifier wiring from the previous commit worked: two of the three targets
ran and passed in CI (3 and 9 tests). The third failed to COMPILE:
error: environment variable `CARGO_BIN_EXE_aprender-train-shell`
not defined at compile time
--> crates/aprender-train-shell/tests/falsify_no_fabricated_fetch_2519.rs:164
let exe = env!("CARGO_BIN_EXE_aprender-train-shell");
This is the SAME CLASS I spent yesterday removing -- the 126 dead
cargo_bin("renacer") references (#2516) and the `realizar` / `aprender-shell`
ones (#2520) -- reintroduced in a brand-new test of my own. Worth stating
plainly: the class is easy to reintroduce precisely because it compiles fine
wherever the binary happens to have been built already.
WHAT I COULD AND COULD NOT ESTABLISH
The package declares `[[bin]] name = "aprender-train-shell"` with no
`required-features`, so the variable should exist. I reproduced CI's exact
command locally after touching the test file to force a rebuild:
cargo test -p aprender-train-shell --test falsify_no_fabricated_fetch_2519
test result: ok. 8 passed
It PASSES here. So my first hypothesis -- that `--test <name>` skips building
the package's bins -- is wrong, and I did not identify the real difference
(cargo version, or a fresh vs warm target dir).
Rather than keep guessing, the fix removes the dependency on compile-time
resolution entirely, which is correct regardless of the cause.
FIX: ask cargo at RUNTIME which executable it produced --
`cargo build --bin ... --message-format=json-render-diagnostics`, then take the
`executable` field. Same pattern already proven for aprender-mcp in #2520, and
the same doctrine as scripts/apr_bin.sh: never construct or assume a binary
path, ask the tool that built it.
The helper FAILS LOUDLY if cargo reports no executable. A test that silently
skipped when the binary was unavailable would be the skip-class escape this repo
bans -- and would have hidden the very defect #2519 is about. It uses a substring
match on the JSON rather than adding a serde dependency to a test crate.
VERIFICATION
cargo test -p aprender-train-shell --test falsify_no_fabricated_fetch_2519
8 passed, 0 failed (all 8, including the -c CLI surface test)
cargo clippy -p aprender-train-shell --all-targets 0 errors
cargo fmt -p aprender-train-shell -- --check rc=0
No compile-time CARGO_BIN_EXE remains in the file; the only `env!` left are a
comment and `env!("CARGO")`, which cargo always sets for tests.
Refs #2519, #2516, #2520
noahgift
enabled auto-merge
August 18, 2026 06:15
…nner host PR #2491 added an "Install cargo-deny (if absent)" step that runs cargo install into the shared ~/.cargo/bin. scripts/check_cargo_install_private_root.sh has been on main all along and rejects exactly this, so #2491 fails that guard on its own -- verified by running the guard against pr-2491 alone (rc=1, same SHARED-INSTALL finding). It went unnoticed because the intel fleet outage meant #2491 never completed a clean CI run. mac-server runs 16 runners under one $HOME, so a shared cargo install replaces a binary another running job is about to exec -- the mechanism behind aprender#2353 (cargo-llvm-cov ENOENT, empty coverage figure), and the same shared-HOME single point of failure as paiml/infra#208. Fix is the guard's own prescription rather than an allowlist entry: a per-run CARGO_INSTALL_ROOT, exported FIRST on PATH for this step and appended to GITHUB_PATH for the steps that follow. Guard: rc=1 before, rc=0 after; --self-test case table still passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sovereign-ci image bakes cargo-nextest but not cargo-mutants, so "cargo mutants" exited 101 (no such command) on every run. The pre-#2514 script treated a missing outcomes.json as "0 mutants in diff. Pass." without consulting the exit code, so the blocking mutation gate passed every PR precisely because the tool did not exist. Visible on #2533, which went green yesterday with this in its log: error: no such command: mutants cargo-mutants exit: 101 No mutants.out/outcomes.json - 0 mutants in diff. Pass. #2514 (in this batch) closed that hole, which is why the gate now fails on #2534 rather than passing: it is correctly refusing to report a result it cannot measure. The gate is right; the environment is wrong. Install cargo-mutants in the job as a stopgap, pinned to 27.1.0, and then assert it RUNS -- an install that half-succeeds must not reach the gate looking like a clean diff, which is the same failure mode #2514 closed. The tool belongs baked into the image; tracked separately against infra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…install My previous commit added "cargo install cargo-mutants" to the mutants job and tripped check_cargo_install_private_root.sh (ci.yml:815), which is the guard doing its job. The install already runs inside the ephemeral container -- CARGO_HOME is /usr/local/cargo and the container is --rm -- so it cannot reach the host toolchain that rust-cache has been deleting. But the guard reads workflow text and cannot see that, and "the guard cannot tell" is not a reason to add an allowlist entry. Declaring CARGO_INSTALL_ROOT makes the property true rather than merely argued, and puts it first on PATH so the installed binary is the one that runs. Guard: rc=1 before, rc=0 after; --self-test case table still passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ies on main cargo deny check advisories fails on origin/main (bc7120d) as well as on this branch, verified by running it against a detached checkout of main and printing the SHA at test time. So this is a newly published advisory rather than anything the batch introduced -- main is red on it independently. Two h2 versions were in the graph: h2 0.4.15 -> updated to 0.4.16, which carries the fix h2 0.3.27 -> the 0.3.x line has NO patched release The 0.3.27 path was reqwest 0.11.27 -> hyper-tls 0.5 -> hyper 0.14, and reqwest 0.11 entered only through aprender-serve: an optional dep behind the bench-http feature, and a dev-dependency. Every other crate in the workspace is already on reqwest 0.12, so these two pins were drift. Bumping them moves that path onto hyper 1.x / h2 0.4 and removes h2 0.3.27 from the graph entirely -- cargo tree -i h2@0.3.27 now reports no matching package. Preferred over a deny.toml exemption for two reasons: the vulnerable crate is actually gone rather than merely un-reported, and the ci/security job runs cargo-audit, which does not read deny.toml, so an exemption would have left that job red anyway. Verified: cargo deny check advisories rc=0; cargo check -p aprender-serve --tests rc=0 with 0 errors on default features. Note: cargo check -p aprender-serve --tests --features bench-http fails with 11 errors, but it fails identically on origin/main (GGUFConfig missing fields, absent http_client::tests::part_* modules). Pre-existing breakage behind a non-default feature, untouched here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Corrects my previous commit, which claimed the reqwest bump removed h2 0.3.27 "from the graph entirely". It removed one of two paths. The remaining one is aprender-data's OPTIONAL, non-default s3 feature: aws-sdk-s3 -> aws-smithy-http-client -> hyper 0.14 -> h2 0.3.27 That is why cargo-deny passed while cargo-audit failed on the same tree: cargo-deny walks the ACTIVATED dependency graph, cargo-audit scans Cargo.lock, and Cargo.lock lists feature-gated dependencies whether or not the feature is on. Two tools, two different questions, both correct. Removal was attempted before exemption and does not work: the 0.3 line has no patched release (upstream fixed only 0.4.16), and updating the AWS SDK leaves hyper 0.14 in place (aws-smithy-http-client 1.3.0 still pins it) while raising MSRV to 1.94.1 against a toolchain pinned at 1.93.0. That update was reverted. What WAS removed rather than exempted: h2 0.4.15 -> 0.4.16 reqwest 0.11 -> 0.12 in aprender-serve (an optional dep and a dev-dep; every other workspace crate was already on 0.12, so these were drift) Reachability measured, not assumed, per the convention in the file: cargo tree -p aprender-data -> 0 hits for h2 0.3 / hyper 0.14 cargo tree -p aprender-data --features s3 -> 4 hits cargo tree --workspace -> 0 hits The entry records the condition for its own removal. Verified: cargo audit rc=0; cargo deny check rc=0 (advisories, bans, licenses, sources all ok); private-root guard rc=0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…it dead check_deny_exemptions_live.sh failed on #2534: FAIL: these exemptions grant permission for an advisory that no longer RUSTSEC-2025-0134 Not a regression — the guard catching a real consequence of this batch. Bumping reqwest 0.11 -> 0.12 in aprender-serve dropped the last consumer of rustls-pemfile 1.x, so the advisory became unreachable and its exemption dead. The guard's own rationale is why this matters rather than being cosmetic: a dead exemption hides which of the remaining entries are load-bearing. An exemption list nobody prunes stops being a record of accepted risk and becomes noise. Deliberately NOT naming the id in the replacement comment: CI greps every RUSTSEC token out of this file, comments included, so prose mentioning an id silently re-exempts it. Same trap the h2 exemption comment had to avoid. VERIFIED check_deny_exemptions_live.sh rc=0 cargo deny check rc=0 advisories ok, bans ok, licenses ok, sources ok cargo audit rc=0
noahgift
added a commit
that referenced
this pull request
Aug 18, 2026
…eline apr_bin.sh: `cd "$here"` where $here is `git rev-parse --show-toplevel` or `pwd`, never user input -- annotate the known bashrs SEC010 false-positive per the existing bench.sh convention rather than restructure working code. check_shell_lint_ratchet.sh baseline (851) was captured against an older main; #2534's 12-PR batch added scripts and grew the true scan-everything count independent of this branch. Individually, apr_bin.sh and check_apr_bin_resolution.sh lint at 0 errors; the delta is corpus growth plus a bashrs cross-file parser-state artifact reproducible on unmodified files (bench.sh + dogfood_surfaces.sh combined also produce phantom errors that neither produces alone). Re-baselined to the honest current count (876) rather than paper over it.
noahgift
added a commit
that referenced
this pull request
Aug 18, 2026
…re-measure README ci.yml: merged the integration-chain line (ci.yml:317) across all 9 commits in this stack plus main's own growth -- 4 concurrent additions (falsify_no_fabricated_*_2519 x3 from main, beat_apr_data_alimentar_reach, beat_apr_rag_zram_reach renamed to beat_apr_sibling_cli_reach mid-stack, cargo build --examples tail from main). Dropped the stale beat_apr_rag_zram_reach reference -- that test file was renamed away by this PR's own commit a695df0. contracts/apr-cli-commands-v1.yaml, README.md: took this PR's own fix for the drifted command-count contract (no hand-maintained total, parse the commands: list), then re-measured every README claim against the fully-rebased tree rather than trust either side's pre-rebase number: 78 crates, 1772 contracts (main gained one), 111 CLI commands (unchanged -- this PR's own count already held), 113/71 book chapters. All four checked by scripts/check_readme_claims.sh, now PASS. crates/aprender-cgp/tests/falsify.rs was wired to actually run for the first time by this PR's own commit ("58 cgp tests that ran nothing") -- first run surfaced three real, pre-existing defects, not caused by the rebase: * FALSIFY-CGP-CONTRACT-002 used a relative path assuming CWD was the repo root; `cargo test -p aprender-cgp` sets CWD to the crate's own directory (verified empirically). Fixed the path. * FALSIFY-CGP-QUANT-ALL-001 always hit "No benchmark data available" because both analysis/compare.rs::run_benchmark_suite() and profilers/quant.rs::find_bench_binary() independently hardcoded /mnt/nvme-raid0/targets/trueno -- the pre-APR-MONO target dir, empty on every checkout since. Added cargo_target_dir(), asking cargo via `cargo metadata` rather than hardcoding, same doctrine as scripts/apr_bin.sh; quant.rs now reuses it instead of a third copy. * FALSIFY-CGP-062 timed a `cargo run` subprocess and asserted < 500ms, so the measurement was dominated by cargo's spawn/freshness-check cost, not the diff analysis it claimed to test. Replaced with a non-vacuity check (verdict line present) plus an assertion against the tool's own self-reported "Diff completed in Nms" line. Two more failures observed only under this session's heavy concurrent build load (falsify_cgp_scaling_002_baseline_is_1x, falsify_cgp_empirical_012_flops_sanity) passed 3/3 in isolation -- live-hardware timing assertions with no CI wiring, left as-is. --no-verify: the pre-commit complexity gate (PMAT_MAX_COGNITIVE=25) fails on this file at cyclomatic 66/cognitive 102 file-aggregate -- identical before and after this commit (verified via `pmat analyze complexity` on both trees). Pre-existing, tracked as #2526 (untracked hook freezing already-over-threshold files with no path to shrink one function at a time); not introduced or worsened here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidates 12 open PRs into one merge-queue entry. No change to any subsumed diff — this is a pure batching of already-reviewed work.
Why
workspace-testmeasured 43m48s on #2533, and the merge queue is serial (max_entries_to_build: 1). Landing these 12 individually is ~10 hours of pure queue time; the 60-minutecheck_response_timeoutcounts fromadded_to_merge_queue, so a queue this deep evicts rather than drains. Same rationale as #2449: the CI bottleneck is one workspace-test per PR, not the work.Subsumed
Closes #2486
Closes #2521
Closes #2513
Closes #2491
Closes #2483
Closes #2509
Closes #2514
Closes #2502
Closes #2508
Closes #2497
Closes #2529
Closes #2520
Why these 12 and not all 22
Measured, not guessed — every open PR was test-merged in a scratch worktree. 19 of 20 touch
.github/workflows/ci.yml, which is the single hot file behind nearly every conflict (the trap already recorded for the beat gate: only one PR may edit that line). These 12 merge cleanly together; the rest are staged as follow-ups:ci.yml.ci.yml, chore: remove whisper-apr — it is a standalone project #2515Cargo.lock, feat(cli): 61 commands across six crates had no route through apr, and simular parsed argv by hand #2493Cargo.lock+ simulate CLI).Verification
ci.ymlparses; 5 jobs; no duplicate job keys.infoinvented a model's architecture from its FILE SIZE — refuse instead of fabricating #2529's three additions (falsify_no_fabricated_{benchmarks,fetch,metadata}_2519) all survive. Nothing any PR added was dropped.cargo check --workspace— clean.If this fails CI, bisect locally against the 12 merge commits rather than re-splitting into 12 PRs.