fix(test): close remaining non-hermetic OODA decide-cap tests under SIMARD_SCALING=auto (#4571) - #4575
Conversation
…uto (#4571) The self-deploy `unit-test` gate runs the full `cargo test` suite with `SIMARD_SCALING=auto` set in the deploy environment. Four integration tests built their `OodaConfig` via `..OodaConfig::default()` while overriding `max_concurrent_actions`. Because `OodaConfig::default()` reads `SIMARD_SCALING` and, when `auto`, installs an AIMD scaler seeded from the *ambient* max, the struct-update leaked a wide scaler that `decide` honors instead of the overridden field — so the cap assertions saw more actions than configured and failed deterministically, red-caring the deploy gate for ~7 hours (target a0173ba). The gate's `truncate_output(stderr, 200)` only captured cargo's opening "Running unittests src/lib.rs" progress line, which mis-pointed diagnosis at a lib test; the real failures were in the integration suite. Fix (mirrors the issue #2732 hermetic pattern already applied to the lib `decide` tests, which this change extends to the integration tests that were missed): - decide-cap tests (`ooda.rs`, `recipe_ooda_step_parity.rs`, `ooda_daemon.rs`) pin `scaler: None` so the explicit `max_concurrent_actions` is the sole cap, independent of host env. - `adaptive_scaling::scaler_current_max_can_override_config` now installs a deterministically-pinned scaler (floor == ceiling == 2) and sets a *higher* config max, so it actually proves the scaler overrides the config value — deterministically under any system pressure. No production code changes; `decide` behavior is correct. Test-only, additive, non-breaking. Verified: all four tests pass under `SIMARD_SCALING=auto SIMARD_MAX_CONCURRENT_ACTIONS=10`; full suite shows 0 failures; fmt + clippy clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
d1c8739 to
f1475a6
Compare
…loy gate The `unit-test` deploy gate failed deterministically under full-suite load on `install::paths::tests::install_lock_is_exclusive_per_simard_home`, red-caring self-deploy (8-commit drift). The captured gate output truncated at the spinner frame "⠋ Drop t...", pointing at a Drop/teardown test. Root cause: `InstallLock` released its `LOCK_EX | LOCK_NB` advisory lock only implicitly, when the backing `File` closed. `flock` locks live on the open file description, so any sibling test that `fork`/`exec`s while the guard is held inherits the same locked description. The kernel keeps the lock until *every* inherited descriptor is closed, which outlasts the guard by the child's brief fork→exec window. Under `cargo test` CPU saturation that window overlapped the test's post-drop re-acquire, which then got EWOULDBLOCK — a deterministic failure at load, invisible in isolation. Fix: add `impl Drop for InstallLock` (unix) that issues an explicit `flock(fd, LOCK_UN)` before the file closes. `LOCK_UN` drops the single shared lock on the open file description immediately, regardless of any inherited descriptors in concurrently forked children, so a subsequent acquire never races the exec window. Additive, non-breaking; production release semantics are strictly more correct (explicit unlock, then close). Verified: `cargo test --lib` passes 0-failure across 3 consecutive full-suite runs under load; the target test previously failed deterministically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Added: fix the genuine lib-unittest failure behind the
|
📊 Coverage Summary
Coverage data from CI run. Test files matching |
Summary
Complements #4529 in fully de-flaking the self-deploy
unit-testgate, which runs the wholecargo testsuite withSIMARD_SCALING=autoset in the deploy environment (src/self_relaunch/gates.rs::run_unit_test_gate).Background / root cause
OodaConfig::default()readsSIMARD_SCALING; whenauto, it installs an AIMD scaler seeded from the ambient max. Tests that build a config via..OodaConfig::default()while overridingmax_concurrent_actionsinadvertently inherit that wide scaler, anddecidehonorsscaler.adjust()over the overridden field — so cap assertions fail deterministically. The gate'struncate_output(stderr, 200)only captured cargo's openingRunning unittests src/lib.rs ⠋line, which mis-pointed diagnosis at a lib test; the real failures were in the integration suite.#4529 already fixed
tests/ooda.rs,tests/recipe_ooda_step_parity.rs, and theadaptive_scalingtest viascaler: None. This PR (rebased on that work) closes the two remaining gaps:Changes (net vs main — 2 files)
tests/ooda_daemon.rs—run_ooda_daemon_with_session_uses_session_for_advance_goalused the same non-hermeticmax_concurrent_actions: 2, ..Default::default()pattern (untouched by fix(meeting-backend): make resolve_agent_workdir tests checkout-independent (#4505) #4529). Pinsscaler: Noneso the per-cycle cap is env-independent, preventing over-dispatch and future flakiness underSIMARD_SCALING=auto.tests/adaptive_scaling.rs—scaler_current_max_can_override_configwas fixed in fix(meeting-backend): make resolve_agent_workdir tests checkout-independent (#4505) #4529 withscaler: None, but that makes the test — named for scaler override — no longer exercise any override (no scaler present). This restores its named purpose: it now installs a deterministically-pinned scaler (floor == ceiling == 2, soadjust()is always ≤ 2 under any pressure) and sets a higher config max (8), proving the scaler overridesmax_concurrent_actions(issue Raise Simard's per-OODA-cycle goal-coverage parallelism ceiling from the current ~6 to 24, and make it env-configurable (it is an arbitrary low cap). CONTEXT: src/ooda_loop/cycle.rs:316 computes `cov #2935) — deterministically.No production code changes. Test-only, additive, non-breaking.
Verification
SIMARD_SCALING=auto SIMARD_MAX_CONCURRENT_ACTIONS=10(post-rebase).adaptive_scalingoverride test: 5/5 repeated runs green.cargo fmt --check,cargo clippy --all-targets --all-features -D warnings, and pre-push suite (478 tests, 0 failed) all green.Refs #4571, #4529
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com