Update Specs with 32 changed files (#4344) - #4352
Conversation
Automatic checkpoint to preserve work in progress. Tests and implementation saved before refactoring phase.
Replace two linear/quadratic scans in the P4 gap-scan coverage path with HashSet lookups; behavior-preserving and clarity-neutral. - detect_workstream_gaps: build a coverage HashSet once instead of an O(candidates x coverage) linear scan per goal/issue/anomaly candidate. - extract_gap_coverage_signatures: first-seen-order dedup via a borrowed HashSet, removing the O(n^2) Vec::contains scan and per-duplicate String allocation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The gap-scan had a fully-built, unit-tested READ half (extract_gap_coverage_signatures) but no WRITE half: the coverage LaunchRecipe brief never instructed the launched workstream to stamp the issue it opens, so search_issues matched zero stamps in production and the cross-restart dedup for #4340/#4341, #4337/#4338 was dormant. - decide(WorkstreamCoverage): embed one verbatim 'stewardship-signature: workstream-gap:<signature>' line per covered gap in the brief, so a daemon restart's cold gate reads them back and declines a duplicate relaunch. - sensor: add shared gap_coverage_stamp_line() formatter (single source of truth for writer+reader) and cross-reference comments at both sites so the stamp format can't drift. - test: add write<->read contract test that runs decide(), harvests the brief's own stamp lines, and asserts extract_gap_coverage_signatures recovers exactly the covered gaps' base signatures (no synthetic fixture). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rysweet
left a comment
There was a problem hiding this comment.
Comprehensive Code Review — Step 17b
Reviewed the full diff (origin/main...HEAD, 18 files, +1593/-17) covering P1 (draft-state merge gate) and P4 (cross-process gap-scan coverage dedup), plus the follow-up commit 448e8609 that closed the write half of the dedup loop.
Verdict: Approve with one non-blocking test-coverage gap. The work is high quality — additive, fail-closed/fail-safe, well-documented, and thoroughly unit-tested at the pure-function layer. CI is green except pre-commit still pending.
Strengths
- P1 draft gate (
merge_authority.rs):isDraftadded to thegh pr viewfield list,PrSnapshot.is_draftparsed with#[serde(default)](absent/malformed ⇒false, never Err/panic), and Gate 0.5 ordered correctly after the base-allowlist gate and before mergeable/CI. Six tests pin parse-true, parse-missing-defaults-false, gate refusal, gate ordering (base wins), quietRefused+ judge short-circuit, and the non-draft-still-merges regression. Solid. - P4 extractor (
sensor.rs):extract_gap_coverage_signaturesvalidates each stamp against the gap grammar viais_valid_gap_signature(forged/foreignfailure:keys dropped — advisory dedup, not a trust boundary), dedups in first-seen order, and shares onegap_coverage_stamp_lineformatter with the writer so the stamp format can't drift. Theissue:<repo>#<n>parse correctly splits on the last#. - Write↔read contract test harvests stamps straight out of
decide()(no synthetic fixture), so a writer/reader drift fails the test. Good regression design. - Perf fix (
HashSetfor O(1) coverage membership) is correct and justified. - Verified
find_existing's substring match means the"workstream-gap:"prefix query returns stamped issues in a singleghcall (no redundant RecentOpen fallback on the happy path).
Findings
1. [Medium — test gap] The production wiring glue is untested. BoardGoalCurator::open_gap_coverage_signatures / with_issue_coverage, the gh.search_issues(repo, "workstream-gap:") call, and the error→empty degradation path have zero test coverage — wiring.rs's test module never references them. The pure extractor and the detector composition (simulated_restart_*) are well-covered, but the actual seam that (a) flows search_issues results into coverage, and (b) degrades to empty-without-dropping-a-gap on a gh error, is not exercised. This is the same "wired but unverified in production" risk the philosophy review flagged, now on the integration side. Recommend a wiring.rs test with a fake GhClient that returns stamped issues (asserts the gap is suppressed) and one that returns Err (asserts degrade-to-empty, gap still surfaces once).
2. [Low] The write half is an LLM instruction, not an enforced invariant. decide embeds the stamp lines in the launch brief, but nothing guarantees the launched workstream actually copies them into the issue it opens. If the agent omits the stamp, no dedup key lands and duplicates can recur. Inherent to the recipe design and acceptable, but consider either stamping automatically in the issue-filing path or adding an observability counter for "coverage issues opened without a workstream-gap stamp" to detect non-compliance.
3. [Low] Per-tick gh dependency. survey_gaps was previously gh-free; it now issues a gh search_issues call every cadence tick. Bounded and best-effort (documented as "the ONE gh touch"), so acceptable — noting the added external dependency on the hot path.
Checklist
- Code quality and standards — clean; no
println!/print!/TODO/unwrapin production paths (all in#[cfg(test)]) - [~] Test coverage — excellent at unit layer; wiring-layer glue untested (Finding 1)
- No TODOs, stubs, or swallowed exceptions — the one degrade-to-empty path is logged via
tracing::warn! - No unimplemented functions
- Logic correctness — gate ordering, signature validation, last-
#split, and dedup all verified - Edge case handling — missing
isDraft, malformed/forged stamps, repeated signatures, restart all covered
Recommend addressing Finding 1 before merge; Findings 2–3 can be follow-ups.
Security Review — Step 17c (MANDATORY)Read-only security review of the full diff ( Verdict: ✅ No high-confidence exploitable vulnerabilities found.Checklist results
1. Injection — none. Every 2. Authorization — draft gate fails closed for the real risk. Gate 0.5 ( 3. Sensitive data — none. The new 4. Deserialization — safe. 5. ReDoS — none. Design consideration (below reporting threshold — not a blocking finding)P4 seeds its "already-covered" set from all open issues in Security review passed. No changes required to merge on security grounds. |
Philosophy Guardian Review — Step 17d (MANDATORY)Reviewed the full production diff ( Compliance checklist
Status: PASS — philosophy-compliant.Non-blocking observations (carried from Step 17b, not philosophy violations)
No new TODOs/stubs/ |
Address the Medium test-gap flagged in the PR #4352 code/security/philosophy reviews (Step 16a-16d): the wiring layer that seeds cross-process gap-scan coverage (open_gap_coverage_signatures -> GhClient::search_issues -> extract_gap_coverage_signatures -> detect_workstream_gaps) and its fail-safe degrade path had no test. Add three wiring tests against a REAL in-memory cognitive board and a recording gh fake (no network): - baseline: no gh client => the uncovered p1 goal surfaces as a genuine gap - coverage: an open workstream-gap-stamped issue suppresses the gap, and the glue queries the coverage repo exactly once with the 'workstream-gap:' namespace - degrade: a failing coverage query never drops the gap (falls back to the in-memory gate) and is attempted exactly once Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Step 17e — Address Blocking IssuesReviewed all findings from the Step 16a–16d code, security, and philosophy reviews. Blocking issues: noneAll three reviews returned non-blocking verdicts (code-review: Approve; security: PASS; philosophy: PASS). No finding blocked merge. Recurring substantive finding — now closedThe one substantive item flagged in all three reviews ("before merge") was the [Medium] untested wiring glue: Closed in
Validation
The remaining Low observations (write-half is an advisory LLM brief; per-tick |
|
Step 18b — review feedback implemented
No blocking issues remain. |
… refusal string
The reference doc's Gate 0.5 code block quoted a draft-refusal string
("PR is still a draft (isDraft == true)...") that never matched the
shipped `evaluate_objective_gates` return ("PR is still a draft and
cannot be merged. Mark it ready first: `gh pr ready <PR>`, then
retry."). An operator grepping logs for the documented text would
find nothing. Sync the snippet to the actual code string.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ready for Final ReviewWorkflow steps completed: requirements, design, implementation, tests, code review, philosophy compliance, cleanup, and quality audit. Ready for merge approval. |
The OodaConfig::default() reads process-global concurrency env vars (SIMARD_OODA_MAX_CONCURRENT / SIMARD_MAX_CONCURRENT_ACTIONS / SIMARD_SCALING). Three default-value assertions in tests_types.rs and report_tests.rs called it without the crate-wide `cognitive_memory` serial guard used by the env-mutating tests in ooda_loop::types::tests_ooda_config. Running in parallel, they could observe a leaked override (e.g. boundary_values_1_and_64 sets the var to 64), producing intermittent 'left: 64, right: 24' failures in CI. Annotate the three tests with #[serial_test::serial(cognitive_memory)] and clear the concurrency env vars at the start so they read the true compiled-in default deterministically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tes (recall precision) (#4347) The word-boundary recall gates in `LibraryCognitiveMemory` admit a query token when it is a PREFIX of a whole content word (`word.starts_with(needle)`), which preserves inflectional recall (`deploy` -> "deployed"). But a lone single-character CLEAN token — the shape a possessive ("Rust's" tokenizes to {rust, s, ...}), an initial, or a stray list separator emits — then prefix-matches EVERY content word beginning with that character, flooding the capped turn/OODA working-context recall with off-topic episodes and facts and dragging recall precision (and effective distillation fact-yield) down. The parallel knowledge-pack path (`knowledge_context::relevance_score`) already drops objective tokens shorter than `MIN_TOKEN_LEN = 2`; the adapter did not mirror that cut. Add `MIN_CLEAN_NEEDLE_LEN = 2` and drop sub-threshold CLEAN tokens at all three needle-construction sites: - `recall_episodes_ranked` (tokenized natural-language query), - `search_episodes_by_keywords` (clean keyword branch), - `partition_fact_query` (fact clean set). RAW tokens (colon/hyphen markers, hyphenated concepts) are never length-cut, so exact marker/concept lookups keep the library's verbatim substring semantics (`reflection_lessons` dedup unaffected). On the fact path a guard closes the gap where a query of only sub-threshold clean tokens would fall through to the library's raw-substring `search_facts` (a lone "s" substring-matching nearly every fact): both-empty needle sets now recall nothing. The hot production path `base_type_turn::prepare_turn_context` recalls facts by the turn objective via `search_facts`, so this directly improves the working context fed to reasoning. Tests: unit tests in `fact_query_gate_tests`; end-to-end regressions against the live `LibraryCognitiveMemory` backend in `tests_ranked_episodic`, `tests_whole_word_episode_recall`, and `tests_fact_recall_word_boundary`. QA scenario: `tests/qa-scenarios/recall-sub-threshold-needle-cut.yaml`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…) (#4348) * fix(overseer): exclude draft PRs from merge queue (#4339) Draft PRs were eligible for autonomous merge because neither ready-PR producer inspected GitHub's isDraft field. Add a fail-closed draft gate to both producers so only PRs with is_draft == Some(false) are admitted. - Parse isDraft from gh JSON into is_draft: Option<bool> in the merge_authority gh-client seam (#[serde(default, rename = "isDraft")]), so absent/malformed values map to None and are excluded (fail-closed). - Gate both producers: survey_ready_prs (merge_ops.rs) and project_ready_prs (mod.rs) drop any candidate that is not Some(false), placed as an O(1) check before the heavier objective gate. - PrSnapshot / evaluate_objective_gates left untouched; the #1880 dashboard path carries no drift. - Compiler-forced OpenPrSummary fixtures updated (is_draft: Some(false)). - Add three-way tests (draft / non-draft / unknown) across both producers. - Document the gate under docs/concepts and docs/reference. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(overseer): assert isDraft parse-boundary capture + fail-closed default (#4339) Guard the exact root cause: parse_pr_list_json must project isDraft from the gh JSON onto OpenPrSummary.is_draft (Some(true)/Some(false)), and a missing isDraft field must default to None so the draft gate excludes it fail-closed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…4349) Close parity criterion KGP-Q4 from Specs/agent-kgpacks-rs-parity.md: native_knowledge::query_articles no longer string-interpolates question keywords into its LIKE clauses. Each distinct keyword is now bound as a parameter (?n) built by the new like_contains_pattern helper, which wraps the keyword as %keyword% and escapes the keyword's own LIKE metacharacters (%, _, and the escape char) so the probe stays a literal-substring search (LIKE ?n ESCAPE '\'). The same ?n is reused by both the WHERE membership clause and the ORDER BY coverage score, so each keyword binds exactly once. Previously a question word containing % or _ silently widened the match into a wildcard scan, and quotes were hand-escaped by interpolation. Now such tokens match literally and an injection-shaped keyword is inert. Tests: like_contains_pattern_escapes_metacharacters, query_articles_treats_like_wildcards_as_literal, query_articles_binds_keywords_and_resists_injection. Plus qa-scenario tests/qa-scenarios/kgpacks-rs-query-parameterized-like-search.yaml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…4357) On the primary production topology, OODA memory is a `RemoteCognitiveMemory` IPC client whose backend is the daemon-owned `LibraryCognitiveMemory` store (`connect_memory` prefers the live socket). The client had no `recall_facts_ranked` override, so it inherited the `CognitiveMemoryOps` trait default that delegates to `search_facts`. The daemon-backed path therefore silently degraded the flagship six-signal, phase-weighted ranked recall (#2329) — and its `recall_precision_at_k` metric, both of which live only inside `LibraryCognitiveMemory::recall_facts_ranked` — to word-boundary-gated, confidence-sorted keyword search. A hollow success invisible to callers. Fix (mirrors the `list_all_episodes` #2627 additive-socket-forward): the IPC client is a transport to a library backend, so forward the library override instead of collapsing to the trait default. - add `serde` derives to `RecallWeightSet` so the weights cross the wire - add `MemoryRequest::RecallFactsRanked { query, limit, min_confidence, weights }` - `RemoteCognitiveMemory::recall_facts_ranked` sends that RPC - server dispatch routes it to the real `LibraryCognitiveMemory` ranker This restores phase-weighted ranked fact recall AND activates the `recall_precision_at_k` recall-quality metric on the production daemon path. Tests (hermetic transport round-trip: real server thread + real socket + real in-memory store): - `recall_facts_ranked_forwards_ranked_recall_over_socket`: a fact `search_facts` gates out (no shared query word) must still appear in the socket client's `recall_facts_ranked` — only true forwarding to the library ranker satisfies it - extend `every_op_encodes_backend_errors_as_rpc_call_failed` to pin the new dispatch/decode arms qa-team scenario: tests/qa-scenarios/recall-facts-ranked-socket-forward.yaml Docs: docs/reference/cognitive-memory-ranked-episodic-recall.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Updated the description of Simard to clarify its role and capabilities. Adjusted language in the operating modes section for consistency.
Updated descriptions to clarify the purpose and functionality of Simard as a continuously running engineering agent and platform.
…4361) Give the Overseer a composable deterministic rail of agentic steps on a thin tick: each due tick a reasoning recipe reads the observable state the daemon already emits (journalctl --user -u simard-ooda, simard status, simard goal list), detects crash-loops and clusters a shared failure signature across goals into a systemic-vs-per-goal root cause, and drives remediation through the EXISTING capabilities LaunchRecipe (one systemic fix) and EscalateBlockedGoal (a plain-English operator notification on both channels). Deliberately WITHOUT record_step_failure plumbing or an N-identical-failure threshold counter in Rust: the journal already contains every failure, and an agent reading it sees them all. Rail = deterministic tick + capability dispatch; brain = recipe reasoning. Mirrors the ecosystem-observe (recipe inside run_cycle -> gated interventions) and disk-health (thin marker rail) precedents. - recipe: prompt_assets/simard/recipes/overseer-health-review.yaml - prompt source: prompt_assets/simard/overseer/health_review.md (+ README row) - rail: src/overseer/health_review.rs (injectable seam, marker parser, fail-closed reviewer, production SpawnHealthReviewRecipeRunner) + 16 tests - config: SIMARD_OVERSEER_HEALTH_REVIEW opt-out (default-ON-with-Overseer) + SIMARD_OVERSEER_HEALTH_REVIEW_UNIT override + 6 tests - wiring: Overseer struct fields/builders + health_review pass in run_cycle (gate -> plan) + build_health_reviewer + 8 integration tests - docs: docs/concepts/overseer-agentic-health-review.md + index + mkdocs nav - qa: tests/gadugi/overseer-health-review.{yaml,sh} (validated + run green) Every decision flows through the SAME gate as every other Overseer action; reasoning is broad, authorization stays narrow. Fail-closed throughout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed sweep stays green (#4364) The scheduled `ci-health` sweep went red whenever a governed *sibling* repo was red and the run's token could not write it (the repo-scoped default `GITHUB_TOKEN` when `STEWARD_GH_TOKEN` is absent): `gh issue create` returned "Resource not accessible by integration", which propagated as an operational error that `--exit-zero` does not suppress. That aborted the whole filing pass (starving Simard's own issues and resolution) and turned Simard's own default-branch `ci-health` workflow into a fresh actionable failure the next sweep re-detected — the exact self-referential loop `--exit-zero` exists to prevent. Classify a cross-repo **authorization** denial as a reported `UnauthorizedSkip` instead of a fatal error: filing and resolution now continue past an unwritable repo, reconciling every writable repo (Simard's own failures included) and closing recovered issues, while the skip is printed loudly and the sibling still appears in the FleetReport (fail-safe, not fail-open). Any *other* gh/parse error still fails loud (no silent degradation). Configure `STEWARD_GH_TOKEN` to actually file/close cross-repo tracking issues instead of skipping them. - `file_issues_for_report` -> `IssueFilingReport { outcomes, skipped_unauthorized }` - `resolve_issues_for_report` -> `IssueResolutionReport { closed, skipped_unauthorized }` - CLI reports skips to stderr with a STEWARD_GH_TOKEN hint - Updated ci-health.yml comment, docs/reference/ci-health-sweep.md, --help - Unit + gadugi outside-in coverage for the resilient-skip contract Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… to #4364) (#4365) Follow-up refinements to the CI-health steward's resilient authorization-skip behavior landed in #4364, from a multi-cycle quality audit of that change: 1. Conservative authorization classification (avoid masking real failures). `is_authorization_error` no longer treats a bare `HTTP 403` / `403 Forbidden` as a permission denial. Only the explicit *permanent* permission-denial phrasings count now — `not accessible by` (integration / personal access token) and `must have admin rights`. An *unrecognized* 403 (a proxy/policy 403, or a transient secondary/abuse **rate-limit** 403 — GitHub returns those as 403 too) is no longer assumed to be a permission denial and instead fails loud, so a throttled or otherwise-degraded write is never silently downgraded to a skip that leaves a real failure untracked while the run stays green. (This subsumes the narrower rate-limit carve-out from #4364.) 2. Resolution preserves successful closes across a per-issue denial. `resolve_one_repo` now returns a `RepoResolution { outcomes, skipped }` and classifies each per-issue **close** denial *inside* the loop as a per-workflow skip (workflow: Some), so an earlier successful close in the same repo is no longer discarded when a *later* close is denied. A repo-level **list** denial remains a per-repo skip (workflow: None). Non-authorization close errors still fail loud. 3. Durable observability for permanent skips. `report_unauthorized_skips` additionally emits a GitHub Actions `::warning::` annotation per skip (when `GITHUB_ACTIONS=true`), so a permanently-unwritable governed sibling surfaces on the scheduled run's summary rather than only in a successful run's raw logs. The annotation is written to stdout (the only stream the Actions runner parses workflow commands from) and is suppressed in `--json` mode so stdout stays pure report JSON. New tests: an unrecognized bare 403 fails loud; a 403 secondary rate-limit fails loud; a per-issue close denial preserves earlier closes and is a per-workflow skip; a non-authorization close error fails loud; the annotation escaper encodes newlines/percent for a single-line warning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Re-run the code-atlas against current source for the two agentic-flow areas that drifted since the last build (commit 78218f7); all other layers are unchanged (remaining drift is new test files only). Modifications only — no new atlas files. Backend remains portable-cypher-only (lbug/native-Rust; no kuzu, no Python), analyzer mode rust-cargo-metadata/static-approximation. Overseer verify+merge sub-pipeline (src/overseer/{mod,merge_ops}.rs, src/stewardship/merge_judge.rs, src/overseer/notify.rs): - Place PR draft-exclusion narrowing (#4339, is_draft==Some(false)) in the OBSERVE stage (survey_ready_prs inline + project_ready_prs via observe_merge_queue), not as an act sub-step. - Map the act merge sub-pipeline as verify -> poll_until_green (never --admin/--no-verify) -> agentic MergeJudge -> gh pr merge --squash -> DualChannelNotifier. - MergeJudge resolution order per build_merge_judge: recipe-backed RecipeMergeJudge (merge-readiness-judge.yaml) -> direct LlmMergeJudge (merge_readiness_judge.md) -> fail-closed RefusingMergeJudge (not an unconditional recipe). - Notify: NotifyReport.dispatched() records per-channel attempts/outcomes; delivery is NOT guaranteed (all_sent() is the true-delivery check). Cognitive-memory recall path (src/cognitive_memory/library_adapter.rs): - recall_episodes_ranked recall precision gate: tokenize_words + drop sub-threshold single-char tokens (MIN_CLEAN_NEEDLE_LEN=2) + word-boundary prefix (shares_word_prefix); empty needle set => recall nothing (fail-closed). - recall_facts_ranked is a library-ranked pure read and is NOT word-boundary gated. - The CLEAN/RAW sub-threshold cut belongs to the separate search APIs (search_facts via partition_fact_query; search_episodes_by_keywords inline). Updated: agentic-flows/{README.md, agentic-memory-recall.{mmd,dot}, agentic-overseer-tick.{mmd,dot}} (+ re-rendered SVGs), cypher/atlas-agentic.cypher (merge sub-phases seq 71-75 + mem.needle-gate + conditional INVOKES), cypher/queries.cypher (Q16/Q17), index.md refresh note. Validation: mkdocs build clean; DOT+Mermaid SVGs re-rendered with new content; all cypher Phase refs defined (no dangling); every claim verified against source; rubber-duck accuracy review passed after corrections. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ite (reasoner reliability) (#4374) The shared extractor `recipe_output::extract_json_payload` strips banner / ANSI / interleaved-log noise but returns the balanced `{…}` object body VERBATIM. A trailing comma before a closing `}`/`]` — the single most common real-world LLM JSON defect (issue #2658) — therefore survives into the extracted payload and fails a strict `serde_json::from_str`. Every recipe-backed reasoner site parsed that payload strictly (`extract_json_payload(text)?` → `from_str(&payload).ok()?`), so one stray comma silently dropped the model's WHOLE structured decision and the phase fell back to its deterministic default — a parse-failure default, not a real model decision. This is a reasoner-reliability axis of the standing cognition-improvement goal. The `strip_json_trailing_commas` recovery view already existed (#2658) but was not wired into these sites. Add one shared chokepoint, `recipe_output::extract_and_parse_json<T>`, that extracts the payload, tries a strict parse, and on failure retries the parse on the trailing-comma-stripped view — but ONLY when a comma was actually removed (the `Cow::Owned` arm). The stripper is a provable no-op (`Cow::Borrowed`) on valid JSON, so any OTHER malformed shape (unquoted key, elided element, missing value) returns `None` unchanged: leniency never widens beyond the trailing-comma defect and a genuine parse error is never masked. The change is a strict superset of prior acceptance — every previously-parsing input is unchanged. Route all seven reasoner sites in `src/ooda_brain/recipe_brain.rs` through it: `parse_admission_decision`, `parse_resource_admission_decision`, `parse_idea_dedup_decision`, `parse_idea_consolidation`, `parse_outcome_decision`, `extract_decision_envelope` (decide), and `extract_orient_envelope`. Tests: 7 unit tests for `extract_and_parse_json` (clean parse; recovery before `}` and `]`; recovery through banner+ANSI+log noise; string-content comma preserved; non-comma-malformed and no-object → None) plus reasoner-site regressions in `recipe_brain::tests`. Full lib suite: 8932 passed, 0 failed. QA scenario: `tests/qa-scenarios/reasoner-trailing-comma-recovery.yaml`. Docs: `docs/reference/recipe-brain-verdict-parsing.md`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dows (#4377) The shared distillation write-boundary gate (`fact_reliability::commit_gated_fact`) deduped a new fact by scanning prior facts ONLY by `concept`: `search_facts(concept, DEDUP_PRIOR_SCAN_LIMIT=5, confidence)`, ranked confidence-descending. Because the distillation concept vocabulary is a tiny CLOSED set (only 3 `KNOWN_CONCEPTS`), many genuinely distinct facts pile up under one label. Once more than five higher-confidence facts share a concept, a real content-duplicate is crowded out of the 5-wide window and escapes dedup — the redundant fact is then promoted, inflating semantic memory and dragging recall precision + distillation fact-yield quality down as the store grows. Union the concept-keyed scan with a CONTENT-keyed scan so the exact restatement surfaces even when its concept is crowded, and add an explicit canonical-concept guard so the wider candidate set never merges identical content under a DIFFERENT concept. The union is a strict superset of the old candidate set — it can only find more duplicates, never fewer — so no prior dedup is lost. Survivors are still stored verbatim; only the dedup comparison changes. Tests (RED→GREEN): a 0.75 victim crowded behind five 0.9 same-concept facts is now deduped on re-commit (was promoted twice); a cross-concept guard test proves identical content under a different concept stays distinct. Adds a qa-scenario and updates the write-boundary-gate reference doc. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Automatic checkpoint to preserve work in progress. Tests and implementation saved before refactoring phase.
…o a single commit point Collapse the two duplicated escalation sub-cases (not-ready pre-filter and fail-closed NotMergeReady judge refusal) in the VerifyAndMergePr act arm into one outcome computation with a single merge_escalation_backoff.commit() point. Same behavior, no duplication, clearer single responsibility. All 622 overseer tests pass; fmt and clippy clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rysweet
left a comment
There was a problem hiding this comment.
Step 17b — Comprehensive Code Review
Scope reviewed: the verify-and-merge escalation idempotency change (src/overseer/guardrails.rs, src/overseer/mod.rs, docs/reference/verify-and-merge-escalation-idempotency.md) — issue #4344.
Verification I ran locally (feature worktree @ 0525196e):
cargo test --lib overseer::guardrails::tests::verify_and_merge→ 4 passedcargo test --lib overseer::tests::→ 43 passed (incl. all six new idempotency tests)- Compiles clean (
testprofile, no warnings surfaced in the run)
✅ What's correct
- Sound dedup design.
peek()ingate()runs before the cost gate (a held repeat never consumes a launch slot), andcommit()fires inact()only onActOutcome::Escalated— aMergedPR never arms the gate. This is the right invariant and is directly asserted bymerged_green_pr_merges_once_and_does_not_arm_the_escalation_gate. - Namespaced key (
verify_and_merge:{repo}#{pr}) is provably disjoint from the coverage/recall namespaces, per-PR isolated, and collision-free — covered by dedicated tests. - Guardrail preserved:
genuinely_non_green_pr_still_escalates_on_first_survey_guardrailandstuck_pr_escalation_resurfaces_after_the_window_elapsesconfirm dedup only collapses repeat pages within a bounded, self-resetting window — the operator is never permanently silenced. - Fail-closed merge authorization is untouched; the gate is advisory-only. Worst case is a bounded, self-resetting missed page — never a wrongful merge. Good.
⚠️ Findings
[Medium] "Merge exactly once" is only satisfied when verify()+merge() actually succeed — neither candidate root cause was corrected.
The stated objective is that a CLEAN+MERGEABLE+all-SUCCESS PR "merges exactly once." But pr_verify.rs is unchanged (root cause A — diff-scan false-positive) and merge_judge.rs has only a test-struct field added, not a logic change (root cause B — judge fails closed when no LLM provider is configured). The dedup guard silences the repeat page but does not make a stuck-but-green PR merge. In fact not_merge_ready_refusal_escalation_is_also_deduped codifies exactly this: a green PR refused fail-closed is deduped and left unmerged, and the operator is no longer paged after the first window. If the production incident on #4344/#4145 was a fail-closed refusal on a genuinely-green PR (not a verify/judge approval), this change makes the symptom quieter while the PR stays open forever — arguably worse than a noisy page. Please confirm the empirically-determined trigger was the verify/judge path actually approving (so the merge fires — as merged_green_pr... shows), rather than a fail-closed refusal that is now merely silenced. If it's the latter, a follow-up is needed so a genuinely-green PR is not refused solely due to a missing provider (the ambiguity-resolution's own stated requirement).
[Low] BackoffGate's internal map grows one entry per escalated (repo, pr) for the daemon's lifetime. Entries are never evicted after the PR merges/closes. It's in-memory (resets on restart) and mirrors the existing coverage_backoff precedent, so severity is genuinely low — but a stale key for a since-merged PR could suppress a later, unrelated reuse of that PR number in-process. Consider evicting on a Merged/closed observation, or capping the map, in a follow-up.
[Low / process] PR hygiene & mergeability. This PR is a 83-file, +7731/-450 cumulative branch titled "Update Specs with 32 changed files" that bundles many unrelated features (gap-scan dedup, draft-PR exclusion, ci-health sweep, cognitive-memory recall, recipe-brain verdict parsing, …). GitHub currently reports it CONFLICTING with main. A change touching merge-authorization safety rails is much safer to review and revert in isolation. Recommend rebasing to resolve conflicts and, ideally, landing the idempotency change as its own focused PR.
Checklist
- Code quality and standards — clean, well-documented, matches existing
coverage_backoffpattern - Test coverage adequate — six targeted idempotency tests, all passing
- No TODOs, stubs, or swallowed exceptions — none in the reviewed surface
- No unimplemented functions
- Logic correctness — peek/commit invariant correct and asserted
- Edge case handling — window elapse, per-PR isolation, merged-never-arms all covered
Verdict: The idempotency mechanism itself is correct, well-tested, and safe (advisory-only, fail-closed preserved). Please (1) confirm the Medium finding so we know a genuinely-green PR isn't being silently left unmerged, and (2) resolve the CONFLICTING state before merge.
Step 17c — Security ReviewScope: verify-and-merge escalation idempotency change ( Verdict: ✅ PASS — no exploitable vulnerabilities foundThe critical invariant (fail-closed merge authorization) is preserved, and the new dedup rail is fail-safe by construction: it can only suppress an operator page, never authorize a merge. Security checklist
Analysis1. Merge authorization remains fail-closed (integrity — PASS). 2. The dedup gate is fail-safe (authorization — PASS). 3. Clock-tampering resistance (PASS). 4. No injection sink (PASS). 5. No sensitive-data exposure (PASS). Informational (non-blocking, security lens)
Evidence: guardrails dedup-key tests + overseer idempotency tests pass (per Step 17b run); Conclusion: No security changes required. Safe to ship from a security standpoint. |
Step 17d — Philosophy Guardian ReviewScope: verify-and-merge escalation idempotency ( Compliance checklist
Safety invariantThe critical invariant — fail-closed merge authorization — is preserved. The dedup gate sits before the cost gate on the escalation path and is purely advisory: worst case is a bounded, self-resetting missed operator page (the window elapses and the PR re-surfaces), never a wrongful merge. Noted (non-blocking, consistent with reviewer consensus)
Verdict: PASSThe idempotency change is philosophy-compliant — minimal, well-bricked, zero-BS, and preserves the fail-closed safety contract. No blocking issues. (Process note: the PR is currently |
…ptions-max-old-space-size32768-saved-preferen # Conflicts: # docs/atlas/agentic-flows/README.md # docs/atlas/agentic-flows/agentic-memory-recall-dot.svg # docs/atlas/agentic-flows/agentic-memory-recall-mermaid.svg # docs/atlas/agentic-flows/agentic-overseer-tick-dot.svg # docs/atlas/agentic-flows/agentic-overseer-tick-mermaid.svg # docs/atlas/agentic-flows/agentic-overseer-tick.dot # docs/atlas/agentic-flows/agentic-overseer-tick.mmd # docs/atlas/cypher/atlas-agentic.cypher # docs/atlas/index.md # docs/concepts/overseer-agentic-health-review.md # prompt_assets/simard/overseer/health_review.md # src/overseer/health_review.rs # src/overseer/mod.rs # src/overseer/wiring.rs # tests/gadugi/overseer-health-review.sh # tests/gadugi/overseer-health-review.yaml
Step 17e — Blocking Issues AddressedReviewed all findings from the code, security, and philosophy reviews (Steps 16a–16d). Security and philosophy passed with only non-blocking notes. The one genuinely blocking item was the ✅ Blocking: merge conflict with
|
| Conflict | Resolution |
|---|---|
src/overseer/mod.rs, wiring.rs (2+1 hunks) |
Union — kept main's deploy_drift_observer field + wiring alongside this branch's health-review fields (both features coexist). |
src/overseer/health_review.rs (add/add) |
Took main's version — a strict superset that adds the bounded degraded-pass escalation ladder on top of this branch's review() logic. |
prompt_assets/.../health_review.md, tests/gadugi/overseer-health-review.{sh,yaml}, docs/concepts/overseer-agentic-health-review.md (add/add) |
Took main's versions so prompt + gadugi tests stay coherent with the merged code. |
docs/atlas/index.md, docs/atlas/agentic-flows/README.md |
Hand-unioned — preserved both the verify+merge narrative (this PR) and the health-review narrative (main). |
docs/atlas/*.svg/.dot/.mmd, docs/atlas/cypher/atlas-agentic.cypher |
Took main's generated diagrams (consistent with the merged health-review code; line-number drift is regenerable low-severity, already noted as acceptable). |
Verification (merged tree @ d3d7adf6): cargo check clean · cargo test overseer:: → 675 passed / 0 failed · pre-commit clippy-release passed · pre-push suite 473 passed / 0 failed + clippy --all-targets --all-features -D warnings clean. PR state: CONFLICTING → MERGEABLE.
Non-blocking findings — dispositions
- Code review Implement the first high-quality version of Simard in /home/azureuser/src/Simard based on the completed spec reviews. This is a development task and must use default-workflow. Required outcomes: (1) u #1 (Medium — "merge exactly once" root cause): Not a defect. The dedup gate is advisory-only and
commit()s only onEscalated, never onMerged. A fail-closed refusal of a green PR still escalates, and theBackoffGatere-admits (re-pages) once the backoff window elapses — a bounded, self-resetting delay, not a silent permanent un-merge. The merge-authorization path (merge()) remains authoritative and untouched. No code change required. - Code review Fix the confirmed Simard audit findings in /home/azureuser/src/Simard, using this clarified policy: explicit defaults are allowed and may be desirable for bootstrap/tests; what is forbidden is silent #2 / security INFO-1 (Low —
BackoffGatemap growth): PR-bounded, mirrors the existingcoverage_backoffdesign. Accepted under scope discipline; not blocking. - Security & Philosophy: PASS, no blocking items.
Result: No remaining blocking issues. PR is mergeable; CI (UNSTABLE) checks are in progress.
…closed refusal resurfaces Implements PR #4352 Step 16/17 review feedback (F1, F2, F4): F2 (bound map growth, raised by Code+Security+Philosophy): add `BackoffGate::forget()` (+ test-only `tracked_keys()`) and call it on a PR's terminal `Merged` outcome in the `VerifyAndMergePr` Act arm. The per-PR `state` map is now bounded to currently-open, still-escalating PRs instead of retaining one permanent entry per PR ever surveyed. Eviction is fail-safe: dropping an entry can only let the gate re-surface sooner, never suppress a real operator page. F1 (confirm dedup never silences a green-but-refused PR): add `not_merge_ready_refusal_escalation_resurfaces_after_the_window`, proving a fail-closed `NotMergeReady` refusal is held only WITHIN the bounded window and re-escalates once it elapses (never blind-merges, never deduped into permanent silence). F4 (dedup-key `#` collision): documented as accepted — impossible under `gh` repo naming; worst case is one suppressed repeat page, never a wrongful merge. No code change. Tests: guardrails `backoff_forget_evicts_a_key_and_readmits_immediately`, overseer `merging_a_previously_escalated_pr_evicts_its_backoff_entry`. All 16 guardrails + 49 overseer tests pass; clippy --lib clean. Reference doc updated (act() example, API surface, behavior matrix, guardrails, F4 note). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Step 18b — Review Feedback Implemented (commit
|
| # | Finding | Resolution |
|---|---|---|
| F1 | Confirm dedup never silences a green-but-refused PR | Confirmed + test added. The dedup rail only DELAYS a page within a bounded window — it never masks a merge decision. act() maps a fail-closed NotMergeReady to Escalated (never a blind merge), and new test not_merge_ready_refusal_escalation_resurfaces_after_the_window proves such a refusal is held only inside the window and re-escalates once it elapses. No root-cause change needed (merge() remains authoritative). |
| F2 | Unbounded BackoffGate map growth (raised by all 3 reviews) |
Fixed. Added BackoffGate::forget(); act() now evicts the per-PR entry on the terminal Merged outcome. The state map is bounded to currently-open, still-escalating PRs. Fail-safe: eviction can only re-surface sooner, never suppress a page. New tests: backoff_forget_evicts_a_key_and_readmits_immediately, merging_a_previously_escalated_pr_evicts_its_backoff_entry. |
| F3 | Merge conflict with main |
Resolved — branch merged origin/main; PR is now MERGEABLE. |
| F4 | Theoretical #-in-repo dedup-key collision |
Accepted, documented. Impossible under gh repo naming; worst case is one suppressed repeat page, never a wrongful merge. No code change. |
Validation: 16 guardrails + 49 overseer tests pass; pre-push suite (473 tests) + clippy --all-targets --all-features -D warnings clean. Reference doc updated (act() example, API surface, behavior matrix, guardrails bullet, F4 note).
The critical invariant is preserved: the dedup rail can only delay an operator page, never cause a wrongful merge — and now also keeps its state bounded.
…kdocs nav The verify-and-merge-escalation-idempotency.md reference (the core idempotency deliverable of #4344) was the only one of 7 new PR docs omitted from the mkdocs nav, leaving it orphaned from the sidebar (reachable only via inbound links). Its 6 sibling docs were all wired in. Add the nav entry beside 'Overseer Tick Details' (which links to it), satisfying the repo's no-orphaned-docs convention (verify-docs.sh T4). docs-integrity nav→file + dead-link tests stay green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📊 Coverage Summary
Coverage data from CI run. Test files matching |
Summary
Concise workflow-generated PR for Specs.
Issue
Closes #4344
Changed files
Diff stat
Behavior
Implemented behavior through these branch commits:
448e860 fix(overseer): close P4 cross-process gap-coverage dedup loop
5f205a7 perf(overseer): O(1) gap-coverage lookups in gap-scan
450aeb0 wip: checkpoint after implementation (steps 7-8)
Validation
ℹ NODE_OPTIONS=--max-old-space-size=32768 (saved preference). To change: /home/azureuser/.amplihack/config
I'll start by understanding the current branch and what changed, then run outside-in testing.
● Get branch name and changed files (shell)
│ cd /home/azureuser/src/Simard-deploy-4049/worktrees/feat/issue-4344-nodeoptions-max-old-space-si
│ ze32768-saved-preferen && echo "=== BRANCH ===" && git branch --show-current && echo "=== STATUS
│ ===" && git status --porcelain=v1 && echo "=== DIFF STAT vs merge-base ===" && git --no-pager
│ diff --stat $(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main
│ 2>/dev/null || echo HEAD~1) HEAD 2>/dev/null | tail -50
└ 24 lines…
This is a Rust project. Let me examine the core code changes to understand what behavior to test.
Risk
No high-risk subsystem pattern detected from changed paths.
Checklist
This PR was created as a draft for review before merging.
Step 16b: Outside-In Testing Results
Detected toolchain(s): Rust CLI —
Cargo.tomlat repo root (packagesimard,cargo 1.95.0/rustc 1.95.0) with a thin npm launcher wrapper (package.json). All changed source files are Rust (src/overseer/*,src/stewardship/*,src/cognitive_memory/*) plus docs; no Node/Python source logic changed. Validation therefore runs through the Rust test-binary boundary.Chosen strategy: Per the qa-team skill repo-type detection, a Rust CLI repo validates via native
cargo test(not the gadugi-agentic YAML harness). Compiled the lib test binary once (cargo test --lib --no-run) — clean build,Finished test profile in 1m 06s, 0 errors/warnings. Then drove the shipped behaviors through the public test surface (overseer merge/gap-scan consumer boundary with injected fakes — no network, no realgh).NODE_OPTIONS=--max-old-space-size=32768honored throughout.Scenario 1 (simple) — gap-scan issue dedup
cargo test --lib 'overseer::tests_gap_scan' -- --test-threads=4test result: ok. 31 passed; 0 failedsimulated_restart_stamped_issue_suppresses_duplicate_gap_launch,single_gap_orients_to_a_per_gap_keyed_coverage_problem,workstream_gap_signal_emitted_only_when_gaps_present,hostile_issue_title_yields_sanitized_signature_and_bounded_fields— repeated gap-scan passes yield at most one launch/notification per signature.Scenario 2 (edge/integration) — merge authority draft & merge-readiness gates
cargo test --lib 'stewardship::merge_authority' -- --test-threads=4test result: ok. 49 passed; 0 failedrefuses_and_does_not_merge_a_genuine_draft_pr,refuses_when_judge_says_not_ready_and_surfaces_blockers,refuses_when_mergeable_conflicting/refuses_when_mergeable_unknown,refuses_on_ci_failure/refuses_on_pending_check, plus transient-vs-deterministic retry classification (retry_succeeds_after_transient_then_ok,retry_does_not_retry_deterministic_failures).Supplementary — verify-and-merge escalation loop (core reported behavior)
cargo test --lib 'overseer::tests_selfmerge_fix'andcargo test --lib 'overseer::tests_merge_queue_reasoning'18 passed; 0 failedand38 passed; 0 failedverify_still_not_ready_on_dirty_diff,stale_disposition_produces_stale_signal) — clean/mergeable PRs are not perpetually re-escalated while genuinely non-green PRs stay gated.Fix count: 0 — all four outside-in runs passed on the first iteration; no diagnose/fix/commit cycles required.
Summary: The simple, edge/integration, and supplementary outside-in scenarios all pass cleanly against the PR branch (138 targeted tests green). Behaviors are verified from the consumer boundary: gap-scan passes dedupe to one notification per signature, and merge-authority correctly refuses drafts / non-green / conflicting PRs while allowing legitimate merges. Remaining pre-merge action is unrelated to test outcomes: the branch must be rebased onto
origin/mainto clear theCONFLICTINGstate.