fix(memory): word-boundary + compound-aware failure-signal detection in episode classifier by rysweet · Pull Request #4080 · rysweet/Simard

rysweet · 2026-07-15T05:52:07Z

Summary

Improves cognition (distillation fact-yield + reasoner reliability, the standing
perpetual goal's axes) by fixing a false-positive failure-signal in the
episode-ingestion classifier (src/memory_consolidation/classifier.rs, issue
#2327).

Problem

Rule 1 (the failure override) detected failure/error episodes with a naive
substring scan over {error, failed, failure, panic, exception}. Substring
matching mis-fired on look-alikes:

content	old (substring)	why wrong
"delivered exceptional results"	stored @0.9 ActionFailure	`exceptional` ⊃ `exception`
"filed the hispanic outreach note"	stored @0.9 ActionFailure	`hispanic` ⊃ `panic`
"night of terror / mirror the cluster"	stored @0.9 ActionFailure	`terror`/`mirror` ⊃ `error`

These benign/positive episodes were promoted to full importance (0.9) as
ActionFailure, polluting episodic memory and flowing into distillation as
phantom failure facts — degrading fact-yield quality and reasoner
reliability.

Fix

Two complementary word-boundary passes (no bare substring), so coincidental
substrings never fire:

lowercase stem + inflection — stems {error, fail, failure, panic, panick, exception} plus benign inflectional suffixes ("", s, es, d, ed, ing). Genuine inflections still fire (errors, failed, failing,
panicked, panicking, exceptions, failures); derivational look-alikes
(exceptional) do not.
case-sensitive compound PascalCase type names — ParseError, IoError,
NullPointerException, … whose stem sits at the end of a delimiter-less
compound the prefix rule can't see. The capitalised segment distinguishes a
real type name from the lowercase look-alike terror.

Net: keeps every genuine failure word the substring scan caught plus compound
error/exception type identifiers, while dropping the false positives.

Merge-ready evidence

1. qa-team scenario (written, validated, run)

New: tests/qa-scenarios/classifier-failure-signal-fidelity.yaml.
gadugi-test validate -f … → ✓ Scenario "classifier-failure-signal-fidelity" is valid (1 valid, 0 invalid).
gadugi-test run -s classifier-failure-signal-fidelity → ✓ Passed: 1 / Failed: 0 / All tests passed! (drives cargo test --locked --lib memory_consolidation::classifier and the full memory_consolidation:: suite).

2. Docs updated (user-facing surface)

docs/reference/episode-ingestion-classifier.md — Rule 1 rewritten to describe word-boundary + inflection + compound-type-name matching.
docs/architecture/episode-ingestion-policy.md — Rule 1 updated to match.
Inline module docs on classifier.rs updated.

3. Quality audit (SEEK→VALIDATE→FIX cycles, ended clean)

Cycle 1 (independent code-review): flagged a Medium regression — compound
type identifiers (ParseError, NullPointerException) missed by the initial
prefix-only matcher. Fixed by adding the case-sensitive compound-type-name
pass + tests.
Cycle 2 (re-review of the fix): verified compound matching works and
lowercase look-alikes (terror) stay excluded; call sites pass original-case
content; content_lc not dead; no new false positives; tokenization edge
cases (empty tokens, unicode, plural stripping) safe. No issues.
Cycle 3 (final): fmt/clippy/tests clean. Zero critical/high; zero medium
correctness/security findings remaining.

4. CI-equivalent gates green (0 failures)

Pre-commit: cargo fmt --all -- --check ✓, cargo clippy --release --no-deps -- -D warnings ✓.
Pre-push (CI race-subset + full clippy): cargo test --release --lib -- cognitive_memory bootstrap memory_ipc memory_consolidation ✓, cargo clippy --all-targets --all-features --locked -- -D warnings ✓ → all push-stage gates passed.
Targeted: 18 classifier unit tests pass (5 new); 135 memory_consolidation:: tests pass, no regressions.

6. Focused diff

4 files: classifier.rs (fix), classifier_tests.rs (tests), 2 docs, 1 scenario. No unrelated edits.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…in episode classifier The episode-ingestion classifier (issue #2327) detected failure/error episodes with a naive substring scan over {error, failed, failure, panic, exception}. Substring matching mis-fired on look-alikes — "exceptional" (⊃ "exception"), "hispanic" (⊃ "panic"), "terror"/"mirror" (⊃ "error") — storing those benign or POSITIVE episodes at full importance (0.9) as ActionFailures. That phantom- failure noise then flowed into distillation, degrading fact-yield quality and reasoner reliability (the standing cognition-improvement goal's axes). Replace the substring scan with word-boundary matching in two complementary passes, both anchored at word boundaries so coincidental substrings never fire: 1. lowercase stem + inflection — the {error, fail, failure, panic, panick, exception} stems plus benign inflectional suffixes ("", s, es, d, ed, ing), so genuine inflections (errors, failed, failing, panicked, panicking, exceptions, failures) still fire while derivational look-alikes (exceptional) do not; and 2. case-sensitive compound PascalCase type names (ParseError, IoError, NullPointerException) — idiomatic error/exception type identifiers whose stem sits at the END of a delimiter-less compound the prefix rule cannot see. The capitalised segment distinguishes a real type name from the lowercase look-alike "terror". Net effect: keeps every genuine failure word the substring scan caught (plus compound type names) while dropping the false positives that polluted episodic memory and distillation. - classifier.rs: FAILURE_STEMS / INFLECTIONAL_SUFFIXES / COMPOUND_FAILURE_SUFFIXES, word_is_failure, word_is_compound_failure_typename, rewritten has_failure_signal (now takes original-case content); Rule 1 and sanitize_transcript pass through. - classifier_tests.rs: false-positive kills, inflection preservation, compound type-name coverage, and a case-sensitivity guard for the compound pass. - docs: episode-ingestion-classifier.md + episode-ingestion-policy.md Rule 1. - tests/qa-scenarios/classifier-failure-signal-fidelity.yaml: outside-in gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-07-15T06:00:02Z

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module	Lines	Covered	Coverage
Total	173460	143284	82.6%

_{Coverage data from CI run. Test files matching tests?/ are excluded from line counts.}

…#4080) The daemon derived the overseer meta-thread's `last_success` from `!panicked && report.errors == 0`, so a single transient per-intervention `act()` failure — documented as "isolated, never fatal" — pinned the meta-thread in "erroring", and a later healthy tick never cleared it. Add an additive `cycle_failed: bool` to `OverseerTickReport` that is set ONLY when the tick's OODA cycle itself fails (`run_cycle()` returns Err or the tick panics). Health is now derived from `!panicked && !cycle_failed`, so isolated act() errors are still counted in `errors` for feed visibility but no longer drive health. Because health is recomputed fresh each tick, the first clean tick flips the row from "erroring" back to "ok". - additive, `#[serde(default)]` — legacy feed records default to false, no SCHEMA_VERSION bump - structured tracing + daemon log now emit `cycle_failed` - regression tests cover erroring→ok recovery, run_cycle Err, panic, and legacy deserialization Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…#4080) (#4214) The daemon derived the overseer meta-thread's `last_success` from `!panicked && report.errors == 0`, so a single transient per-intervention `act()` failure — documented as "isolated, never fatal" — pinned the meta-thread in "erroring", and a later healthy tick never cleared it. Add an additive `cycle_failed: bool` to `OverseerTickReport` that is set ONLY when the tick's OODA cycle itself fails (`run_cycle()` returns Err or the tick panics). Health is now derived from `!panicked && !cycle_failed`, so isolated act() errors are still counted in `errors` for feed visibility but no longer drive health. Because health is recomputed fresh each tick, the first clean tick flips the row from "erroring" back to "ok". - additive, `#[serde(default)]` — legacy feed records default to false, no SCHEMA_VERSION bump - structured tracing + daemon log now emit `cycle_failed` - regression tests cover erroring→ok recovery, run_cycle Err, panic, and legacy deserialization Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ures (#893) #4080 stopped an isolated per-intervention act() error from pinning the `overseer` meta-thread in "erroring", but a genuine `run_cycle()` failure whose cause is transient and external (an upstream 5xx, a timeout, a connection reset, a rate-limit) still latched "erroring" until a later clean tick — degrading the outer OODA stewardship loop even though nothing is actually wrong with Simard. This adds a bounded self-healing rung, purely additive to the post-#4080 baseline: - `wiring::is_transient` — a pure, fail-closed classifier over `OverseerError`. Only a `Capability` error whose detail names a known-retryable upstream fault is transient; every unknown detail and every non-`Capability` variant is fatal (SR-1). - `OverseerTickReport::transient_cycle_failure` — additive `#[serde(default)]` flag set only when a `cycle_failed` tick's error was transient (never on a panic). Legacy feed records deserialize to the conservative `false`; SCHEMA_VERSION is unchanged. - `OverseerThreadStatus::overseer_meta` — extended to consume the report plus a running consecutive-transient count and a configured ceiling: a transient failure within the ceiling maps to a self-clearing "backoff" (one cadence, no erroring counter); beyond the ceiling it escalates to "erroring" so a hard-down dependency cannot hide behind an infinite backoff (SR-2); fatal and panicked failures still map straight to "erroring". - `config::overseer_transient_backoff_ceiling{,_from}` — fail-safe env-injectable ceiling resolver (`SIMARD_OVERSEER_TRANSIENT_BACKOFF_CEILING`, default 3, always >= 1). - daemon loop owns the consecutive-transient counter across ticks (reset on a completed tick, incremented on a transient failure, unchanged on a fatal one). Docs (self-healing reference, activity-feed reference, index) move from "design — not yet implemented" to shipped. Structured tracing + the tick log carry the new `transient` signal; no println!/print!. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…lity / fact-yield) (#4245) The episode-ingestion classifier (`memory_consolidation::classifier::classify`) decides, at every `store_episode` intake site, whether an event is operational noise (DROP), low-value bookkeeping (DOWN-SCOPE), or a meaningful episodic worth full-importance storage (STORE). Distillation later mines the STORED, high-importance episodes into semantic facts, so a misclassification here propagates straight into recall top-k. Rule 3's durable-completion signal scored a single token with a raw-substring `content_lc.contains("merged")`. That fired inside the git-status vocabulary `unmerged` / `submerged` — which name the OPPOSITE of a completed merge (an outstanding, un-completed one) — promoting benign/negative bookkeeping to a durable `ActionCompleted` episode at 0.7 importance. Distillation would then mine that phantom completion into a phantom "action completed" fact, polluting recall top-k and dragging recall precision down. The `merged` signal now matches at WORD BOUNDARIES via a new `contains_word` helper, aligning this seam with the word-boundary policy already adopted by the same classifier's failure-signal pass (#4080) and the knowledge-pack relevance scorer (#4241). `unmerged`/`submerged` down-scope to the operational tier; a genuine whole-word `merged` still classifies as a durable completion, so no real completion changes disposition. The multi-word phrases (`opened pr`, `pull request`) cannot embed in a single word, so they remain substring checks. Internal surface only — `classify` signature/contract unchanged, no CLI/operator surface touched. New unit tests pin both the negative (`unmerged`/`submerged`) and positive (whole-word `merged`) contract; a qa-team gadugi scenario drives them outside-in. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ures (#893) (#4236) #4080 stopped an isolated per-intervention act() error from pinning the `overseer` meta-thread in "erroring", but a genuine `run_cycle()` failure whose cause is transient and external (an upstream 5xx, a timeout, a connection reset, a rate-limit) still latched "erroring" until a later clean tick — degrading the outer OODA stewardship loop even though nothing is actually wrong with Simard. This adds a bounded self-healing rung, purely additive to the post-#4080 baseline: - `wiring::is_transient` — a pure, fail-closed classifier over `OverseerError`. Only a `Capability` error whose detail names a known-retryable upstream fault is transient; every unknown detail and every non-`Capability` variant is fatal (SR-1). - `OverseerTickReport::transient_cycle_failure` — additive `#[serde(default)]` flag set only when a `cycle_failed` tick's error was transient (never on a panic). Legacy feed records deserialize to the conservative `false`; SCHEMA_VERSION is unchanged. - `OverseerThreadStatus::overseer_meta` — extended to consume the report plus a running consecutive-transient count and a configured ceiling: a transient failure within the ceiling maps to a self-clearing "backoff" (one cadence, no erroring counter); beyond the ceiling it escalates to "erroring" so a hard-down dependency cannot hide behind an infinite backoff (SR-2); fatal and panicked failures still map straight to "erroring". - `config::overseer_transient_backoff_ceiling{,_from}` — fail-safe env-injectable ceiling resolver (`SIMARD_OVERSEER_TRANSIENT_BACKOFF_CEILING`, default 3, always >= 1). - daemon loop owns the consecutive-transient counter across ticks (reset on a completed tick, incremented on a transient failure, unchanged on a fatal one). Docs (self-healing reference, activity-feed reference, index) move from "design — not yet implemented" to shipped. Structured tracing + the tick log carry the new `transient` signal; no println!/print!. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

rysweet merged commit ab658e7 into main Jul 16, 2026
16 checks passed

rysweet deleted the engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784091717-49fc35 branch July 16, 2026 18:11

rysweet mentioned this pull request Jul 16, 2026

fix(overseer): decouple meta-thread health from isolated act() errors (#4080) #4214

Merged

This was referenced Jul 17, 2026

fix(overseer): self-heal the meta-thread through transient cycle failures (#893) #4236

Merged

fix(cognition): whole-word merged in episode classifier (recall quality / distillation fact-yield) #4245

Merged

rysweet mentioned this pull request Jul 17, 2026

fix(cognition): whole-word signal-keyword matching in research idea extraction (fact-yield) #4260

Merged

Navigation Menu

fix(memory): word-boundary + compound-aware failure-signal detection in episode classifier - #4080

fix(memory): word-boundary + compound-aware failure-signal detection in episode classifier#4080
rysweet merged 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1784091717-49fc35

rysweet commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

rysweet commented Jul 15, 2026

Summary

Problem

Fix

Merge-ready evidence

1. qa-team scenario (written, validated, run)

2. Docs updated (user-facing surface)

3. Quality audit (SEEK→VALIDATE→FIX cycles, ended clean)

4. CI-equivalent gates green (0 failures)

6. Focused diff

Uh oh!

github-actions Bot commented Jul 15, 2026

📊 Coverage Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant