feat(capture-attention): env-var override + privacy-harden calibration fixture (0.7.0rc4)#162
Merged
Merged
Conversation
scripts/calibrate_capture_threshold.py:79 calls vs.get(vec_id) but VecStore had no such method, blocking the Phase A soak activation workflow (calibration is the gate before flag-flip). Script was added in #153 (capture-attention Phase A) but never run end-to-end. Add get(vec_id) -> np.ndarray | None mirroring the has/delete single-id shape; returns a defensive copy matching export_all's mutation-safety contract. 3 new tests (returns vector, missing→None, defensive-copy invariant). Suite 855 → 858 passing.
Random sampling across a 2510-memory vault produces pair cosines clustered at 0.1-0.4 (clearly-different topics) — operator verdicts on those pairs carry no information about whether CAPTURE_ATTENTION_THRESHOLD should be 0.80 or 0.85. The decision region lives at cosine 0.70-0.95. Replace with near-neighbor sampling: pick a random anchor, take its top non-self neighbor via vs.search(), accept if cosine ≥ 0.55 (well below the lowest calibration threshold so edge-negatives survive). Pairs sorted by cosine descending so the operator sees high-confidence near-dupes first. Verified against /tmp/mnemon-prod-snap.sqlite (2510 live memories): 20-pair sample now spans cosine 0.751-0.999 — entirely in the calibration-relevant range, vs. 4/4 obvious-different of the prior uniform-random run.
…n fixture (0.7.0rc4) Activation infrastructure for the Phase A capture-attention soak. env-var override: - New MNEMON_CAPTURE_ATTENTION_ENABLED env var takes precedence over config.CAPTURE_ATTENTION_ENABLED. Mirrors the standing-tier pattern (MNEMON_STANDING_TIER_ENABLED) so operators can flip activation on Fly via `flyctl secrets set` without a code change + redeploy, and the next save picks it up without restarting the server. New store._capture_attention_enabled() helper called at request time from Store.save and cli attention-status. - cli attention-status now reports the EFFECTIVE flag value (env-var override applied), not the static config default — Fly secret flips show up here immediately. - 15 new tests (5 helper resolution cases x parametrize over truthy/falsy aliases + whitespace). Privacy hardening of calibration output: - tests/fixtures/capture_attention_pairs.json is now gitignored — every operator run overwrites it with real vault titles + snippets (personal context) that must not land in a public-repo commit. PR #153 originally shipped the path tracked with a placeholder schema; the placeholder moves to capture_attention_pairs.example.json so the format is still discoverable. Suite 858 → 873 passing; coverage 86.46% (gate ≥80%). Version 0.7.0rc3 → 0.7.0rc4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Activation infrastructure for the Phase A capture-attention soak, ready for operator flag-flip on Fly.
MNEMON_CAPTURE_ATTENTION_ENABLEDenv-var override mirrors the standing-tier pattern (MNEMON_STANDING_TIER_ENABLED) — operator can flip activation on Fly viaflyctl secrets setwithout a code change + redeploy. Newstore._capture_attention_enabled()helper called at request time.mnemon attention-statusreports the effective flag value (env-var applied), not the static config default.VecStore.get(vec_id) -> np.ndarray | Noneadded —scripts/calibrate_capture_threshold.py:79called the method but it didn't exist.tests/fixtures/capture_attention_pairs.jsonis now gitignored — every operator run overwrites it with real vault content. Schema template atcapture_attention_pairs.example.json.Calibration result (2026-05-24, vault snapshot 2510 live memories)
Matches existing
CAPTURE_ATTENTION_THRESHOLD = 0.85default — no config change needed. Precision plateaus above 0.85 while recall craters → diminishing returns confirm the cut.Calibration-sample precision of 0.75 is naturally below the soak target of ≥0.80 because the sampler biases to near-neighbor pairs (every pair has cos ≥ 0.55, lots of edge negatives). Production gate (N≥2 distinct sessions) is much stricter than the single-pair test; soak target is measured against real save events.
Post-merge operator workflow
mnemon upgrade web --app-name mnemon-memory --mnemon-version 0.7.0rc4flyctl secrets set MNEMON_CAPTURE_ATTENTION_ENABLED=true -a mnemon-memorymnemon attention-statusagainst live remote — should showFlag enabled : Trueboost_rate ≤ 0.25(auto) + ≥80% precision on 20-canonical manual review (operator).Test plan
pytest tests/test_vecstore.py tests/test_capture_attention.py -v— 41 passedpytest --cov— 873 passed, coverage 86.46% (gate ≥80%)/tmp/mnemon-prod-snap.sqlitegit statusshowsD tests/fixtures/capture_attention_pairs.json; local file gitignored)mnemon upgrade webafter merge + PyPI publishflyctl secrets set MNEMON_CAPTURE_ATTENTION_ENABLED=truemnemon attention-statusagainst live remote confirms flag flip