napkin-math(compress): paraphrase-tolerant quote verification

[neoneye]

Summary

• Problem. quote_is_in_source was a strict substring check after normalisation. When the compress LLM paraphrased (dropped intermediate words, reordered the noun phrase), the check failed even though every content token came from the source. Because composite_score adds a +10 verified-quote bonus on a 1-25 e × r base, one missed verification can drop a load-bearing tripwire below the MAX_ITEMS_PER_BUCKET = 6 cutoff. This is the ranking-layer gap PR napkin-math(compress): second-pass shifts the variance failure from emission to ranking #743 left open: in v52 inspection, the paperclip $75,000 OPC UA bid had qv=False due to a reordered noun phrase ("lowest qualified middleware bid" vs source's "lowest qualified bid for OPC UA middleware") and lost to lower-e×r verified explicits.
• Fix. Keep the substring check as the fast path. On miss, fall back to token-overlap that requires every quote token to appear in the source token set, after a min-3-token gate. Tokenisation is Unicode word boundaries (\w+), language- and domain-neutral. The all-tokens rule subsumes any digit-bearing anchor (numbers can't be substituted) and rejects any content-word substitution ("highest" for "lowest") regardless of quote length.
• Out of scope. The emission-layer gap (LLM not producing a relevant item in either compress pass) is not addressed here; this PR only changes verification of items the LLM does emit.

Empirical posture

Same-LLM (Gemini Flash Lite) same-session re-runs, so this is a regression check on the deterministic verifier and a behavioural probe — not a same-LLM improvement claim.

• Unit tests: 28 pass (21 prior + 7 new). New cases cover substring fast-path (regression), NOT IN SOURCE and empty, paraphrase via reorder/drop (intended True), hallucinated number (intended False), short-quote substitution (intended False), short unrelated quote (intended False), and a 14-token long-quote highest/lowest substitution (intended False — locks in the all-tokens rule against any future loosening).
• Threshold-tightening cost: simulated 90% vs 100% overlap against the v53 outputs (1366 scored candidates, 8 compress runs across 6 plans): 0 items lose qv=True under tightening. The 90% rule was never functionally looser than the all-tokens rule on observed data; LLM paraphrases either reorder/elide (full token-overlap) or hit the substring fast path.
• Paperclip acceptance (3× compress):
  • v53a: $75k bid in public gates_and_thresholds top-6 at position 2, qv=True ✓
  • v53b: $75k bid in public gates_and_thresholds top-6 at position 3, qv=True ✓
  • v53c: $75k bid emitted with qv=True, but in the risks_and_shocks/premortem bucket (candidate Staging #12, e=5 r=4 stress_test, score ~30). Three other score-30 items beat it on tie-break for the last 3 public slots. Net: not in any public top-6 in v53c. This is bucket-categorisation variance (LLM placed the gate under "risk") plus tie-break, not emission failure or quote-match failure.
  • Net: 2 of 3 paperclip runs have $75k in public top-6 (vs 1 of 3 before); 3 of 3 verified-when-emitted.
• Regression sweep across 5 other probes (euro_adoption, yellowstone, crate, mars_gtld, datacenter_france): 165 fallback flips (13.7% of 1206 qv=True). Random-30 audit shows all legitimate paraphrases (dropped intermediate words, em-dash/hyphen variants, ellipsis-elided clauses) — no hallucinations, no content-word substitutions.

Test plan

• pytest worker_plan/worker_plan_internal/parameter_extraction/tests/test_compress_report_section.py (28 pass)
• Paperclip 3× run with $75k bid survival check (2/3 in public top-6 with qv=True; 1/3 emitted-and-verified-but-tie-broken in a different bucket)
• 5-other-plan regression sweep with flip audit (165 flips, 30 sampled, all legitimate)
• Threshold-tightening cost simulation (0 lost qv=True items under 90% → 100%)
• CI green (tests, lint, typecheck)

🤖 Generated with Claude Code

[neoneye]

Addressed review feedback:

1. Matcher tightened from ≥90% overlap to all-tokens-required (9521b68a). The 90% rule allowed long-quote single-word substitutions (e.g. highest for lowest over 14 tokens) to verify; all-tokens rule rejects on any missing token. Added a long-quote highest/lowest substitution test. Threshold-tightening cost simulated against the existing v53 outputs (1366 candidates): 0 items lose qv=True — the 90% rule was never functionally looser than 100% on observed data, so the tightening is empirically free.
2. Mars_gtld expert_criticism/gates_and_thresholds 0-public: investigated via a fresh rerun. LLM emits 0 candidates in both the first and second pass for this one bucket-section combo (other 4 buckets emit normally). This is Gemini Flash Lite emission variance, not anything the matcher can affect. PR notes updated to call this out as LLM run variance rather than "no regression."
3. Paperclip v53c $75k framing corrected: the bid is emitted with qv=True, but in the risks_and_shocks/premortem bucket (candidate Staging #12, score ~30). Three other score-30 items beat it on tie-break for the last 3 public slots. So it's bucket-categorisation + tie-break, not emission or quote-match. PR notes now state "2 of 3 paperclip runs have $75k in any public top-6" (vs 1 of 3 before).

PR description updated. 28 unit tests still pass. Ready for re-review.