napkin-math: prior-signal ledger orchestration for extract skill (proposal 141 PR 3)

[neoneye]

Summary

Builds on PR #751 (Fork B audit) and PR #752 (dropped_signals schema + strict consumption) by closing the prior_baseline loop: the extract skill now has a way to see what the previous iteration emitted, so it can decide what to preserve and what to explain-drop.

Per review direction, the orchestration is intentionally narrow: the full prior parameters.json is not passed in. Instead, prepare_extract_input.py builds a compact Prior Signal Ledger and appends it to the combined digest at the end.

What the ledger contains (and doesn't)

Included — just enough for the LLM to recognise prior signals and reason about structural relationships:

• Signal name (entry id or output_name)
• Section (key_values, missing_values_to_estimate, ...) and kind (id or output_name)
• formula_hint when present
• depends_on when non-empty

Intentionally excluded — these would anchor the LLM on old phrasings/framings, undermining the "preservation budget, not target" posture:

• label, source_text, comment, value

Changes

1. prepare_extract_input.py gains a --prior CLI flag. When omitted (first-iteration extraction), no ledger is appended and behaviour is unchanged. When provided, build_prior_signal_ledger emits a compact markdown section appended after the bundle so the source remains authoritative.
2. Both extract prompts (from-digest and from-full) gain a Prior Signal Ledger subsection in the dropped_signals rules. Posture: ledger is advisory metadata, source remains authoritative; preserve when source-supported, record dropped_signals when not; do not invent dropped_signals for signals not in the ledger or source.
3. 12 synthetic unit tests cover ledger construction (kv ids with section/kind, output_names tracked separately, formula_hint inclusion, formula_hint omission when null, id-equals-output_name dedupe, unmodelled_gates inclusion, first-iteration empty-ledger message, explicit exclusion of label/source_text/comment/value) plus end-to-end build_combined_digest with and without --prior.

End-to-end empirical check

Ran prepare_extract_input.py --prior on paperclip's v49 parameters.json. The ledger lands at the end of the digest with all 18 prior signals — matching audit_source_preservation.build_signal_index one-for-one. The latency-tripwire trio (api_latency_p99_threshold_ms, api_latency_margin_ms, actual_api_p99_latency_ms) that v51 silently dropped per the v49→v51 audit is present in the ledger. Infrastructure is now in place for the LLM extract skill to see these prior signals and either preserve them OR record dropped_signals.

What this PR explicitly does NOT do

• Does not re-run the LLM extract skill end-to-end. That is the user's next step via the standard skill workflow — re-run extract-parameters-from-digest with the new digest, then audit_source_preservation.py against v49, then inspect whether absent_unexplained moves to explained_drop honestly (without hiding invalid drops via the PR napkin-math: dropped_signals schema + validator + audit consumption (proposal 141 PR 2) #752 strict consumption).
• Does not pass the full prior parameters.json (per review direction — anchoring risk).
• Does not change strict mode, CI gating, or Fork A scope.
• Does not bundle Phase 5 verify-bounds-citations or different-LLM validation.

Test plan

• pytest experiments/napkin_math/tests/test_prepare_extract_input_prior_ledger.py (12 pass)
• python3 experiments/napkin_math/tests/run_smoke.py (9/9 pass)
• End-to-end smoke: prepare_extract_input.py --prior produces a digest with the ledger correctly appended (verified on paperclip v49 — 18 signals, matches the audit's signal universe one-for-one)
• No label / source_text / comment / value leaks into the ledger (covered by test_ledger_does_not_include_source_text_or_label)
• CI green on latest head

🤖 Generated with Claude Code