feat(probe): /probe-resume — one-drop résumé sweep + corpus-match engine (#469)

[s-annam]

Summary

Adds /probe-resume — a read-only, one-drop sweep over a real résumé — plus the corpus-match engine that answers the question the six single-section probes cannot: this résumé exposes a defect; does a fixture in tests/fixtures/pdfs/ already reproduce it?

One runCascade() drives all six section localizers plus one export→re-parse hop, producing a PII-free ReproArtifact + a boolean-only DerivedSignals bag, matched against the 45 baked corpus fixtures. Each defect prints COVERED <fixture> — stop, fix the parser against the existing fixture, never open the real résumé again — or NO FIXTURE COVERS THIS, the only case that justifies minting a new one. The harness mints nothing, commits nothing, and is inert unless RL_RESUME_PDF is set, so CI never runs it.

Closes #469

What shipped

• defect-classes.ts — 21 DefectClasses derived 1:1 from the six probes' existing verdict strings, plus an oracle model: each class declares which oracles it requires (text / header / roundtrip), and spec() gates its predicate on them. See Adversarial review below — this is the part that took three rounds to get right.
• fixture-match.ts — pure, no-I/O matchCorpus(). A cover is exhibits() on the fixture's own parse, never artifact similarity; divergence only orders the output. The near-miss cap is printed (showing 3 of 41), never a silent truncation.
• localize/ — the six harnesses' localization logic extracted into pure importable functions, so the sweep reuses them instead of copy-pasting a seventh set of detectors. The six sibling probes keep byte-identical output (verified by diffing both trees).
• sweep.ts — the single place the localizers' derived signals merge and the oracle gate applies. Both the harness and the corpus bake call it, so they cannot drift.
• roundtrip-hop.ts — the one render→re-parse path, shared by the bake and the sweep. A crash anywhere in it is data, not a throw.
• Corpus snapshots → schemaVersion: 5 with baked reproArtifact + derived blocks; all 45 regenerated.
• .claude/skills/probe-resume/SKILL.md + cross-links from the six siblings.

PII

The repo is public and the input is a real résumé, so this is the load-bearing constraint:

• ReproArtifact gained no string field — repro-artifact.test.ts's PII assertion is unmodified and passing. DerivedSignals carries the same boolean-only assertion (with a smuggled-string test proving the guard has teeth).
• The 45 committed *.expected.json were audited leaf-by-leaf: every string in the new blocks is a fixed enum (v1, markdown/regex, section names, two_column). No name, email, phone, company, school, or bullet text.
• The harness console prints only counts, defect classes, axis names, fixture paths, and booleans — never a candidate's value. The JSON report goes only to internal/ (gitignored); an in-repo non-gitignored RL_RESUME_OUT is a hard error by design.
• No fixture binary was added or changed.

Test plan

• npm run verify green locally (typecheck → lint → coverage → build → fallow)
• 2226+ tests pass; harness inert (skipped) with RL_RESUME_PDF unset
• End-to-end: a fixture pointed at itself comes back COVERED by itself, including a value-level round-trip class (proves the derived half of the chain, not just the structural half)
• Golden re-bake proven mechanically per-file: only schemaVersion 4→5 and the two added blocks changed; zero pre-existing keys moved
• The six sibling probe harnesses print byte-identical output after the extraction
• verify green in CI

Provenance

Stage	Model	Effort
Implementation — defect-classes	Claude Opus 4.8	ultra
Implementation — fixture-match engine	Claude Opus 4.8	ultra
Implementation — localizer extraction	Claude Sonnet 5	high
Implementation — schemaVersion 5 bake	Claude Opus 4.8	high
Implementation — probe-resume harness	Claude Opus 4.8	ultra
Implementation — SKILL.md + cross-links	Claude Sonnet 5	medium
Adversarial review — round 1	Claude Opus 4.8	high
Review fixes — round 1	Claude Opus 4.8 (1M context)	high
Adversarial review — round 2	Claude Opus 4.8 (1M context)	high
Review fixes — round 2	Claude Opus 4.8	high
Adversarial review — round 3 (confirmation)	Claude Opus 4.8	high
Review fixes — round 3 (nits)	Claude Sonnet 5	medium
Orchestration + PR	Claude Opus 4.8 (1M context)	medium
Verification	npm run verify — green in CI	—

Adversarial review

Three rounds ran against the local diff before this PR opened (reviewer: an independent adversarial agent, prompted to break the change rather than approve it, and to reproduce suspected defects end-to-end rather than reason about them). Every blocking finding was fixed in this PR. Recording them here because two of them are the kind of bug that would have shipped as a green test.

All three blocking findings shared one root cause. A DerivedSignals bit computed from an optional cascade field reads false when the field is absent — but false there means "unknowable", not "observed absent". The predicate then silently doesn't fire, and the tool affirmatively reports no defect.

Round 1 — 2 blocking

• The headline COVERAGE n/m was routinely wrong. The three *-no-section classes were reported as defects. "This résumé has no Awards section" is not a defect — 34 of 45 fixtures have none — yet it was listed under DEFECTS FOUND, corpus-matched, and counted in the ratio. Fixed: those classes are now advisory, printed in a separate INFORMATIONAL block and excluded from the ratio.
• A false COVER — the worst possible failure here. The only bit separating skills-header-unrecognized (which no fixture reproduces) from skills-no-section (covered by 9) is derived from cascade.markdown, which is undefined on scanned/sparse PDFs. So a scanned résumé with a rejected skills header silently degraded to COVERED, telling the maintainer "stop, the corpus already has this" when nothing did.

Round 2 — 3 blocking (including two siblings of the same bug, and a flaw in round 1's own fix)

• Round 1's fix traded a false COVER for a false CLEAN. The INFORMATIONAL advisory told the reader to "check 'Sections detected'" — a line the harness never printed (the string was copied from a sibling probe that does). Reviewer minted a PDF with a real skills block under TECHNICAL PROFICIENCIES; the skills vanished from the parse and the tool printed DEFECTS FOUND (0).
• A scanned PDF reported clean. DEFECTS FOUND (0) — no defect class is exhibited by this parse, for a parse that extracted zero characters — resumelint's single most severe failure mode, reported as healthy.
• A throw in the round-trip model build escaped the hop, aborting the whole sweep and npm run bake-fixtures, despite the file's own header promising a render crash is data.

The fix generalized instead of patching. Rather than repair the three instances, the class of bug is now closed structurally: three named oracles (text, header, roundtrip), each defect class declares which it requires, and spec() gates the predicate on them — so a class cannot be added without declaring what it depends on. Blind oracles withhold their classes with a loud banner naming exactly which; an unreadable parse refuses to print a defect report at all. All 23 derived keys were then audited one by one.

Round 3 — confirmation, clean → ship

Walked all 23 keys × 21 classes (the gate is sound and not over-gated — no false non-covers), and tried five separately-minted PDFs to hole the dead-parse gate, including a prose-only PDF with 2,510 raw chars but zero extraction. No escape found. Five nits raised; all five fixed rather than deferred. Verdict: ship.

Verified and cleared across the rounds

No PII in the 45 committed snapshots (every string leaf is a fixed enum) or in the console; no silent golden regression (per-key diff of all 45 against HEAD — zero pre-existing keys moved); the six sibling probes' output is genuinely byte-identical after the extraction (both trees rebuilt and diffed); matchCorpus membership is exactly exhibits() with no distance logic in the cover decision.

Left for the human reviewer

fallow reports one complexity finding (loadCorpus, 12 cyclomatic) — it is the fail-loud snapshot-validation ladder, and fallow is report-only in CI. Dead code is 0.

[cloudflare-workers-and-pages]

Deploying resumelint with    Cloudflare Pages

Latest commit:	325b4da
Status:	✅  Deploy successful!
Preview URL:	https://e11daa69.resumelint.pages.dev
Branch Preview URL:	https://gh-469-probe-resume.resumelint.pages.dev

View logs

[s-annam]

Adversarial Review (Round 4 - Final)

I've carefully reviewed the changes and ran a full npm run verify (which is green with 2,229 passing tests). The structural fixes implemented here elegantly close the loopholes identified in the previous review rounds.

Specifically:

1. The Oracle Gate (text, header, roundtrip): The structural fix to gate exhibits() on required oracles is exceptionally sound. By preventing a blind oracle from silently returning false (which previously meant "unknowable" but acted as "observed absent"), you've eliminated the risk of a false CLEAN or false COVER. The WITHHELD explicit reporting is the right tradeoff.
2. Advisory Classes: Pulling *-no-section into an INFORMATIONAL block prevents COVERAGE n/m inflation and false COVER results.
3. Roundtrip Hop encapsulation: Moving the renderAtsResumePdf into try/catch within runRoundtripHop() ensures a crash is treated as data (roundtrip-render-crash) rather than blowing up the sweep or the corpus bake.

Regarding the decisions made without asking:

• No probe-roundtrip.test.ts: I agree with your decision. The harness is accurately represented by the RL_RT_PDF block within corpus-roundtrip.test.ts. The code should follow reality.
• Per-field contact defects: I strongly agree. A parser miss for an email is fundamentally different from a phone number (e.g., regex differences), and treating them as a single contact-field-dropped class would indeed allow a phone-dropping fixture to falsely cover an email-dropping résumé, defeating the purpose of fixture matching.

Verdict: Ship it. The defect class taxonomy is tight, the PII boundaries are strictly maintained through boolean-only DerivedSignals, and the orchestrator provides immense value for triage.

[s-annam]

To expand on the previous review, here are the answers and analysis regarding the changes and the previous LLM's adversarial review rounds:

1. Why did it become necessary to change so many files?
The high file count is almost entirely because all 45 corpus fixtures (*.expected.json) were regenerated to bump them to schemaVersion: 5. The new schema bakes the reproArtifact and derived blocks directly into each fixture snapshot. The corpus-match engine now uses these baked blocks to check if a real résumé's defect is already covered by a fixture. Crucially, as the PR body notes, zero pre-existing keys moved—the changes were purely additive to support the sweep tool.

2. Why did the other LLM struggle to fix the issues it found in Rounds 1 & 2?
The other LLM initially struggled because it attempted to patch the symptoms (a false COVER here, a false CLEAN there) rather than diagnosing the underlying structural flaw. The core problem was an epistemic one: when an optional parser input (like rawText or markdown) is completely missing (e.g., on a scanned PDF), any DerivedSignals bit computed from it defaults to false. The early rounds failed to recognize that false in this context means "unknowable/undecidable", not "observed absent". By trying to tweak individual predicates, the LLM just pushed the bubble under the rug—trading a false COVER for a false CLEAN—because the predicates were still blindly evaluating over missing data.

Analysis of the final fix and whether it was done correctly:
The final architecture the other LLM arrived at is absolutely correct. By stopping the symptomatic patching and introducing a structural Oracle Gate, the LLM fixed the entire class of bugs at the root.

Here is why the final implementation is highly robust:

• The spec() Gate: By requiring every defect class to declare its dependencies (requires: ["text", "header", "roundtrip"]), the engine now structurally prevents a predicate from firing if its oracle was blind. The sweep correctly halts and reports WITHHELD or ⛔ PARSE UNREADABLE instead of confidently outputting a false clean.
• PII-Safety by Type: The decision to keep DerivedSignals strictly boolean is brilliant. Instead of widening ReproArtifact to hold string snippets (which would inevitably leak PII into the public repository), the value-level checks are squashed down to primitive boolean facts. This guarantees PII safety by construction, rather than relying on brittle filtering logic.
• Advisory Separation: Pulling *-no-section into INFORMATIONAL shows a strong product understanding, preventing the COVERAGE ratio from being inflated by résumés that simply don't have certain sections.

In short, while the previous LLM floundered in the early rounds by applying band-aids, the final generalization it reached is the exact right design pattern for this system.

[Vaishnavi1709]

Reviewed under the resumelint pr-review gates. 0 blocking → approve.

Gates

Gate	Result
3a — fixture PII	N/A / PASS. Verified "no fixture binary was added or changed" — git diff --diff-filter=AM -- 'tests/fixtures/**' returns zero PDF/image files. Only .expected.json snapshots changed.
3b — design-system	N/A. No src/components/** changes.
3c — style tokens	N/A. Grep hits all #469 issue-refs in comments.
3d — fallow	Skipped locally. The remaining loadCorpus complexity finding (12 cyclomatic) is the intentional fail-loud snapshot-validation ladder; report-only in CI.
3e — SKILL.md as code	PASS. 7 files: probe-resume is new (196 lines); 6 siblings get only cross-link additions. probe-resume/SKILL.md has one bash command (well-formed), no gh commands, PII discipline exemplary.

Verified claims

• "Zero pre-existing keys moved" in the 44 snapshots. Confirmed: git diff -- '**/*.expected.json' \| grep '^-[^-]' \| sort -u returns exactly one line — "schemaVersion": 4,. Every other change is additive.
• RL_RESUME_OUT hard-error is real. probe-resume.test.ts:80-96 executes git check-ignore -q -- <dir> and throws with a targeted message when the override resolves inside the repo and is not gitignored.
• Oracle model is structurally sound. defect-classes.ts:436-449 — spec() is the ONE choke point wrapping every row of DEFECT_SPECS. exhibits = requires.every(oracleAvailable) && detect(...). Record<DefectClass, DefectSpec> totality guard at line 457 makes an unspec'd class a compile error. Per-class requires audit clean: skills-header-unrecognized / education-header-unrecognized require ["text","header"]; roundtrip value classes require ["text","roundtrip"]; roundtrip-render-crash requires [] with the right rationale (gating it on the roundtrip oracle would make it unreachable).
• Type-level PII discipline. defect-classes.ts:253 — DerivedSignals = { [K in DerivedSignalKey]: boolean }. Mapped-type shape means no string slot can slip in.
• Corpus loader fails loud. corpus-snapshots.ts:85-129 accumulates every degradation path (unparseable JSON, wrong schemaVersion, missing reproArtifact, missing/malformed derived, non-boolean derived key, missing sibling PDF) and throws one message. Also throws on empty corpus.
• runRoundtripHop — every layer inside try. All four layers (scoreForCascade → buildAtsResumeModel → renderAtsResumePdf → runCascade) inside one try/catch, with a layer variable that names the thrower. Matches the "crash is data" claim.
• sweepParse gate applies once. sweep.ts:135-159 — textOracleUnavailable is set AFTER the localizer spreads (so localizers can't forge it clean), then defects = DEFECT_CLASSES.filter(c => claimed.has(c) && !isWithheld(c, derived)). withheld computed separately.
• matchCorpus cover = exhibits(), not similarity. Divergence only orders the output (.sort(), never .filter()). Near-miss cap showing N of M is honest (M = ranked.length pre-slice).
• localize/skills.ts extraction is mechanical. Inline looseSkillsReason, header-candidate scan, missed-header filter, orphan-block walk all reproduced. New behavior on top: emits typed defects + derived slice, correctly withholds when markdown absent. Old verdict string unchanged.

Nits (non-blocking)

N1. Micro-perf in the withheld computation. sweep.ts:159 computes isWithheld(c, derived) per class, and defect-classes.ts:794-796 recomputes withheldOracles inside it — O(classes × oracles) = 21 × 3 per sweep. Fine at this scale; could share a computed unavailableOracles(derived) set with the defects filter on the line above. Not worth changing unless it grows.

N2. renderError message can embed a résumé char. Documented at roundtrip-hop.ts:83-88 — pdf-lib's WinAnsi cannot encode "X" etc. Not persisted (only the boolean renderThrewOnRoundtrip reaches snapshots); the RL_RT_PDF local harness prints it, nothing outward-facing. Deliberately kept.

Recommendation

APPROVE. PII discipline is at the type level (mapped boolean-only DerivedSignals), the oracle gate is a single-choke-point spec(), and the corpus loader fails loud on every degradation path — which are exactly the three places a false COVER could hide. The three adversarial rounds documented in the PR body targeted the right failure modes and the fixes (advisory separation, oracle model, ⛔ PARSE UNREADABLE gate) generalize the class rather than patching instances. The 44-file expected-JSON rebake is provably additive. No blocking or secondary findings.