feat(parser): capture DOCX headers/footers into structured config + raw sidecar

[thewrz]

Why

Ingress needs to learn real firm/client header/footer layouts from uploaded DOCX files. Recognizable content should become editable structured config; unsupported OOXML should be retained in a raw sidecar with warnings, not silently discarded.

Design: docs/superpowers/specs/2026-06-26-header-footer-fidelity-design.md
ADR: docs/adr/039-header-footer-fidelity.md, docs/adr/068-header-footer-capture-mapping.md

What

Parses word/header*.xml / word/footer*.xml, their relationships, and section-level references (default/first/even, first-page and even/odd settings) into the #208 composition model:

• Recognized literals, known Word fields (PAGE, NUMPAGES, DATE, etc.), simple rule lines, and common section-number/title fields map to field references, not copied text.
• Unsupported image/table/raw OOXML content is preserved in a raw.unmodeled sidecar rather than discarded, each entry tagged with a kind (including a new inactiveVariant case for an unpromoted first/even reference whose content is still captured) and paired with a header-footer-content-skipped parse warning.
• SpecTree.headerFooter round-trips through the parse-worker boundary and is documented in openapi.yaml.
• New DOCX_HEADER_FOOTER_XML_INVALID error code for genuinely malformed header/footer XML — capture itself never throws for document-content reasons (see design decision 8).

New modules: header-footer-parts.ts (JSZip glob-read I/O), header-footer-relationships.ts, header-footer-field-recognition.ts, header-footer-region.ts, header-footer.ts (orchestrator). Prerequisite extractions to stay under the 400-line file cap: core-metadata.ts, source-detection.ts, and an ast/schemas.ts → spec-tree-schemas.ts split.

Design decisions

Corrections made to the pre-spike design based on concrete spike evidence (each independently reproduced, not taken on the spike's word alone — re-confirmed via wc -l/grep at the top of this pass):

1. index.ts line budget: the original design assumed ONE extraction (detectSource/detectArticleIlvl) bought enough headroom. Spike proved this false — index.ts lands at 448/400 lines after that single extraction plus the header-footer wiring. Added a SECOND prerequisite extraction (core-metadata.ts) AND moved the JSZip glob-read I/O itself into a dedicated header-footer-parts.ts helper rather than inlining it in extractEntries, both to satisfy the file-level 400-line cap and the function-level 50-line cap (extractEntries alone hit 52 lines as originally sketched). Chose extraction over trimming comments/whitespace since code.md requires "extract first," not squeeze.

2. WARNING_SUGGESTIONS exhaustive map (src/parser/text/index.ts) was entirely absent from the original design's todos despite being a hard TypeScript compile error the moment the new ParseWarningType literal is added. Added as its own line item rather than folding into the schema task, since it's a distinct file/module.

3. header-footer-schemas.ts (feat(ast): header/footer variants + page-number policy + raw sidecar #302's shipped file) — the original design explicitly forbade touching it. Spike hit a real TS2375/TS2322 type error proving that constraint unworkable: a plain detail: unknown interface isn't structurally assignable into z.json()'s inferred type even via an open .catchall(). Resolved by adding a small, additive, non-breaking field (HeaderFooterUnmodeledEntrySchema + unmodeled on the raw sidecar) rather than working around it with unsafe casts (as unknown as, forbidden by this repo's TS strictness) or abandoning content-preservation (which would violate acceptance criterion 3). Verified no circular-import risk: header-footer-schemas.ts already owns its own local JsonValue const, independent of the ast/schemas.ts / spec-tree-schemas.ts split.

4. resolveReferenceTargets's Map-keyed-by-target-path return shape was replaced with an array-of-resolved-pairs shape, closing a real (if narrow) correctness gap the pre-spike design didn't consider: two different reference slots resolving to the same physical part would silently collide in a Map. An array with a linear (variant, region) filter is simpler and removes the invariant dependency on target-path uniqueness entirely, at negligible cost given at most ~6 references per document.

5. Added a 6th unmodeled 'kind': 'inactiveVariant', because the pre-spike design's own prose already required this exact case (an unpromoted first reference still gets its content captured into raw.unmodeled + warning) but its enum had nowhere honest to put it — 'unresolvedReference' would misstate that the reference didn't resolve, when it did (it's the toggle that's off). This is a precision fix, not a scope change.

6. w:tbl detection: corrected from "anywhere in the paragraph" (pre-spike design's imprecise phrasing) to "the part's root-level direct children" — an OOXML structural fact (tables cannot nest inside w:p) the spike verified by direct implementation, not merely inferred. Captured as a positive fixed fact for the real build and for ADR-068, not left as a probable bug for a future PR to discover.

7. Internal complexity/line-cap decomposition (collapseComplexFields, buildRegionFromParagraph/captureRegion helpers, parseSectionHeaderFooterInfo, captureHeaderFooter's variant logic) is now named explicitly in the interfaces section as private helper functions each public function must be decomposed into, rather than left as an implicit "the implementer will figure it out" gap — the spike quantified exactly which functions and by how much they exceeded eslint's complexity 10 / sonarjs/cognitive-complexity 10 / max-lines-per-function 50 caps, so the real build budgets for it from the start instead of discovering it at pnpm lint time.

8. .parse()-failure error-attribution ambiguity (spike finding feat(db): migration runner and core schema #7, a real risk the spike flagged but did not resolve) is now resolved explicitly: guarantee JSON-safety at every unmodeled-detail construction site via the existing compact() helper (already used elsewhere in the module) BEFORE building the composition object, so the final HeaderFooterCompositionSchema.parse() call is expected to always succeed on any real document input; if it nonetheless throws, that is treated as an uncaught internal defect (a bug the test suite should have caught), never remapped to the document-blaming DOCX_HEADER_FOOTER_XML_INVALID code. This keeps captureHeaderFooter's "total, never throws (for document-content reasons)" contract honest rather than silently laundering a capture-code bug as a source-document problem.

9. Everything the spike explicitly confirmed as correct in the original design (407/392-line prerequisite facts, single openapi.yaml enum site, schemas.ts split boundary cohesion, RunProperties field mapping, HeaderFooterRegionSchema's single-paragraph ceiling, async Promise.all JSZip glob-read pattern, zero MCP contract-map.ts changes, zero regression across the 55 pre-existing index.test.ts tests) is carried forward unchanged — re-litigating settled, verified facts wastes the next agent's budget.

Testing

• Unit tests pass (pnpm test — 159 files / 2143 tests)
• Integration tests pass (pnpm test:integration — 130 files / 1321 tests, 10 pre-existing skips)
• pnpm build (tsc) clean
• pnpm lint (eslint + tsc --noEmit + prettier --check) clean
• Manual verification: upload a DOCX fixture with default/first/even headers and footers and confirm the resulting SpecTree.headerFooter composition and raw.unmodeled sidecar
• CI green

🤖 Co-authored by Claude Sonnet 5. Closes #306

Summary by CodeRabbit

• New Features
  • DOCX parsing now captures header and footer content (default, first-page, and even-page variants) when present.
  • Parse output can include optional header/footer composition data (e.g., modeled fields, styles, borders, and tab-separated layout).
• Bug Fixes
  • Introduced a dedicated parse warning when header/footer content is skipped (header-footer-content-skipped) and preserved unmodeled or inactive content for review instead of dropping it.
  • Added clearer handling for malformed DOCX header/footer XML with a specific parser error code.
• Documentation
  • Added an ADR detailing the header/footer capture mapping behavior and limitations.

[coderabbitai]

📝 Walkthrough

Walkthrough

DOCX parsing now captures header/footer content into structured composition data, preserves unsupported content in a raw sidecar, emits parse warnings, and exposes the result through validated AST and worker outputs.

Changes

Header/footer capture

Layer / File(s)	Summary
Contracts and capture rules docs/adr/..., openapi.yaml, src/ast/*	Adds ADR-068 rules, raw unmodeled-entry schemas, SpecTree.headerFooter, and the new parse-warning contract.
DOCX inputs and metadata src/parser/docx/core-metadata.ts, src/parser/docx/header-footer-*.ts, src/parser/docx/types.ts	Reads metadata, relationships, settings, section references, and header/footer XML parts.
Field and region modeling src/parser/docx/header-footer-field-recognition.ts, src/parser/docx/header-footer-region.ts	Recognizes fields, maps styles, splits paragraph cells, and preserves unsupported content.
Composition orchestration src/parser/docx/header-footer.ts	Builds variants, preserves unresolved/inactive/duplicate content, validates compositions, and aggregates warnings.
Pipeline wiring and validation src/parser/docx/index.ts, src/lib/parse-worker.ts, src/parser/docx/*test.ts, src/api/*, src/mcp/*	Connects capture to parsing and worker boundaries and adds schema, integration, corpus, contract, and line-budget tests.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

• #306 — Directly implements the DOCX header/footer structured capture and raw-sidecar objectives.
• #301 — Parent issue for the DOCX header/footer ingress work.
• #484 — Concerns the region capture and rule-line behavior implemented here.

Possibly related PRs

• wrzonance/SpecR#314 — Extends the same header/footer AST sidecar schema.
• wrzonance/SpecR#21 — Shares the DOCX parser entrypoint and pipeline wiring.
• wrzonance/SpecR#420 — Shares parse-warning enum and AST contract changes.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 47.71% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes satisfy #306 by parsing H/F parts, relationships, section refs, first/even settings, structured mapping, raw preservation, warnings, and v2 output.
Out of Scope Changes check	✅ Passed	The added docs, schema updates, tests, and supporting parser modules all align with the header/footer capture feature.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and accurately summarizes the main change: DOCX header/footer capture into structured config with a raw sidecar.

---

_{Comment @coderabbitai help to get the list of available commands.}

[thewrz]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

[coderabbitai]

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/mcp/contract.integration.test.ts`:
- Around line 219-220: Remove the Record<string, unknown> cast from the payload
assignment in the contract integration test and assert the absence of
headerFooter directly on the unknown value returned by parsePayload(result).
Preserve the existing Object.keys and negative containment assertion without
introducing another cast.

In `@src/parser/docx/core-metadata.ts`:
- Around line 29-47: Update the metadata parsing flow around coreParser.parse in
the try block to call XMLValidator.validate(xml) before parsing. When validation
fails, use the existing warning/error handling for unreadable core metadata and
avoid returning the unknown fallback; only invoke parse after validation
succeeds.

In `@src/parser/docx/header-footer-region.ts`:
- Around line 291-317: Update assignSegmentsToCells so the overflow warning is
emitted only when at least one segment after index 2 contains content, rather
than whenever segments.length exceeds 3. Preserve the existing unmodeled warning
details and cell assignment behavior for genuine overflow while ignoring empty
trailing segments.

In `@src/parser/docx/header-footer.ts`:
- Around line 11-13: Update the ParseWarning type import in header-footer.ts to
come from the ../../ast/index.js barrel, matching HeaderFooterCompositionSchema
and HeaderFooterComposition, and remove the direct ../../ast/types.js import.

In `@src/parser/docx/index.test.ts`:
- Around line 984-991: Declare the missing OOXML namespace prefixes in both test
fixtures: update docWithSectPr in src/parser/docx/index.test.ts at lines 984-991
to add the r namespace on the document root, and update the header fixture at
lines 1066-1070 to add the wp namespace on the header root. No other fixture
behavior should change.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d3ca6e84-fe45-4e0f-bd07-c5bd394f4ac8

📥 Commits

Reviewing files that changed from the base of the PR and between d5f7797 and a07ea02.

📒 Files selected for processing (36)

• docs/adr/068-header-footer-capture-mapping.md
• openapi.yaml
• src/api/header-footer-warning-openapi.test.ts
• src/ast/header-footer-schemas.test.ts
• src/ast/header-footer-schemas.ts
• src/ast/index.ts
• src/ast/schemas-line-budget.test.ts
• src/ast/schemas.test.ts
• src/ast/schemas.ts
• src/ast/spec-tree-schemas.ts
• src/ast/types.ts
• src/lib/parse-worker.test.ts
• src/lib/parse-worker.ts
• src/mcp/contract.integration.test.ts
• src/parser/docx/core-metadata.ts
• src/parser/docx/corpus-parts.integration.test.ts
• src/parser/docx/document.ts
• src/parser/docx/header-footer-field-recognition.test.ts
• src/parser/docx/header-footer-field-recognition.ts
• src/parser/docx/header-footer-parts.test.ts
• src/parser/docx/header-footer-parts.ts
• src/parser/docx/header-footer-region.test.ts
• src/parser/docx/header-footer-region.ts
• src/parser/docx/header-footer-relationships.test.ts
• src/parser/docx/header-footer-relationships.ts
• src/parser/docx/header-footer.test.ts
• src/parser/docx/header-footer.ts
• src/parser/docx/index.test.ts
• src/parser/docx/index.ts
• src/parser/docx/line-budget.test.ts
• src/parser/docx/source-detection.ts
• src/parser/docx/types.test.ts
• src/parser/docx/types.ts
• src/parser/error.test.ts
• src/parser/error.ts
• src/parser/text/index.ts

[thewrz]

Second adversarial review (GPT-5.5, xhigh) — dispositions

Ran a second independent reviewer over the branch diff. Three findings; dispositions below.

[P1] Deprecated XMLValidator fails enforced lint — HANDLED (a39b3b9).
fast-xml-parser's XMLValidator is flagged sonarjs/deprecation (relocated to a separate fast-xml-validator package in 5.x). It still ships and works in the pinned version, so rather than take on a whole new dependency (added attack surface) for one core.xml validity check, it is used with a scoped, justified eslint-disable-next-line sonarjs/deprecation on the import and call site. pnpm lint passes locally.

[P2] w:fldSimple fields captured as literals, not field references — DECLINED here, filed #485.
Confirmed: collectRuns flattens the inner cached-result run and drops the w:fldSimple wrapper + w:instr, so a simple PAGE/DATE field is captured as a literal (and unrecognized simple fields get no unmodeled warning). The cached text is preserved (not lost), so this is a field-recognition gap, not data loss. Full simple-field recognition is a feature addition beyond this PR's complex-field scope and warrants its own tests — tracked in #485.

[P2] Standalone border-only rule-line paragraph silently dropped — DECLINED here, filed #484.
Confirmed: a rule line authored as an empty paragraph with w:pBdr and no runs is not content-bearing, so captureRuleLine is never reached and it is dropped with no raw.unmodeled entry — an ADR-068 criterion-4 gap for that shape. A faithful fix promotes the standalone rule line into the region model (changing the "first content-bearing paragraph" rule-line rule) and warrants a short ADR-068 addendum + regression tests, so it is tracked in #484 rather than expanding this PR.

Both P2s are genuine but edge-case OOXML shapes (Word overwhelmingly authors border-on-text and complex fields); keeping them out of this already-large, CI-green PR follows the one-demonstrable-change / separate-issue conventions.

[thewrz]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.