feat(ast): semantic article-role tagging — Related Sections / References / Submittals

[thewrz]

Closes #258.

Why

The canonical CSI AST was semantic-light: every PART child was a generic article node with no role tag, so there was no deterministic way to ask "which article is the Related Sections article? References? Submittals?". This is the keystone (Task-0) of the #256 wishlist: it unblocks the article↔body coordination checks (A2/A3 — Related Sections ↔ body refs, B2 — References ↔ cited standards) and the deterministic submittal register (item D). It lands first because it is the single most reusable foundation.

What

Classifies PART articles by role deterministically from the article heading and surfaces it as an optional meta.articleRole on SpecNode, without mutating any existing field:

• 8 recognized roles (closed enum): summary, references, definitions, related-sections, submittals, quality-assurance, delivery-storage-handling, warranty — matched against a data table of canonical CSI titles + documented variants ("Related Requirements", "Reference Standards", "Section Includes", …).
• Tolerant matching: strips an optional leading CSI numbering prefix (1.1, 1.02, 1.1.1), uppercases, and collapses whitespace before an exact-title lookup. Robust whether or not a parser already stripped the prefix.
• Surfaces through the read API / MCP get_spec automatically — both serialize the SpecTree from getSpecTree, and the new field rides that serialization (no MCP tools.ts change needed). Documented in openapi.yaml's SpecNode.meta schema (CI contract gate green).
• Unknown/non-standard articles carry no role (the key is absent), never a wrong one.

Design decisions

• Role is DERIVED, not stored (ADR-033). It is a pure function of the heading text — no DB column, no migration. The single deriver is applied at two chokepoints: at parse time (parser/index.ts) so freshly-parsed trees carry it, and on DB read (db/queries/specs.ts buildNodeTree) so reconstructed trees carry it. Same pure function, two call sites — no drift, and editing a heading re-classifies for free. This mirrors how meta.editability/conflicts are already shaped on read.
• Lives in src/ast/ (the foundational leaf both parser/ and db/ import from via the barrel), so the module boundary stays clean.
• CPI ilvl offset is irrelevant to classification — the 5-signal inference engine already normalizes the offset into node_type='article' before the deriver runs, so ARCAT and CPI headings classify identically. Guarded by gated ARCAT + CPI fixture tests.
• Only article nodes are tagged. note/continuation never get a role (a note reading "REFERENCES" stays a note with no role) — enforced at both chokepoints.
• tagArticleRoles/withArticleRoles are identity-preserving on a no-op (return the same reference when nothing is tagged), so parse() allocates nothing when there's nothing to classify — preserving the existing reference-identity invariant in parse.test.ts.
• Genuine ambiguity is documented, not silently resolved: a nested "REFERENCES" sub-heading classifies identically to the PART-1 References article because the deriver sees text, not tree position — captured as a // KNOWN AMBIGUITY: test per the OOXML ambiguity rule.
• Rejected alternatives (in ADR-033): a persisted article_role column (adds a migration + staleness risk for no benefit) and a dedicated role node-type (forks every existing article consumer).

Testing

• Unit tests pass — pnpm test 1030/1030 (incl. 32 new: deriver classification, prefix-strip safety, immutability, KNOWN AMBIGUITY, parse-path .SEC, buildNodeTree read-path).
• Lint clean — pnpm lint (eslint + tsc + prettier).
• Integration: 620 passed, contract gate (contract.integration.test.ts) 8/8 — the openapi.yaml articleRole addition is validated against the live routes. (The 2 docs.integration.test.ts failures observed locally are an express-sendFile dotfile artifact of the .worktrees checkout path — they pass in CI / a non-dot checkout and are untouched by this branch.)
• Real ARCAT + CPI DOCX fixtures (gated skipIf; run in CI where the copyrighted fixtures are present) — assert References classifies and the CPI offset doesn't break it.
• CI green

🤖 Co-authored by Claude Opus 4.8 (1M context). Closes #258.

Summary by CodeRabbit

• New Features

  • Added automatic semantic tagging for article headings, so recognized articles now include an articleRole value in metadata.
  • Supported roles are documented in the API schema and available across parsed and loaded spec content.

• Bug Fixes

  • Improved heading matching to handle case differences, extra whitespace, and leading numbering prefixes.
  • Ensured unknown or unmatched headings remain untagged instead of being misclassified.

[coderabbitai]

Warning

Review limit reached

@thewrz, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 5 minutes and 43 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e68bdb17-14e6-47e3-83da-b1383635a7db

📥 Commits

Reviewing files that changed from the base of the PR and between eb05a17 and ac10ee4.

📒 Files selected for processing (6)

• src/api/generate-revision.integration.test.ts
• src/api/paragraphs.integration.test.ts
• src/ast/article-role.test.ts
• src/ast/article-role.ts
• src/db/queries/paragraphs.ts
• src/db/queries/revisions.ts

📝 Walkthrough

Walkthrough

Adds derived meta.articleRole tagging for article nodes, defines the schema/type and OpenAPI field for the new role enum, applies tagging during parse and DB tree reconstruction, and adds unit and fixture-based tests plus ADR and implementation-plan documentation.

Changes

Semantic article-role tagging

Layer / File(s)	Summary
Decision and rollout docs docs/adr/033-article-role-tagging.md, docs/superpowers/plans/2026-06-24-article-role-tagging.md	Documents the derived articleRole field, matching rules, parser and DB derivation points, API schema update, and validation checklist.
AST role contract and classifier src/ast/article-role.ts, src/ast/schemas.ts, src/ast/types.ts, src/ast/index.ts, openapi.yaml, src/ast/article-role.test.ts	Adds the closed ArticleRole enum/type surface, the heading classifier and tree tagger, the AST barrel re-exports, the OpenAPI field, and unit tests for classification and immutable tagging.
Parser and DB tagging src/parser/index.ts, src/db/queries/specs.ts, src/parser/article-role.test.ts, src/db/queries/specs.test.ts, src/parser/docx/arcat.integration.test.ts, src/parser/docx/cpi.integration.test.ts	Applies article-role tagging during parse() for .sec, .docx, and .txt, derives the same field in buildNodeTree, and adds SEC, DB, ARCAT, and CPI coverage.

Sequence Diagram(s)

sequenceDiagram
  participant parse
  participant tagArticleRoles
  participant deriveArticleRole
  participant buildNodeTree
  parse->>tagArticleRoles: post-process parsed trees
  tagArticleRoles->>deriveArticleRole: derive articleRole from heading text
  buildNodeTree->>deriveArticleRole: derive articleRole from stored article text

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• wrzonance/SpecR#62: Both PRs modify src/parser/index.ts’s parse() post-processing pipeline to add derived metadata to parsed trees.
• wrzonance/SpecR#205: Both PRs update src/db/queries/specs.ts/buildNodeTree to populate a new derived SpecNode.meta field on read.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The PR adds a large implementation-plan document under docs/superpowers/plans, which is outside the linked issue's required ADR and feature changes.	Remove the planning doc from this PR or split it into a separate planning-only change, keeping the feature/ADR implementation focused.
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the core feature: semantic article-role tagging for article nodes, and names the main roles affected.
Linked Issues check	✅ Passed	The changes add derived articleRole tagging, parser/DB propagation, OpenAPI exposure, ADR documentation, and tests that match the linked issue requirements.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/issue-258

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/ast/article-role.ts`:
- Around line 53-55: The normalizeHeading helper in article-role.ts is stripping
NUMBER_PREFIX_RE before trimming, so headings with leading whitespace can keep
their numeric prefix and fail lookup. Update normalizeHeading to trim the input
first, then remove the numbering prefix, then normalize internal whitespace and
uppercase the result. Keep the fix localized to normalizeHeading and preserve
its existing normalization behavior otherwise.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7c8c03cc-88aa-4e8f-a9a5-11141c8be8c5

📥 Commits

Reviewing files that changed from the base of the PR and between 6e00857 and eb05a17.

📒 Files selected for processing (14)

• docs/adr/033-article-role-tagging.md
• docs/superpowers/plans/2026-06-24-article-role-tagging.md
• openapi.yaml
• src/ast/article-role.test.ts
• src/ast/article-role.ts
• src/ast/index.ts
• src/ast/schemas.ts
• src/ast/types.ts
• src/db/queries/specs.test.ts
• src/db/queries/specs.ts
• src/parser/article-role.test.ts
• src/parser/docx/arcat.integration.test.ts
• src/parser/docx/cpi.integration.test.ts
• src/parser/index.ts

[thewrz]

Codex review (GPT-5.5, xhigh) — additional eyes: 1 finding (P2), fixed.

• [P2] Derive articleRole for subtree responses too (src/db/queries/paragraphs.ts) — Confirmed valid. buildSubtree (the PATCH /specs/:id/paragraphs/:nodeId response path via fetchSubtreeNode) shaped meta but did not mirror buildNodeTree's new articleRole derivation, so editing an article heading to/from a recognized CSI title omitted meta.articleRole from the immediate PATCH response until a full-tree refetch. Fixed in 78dfc89: the subtree builder now applies the same node_type='article' → deriveArticleRole(text) derivation. Added an API-boundary regression test (PATCH a heading to 1.2 REFERENCES → meta.articleRole === 'references'), verified RED before the fix and GREEN after.

No P1 findings; parse path, full-tree read path, and now the subtree response path all derive the role from the single deriveArticleRole source of truth.

[thewrz]

Review loop complete ✅

CI: Build · Lint · Test (unit + integration) · LOC-delta · contract gate — all green on e01c036.

CodeRabbit (genuine review, not a no-op — "Actionable comments posted: 1"):

• ✅ normalizeHeading trim-order (src/ast/article-role.ts) — valid. The ^-anchored prefix regex was applied before trim(), so a heading with incidental leading whitespace (" 1.1 REFERENCES") kept its numeric prefix and failed the role lookup. Fixed in e01c036 (trim → strip → collapse → trim); the "3D MODELING" anchor still holds. Pinned with a regression test. Thread resolved.

Codex (GPT-5.5, xhigh — additional eyes): 1 finding (P2), fixed.

• ✅ articleRole missing from the PATCH /specs/:id/paragraphs/:nodeId (subtree) response path (src/db/queries/paragraphs.ts) — valid. buildSubtree did not mirror buildNodeTree's new derivation, so editing an article heading to/from a recognized CSI title omitted meta.articleRole until a full-tree refetch. Fixed in 78dfc89; API-boundary regression test added (RED→GREEN verified).

Both fixes reuse the single deriveArticleRole source of truth — parse path, full-tree read, and subtree response now derive the role identically. Leaving open for @thewrz to merge.

[thewrz]

Codex re-review (GPT-5.5, xhigh) on e01c036 surfaced a second, subtler P2 — confirmed valid and fixed.

• [P2] Derived articleRole leaks into revision fingerprints (src/db/queries/revisions.ts treeFingerprint / changedSpecs) — valid. snapshotMemberTrees freezes buildNodeTree output into package_revision_specs.tree, and changedSpecs compares revisions via JSON.stringify(SpecTreeSchema.parse(tree)). Snapshots frozen before this PR have no articleRole key; a target frozen after does — so an addendum diffing a post-change target against a pre-change base flagged every role-bearing section as changed and listed unchanged sections in Affected Sections. Fixed in 74f57d7: treeFingerprint now strips derived meta (articleRole) before serializing, so change-detection compares authored content only (the role is a pure function of the heading text, which is already fingerprinted). Added an integration regression test that simulates a pre-tagging base snapshot and asserts only the genuinely-edited section is affected — verified RED (addendum rendered all 3 sections) → GREEN (only the changed section).

This is why I re-ran Codex against the final HEAD after the first two fixes — the deriver's reach into the persisted revision-snapshot path wasn't visible from the original diff against the pre-tagging base. All CI + the contract gate stay green.

[thewrz]

Codex re-review (GPT-5.5, xhigh) on 74f57d7 — one more P2, confirmed valid and fixed.

• [P2] Narrow prefix stripping to CSI article numbers (src/ast/article-role.ts:51) — valid. NUMBER_PREFIX_RE used (?:\.\d+)* (zero-or-more dotted groups), so the \s+ terminator branch also stripped a bare integer or year: "1 REFERENCES" and "2024 REFERENCES" both normalized to "REFERENCES" and classified references. A bare integer/year is not a CSI article number (those are dotted — 1.1, 1.02, 1.1.1), so this misclassified non-CSI headings, violating ADR-033's absent rather than wrong contract. Fixed in ac10ee4: require at least one dotted group ((?:\.\d+)+). All real CSI/CPI forms still strip; bare integers/years are now left intact and derive no role; the "3D MODELING" guard is unchanged. Regression test pins it ("2024 REFERENCES"/"1 REFERENCES" → no role; "1.1 REFERENCES"/"1.1.1 REFERENCES" → references), RED→GREEN verified.

Note: the existing deriveArticleRole('2024 REQUIREMENTS') test passed only because REQUIREMENTS isn't a role title — it didn't actually exercise the strip. The new test uses a real role title (REFERENCES) so it genuinely guards the regression.

Three Codex passes total on this PR (initial + two re-reviews as fixes landed), each catching a distinct reach of the deriver: subtree PATCH response, revision fingerprint, and now the prefix grammar. Running one more pass on ac10ee4 to confirm clean.

[thewrz]

Review loop converged ✅ — final Codex pass clean

After the dotted-prefix fix, a 4th Codex pass on ac10ee4 returned: "No actionable correctness issues were found in the diff. The article role derivation is applied consistently on parse/read paths, documented in the schema, and derived metadata is excluded from revision fingerprints."

Total findings handled on this PR — 4 (1 CodeRabbit + 3 Codex), all valid, all fixed + pinned with RED→GREEN regression tests:

#	Source	Issue	Commit
1	Codex	articleRole missing from PATCH/subtree response path	78dfc89
2	CodeRabbit	normalizeHeading stripped prefix before trimming (leading-ws headings)	e01c036
3	Codex	derived articleRole leaked into revision fingerprints → unchanged sections in addendum	74f57d7
4	Codex	bare integer/year prefix (1 REFERENCES, 2024 REFERENCES) misclassified	ac10ee4

Each Codex pass (run synchronously after fixes landed) caught a distinct reach of the new deriver that the prior diff couldn't see — subtree response, revision snapshot, prefix grammar — which is exactly why I re-ran it to convergence rather than once.

Status: all CI green on ac10ee4 (Build · Lint · Test · LOC · contract gate), CodeRabbit review completed, 1/1 threads resolved. Leaving OPEN for @thewrz to merge per the admin-merge default.