feat: external content association (paragraph ↔ external document reference) (#109)

[thewrz]

Why

ADR-019 affirms external content association as in-scope: a firm links its own
collateral (e.g. a PDF datasheet) to a Part 2 product paragraph. SpecR stores the
link + provenance only — never the licensed bytes (the DMS owns transport, ADR-014).
This lets a spec writer point a reviewer at the authoritative manufacturer document for a
product paragraph without SpecR becoming a content host. Closes #109.

What

A new paragraph_associations table keyed on the paragraph UUID (the stable w:sdt
round-trip anchor) so associations survive spec regeneration. Each row carries a dual-mode
external reference and is surfaced read-only inside the spec tree and MCP get_paragraph,
and managed through a small REST sub-resource.

• Data model (migration 032): paragraph_associations — paragraph_id (FK→paragraphs
ON DELETE CASCADE, the survival key) + denormalized spec_id; label; dual identity
  external_provider+external_id (ADR-014 D5) or url+content_hash; opaque
  external_metadata JSONB. A CHECK enforces at least one identity; no bytes column.
• AST: ParagraphAssociation DTO + SpecNodeMeta.associations? + the Zod
  CreateAssociationBodySchema (v4 idiom, .check()).
• Query layer src/db/queries/associations.ts: create / list-for-paragraph /
  list-for-spec / delete + typed AssociationParagraphNotFoundError.
• Surfacing: getSpecTree attaches meta.associations per node (immutable rebuild);
  getParagraphWithAncestors attaches them to the leaf node only. MCP get_paragraph /
  get_spec ride the same DB reads — no handler change needed.
• REST: GET/POST /specs/:id/paragraphs/:nodeId/associations and
  DELETE …/:associationId, documented in the CI-enforced openapi.yaml (contract gate green).

Design decisions

• Dual identity (DMS pair OR url+hash), one table. ADR-014 D5 fixes the DMS identity as
  opaque external_provider+external_id; ADR-019 requires link + provenance. A firm with a
  DMS connector links by the connector pair; a firm without one (the common near-term case)
  links by url+content_hash. One nullable-column table with a presence CHECK serves both —
  no fork into two tables. The same "url OR provider+id" rule is encoded consistently in the DB
  CHECK, the Zod refinement, and the openapi schema.
• Keyed on paragraph_id, not content. The w:sdt paragraph UUID is stable across
  AST→DOCX regeneration, so an association keyed on it survives a regenerate/merge that rewrites
  text and bumps content_version. spec_id is denormalized purely so the spec-tree read is one
  indexed query; the survival guarantee rides on paragraph_id. Pinned by an explicit
  regeneration test.
• Cross-spec request → 404, not 403. When :nodeId doesn't belong to :id, the association
  sub-resource simply doesn't exist under that path, so 404 is the honest status.
• Provenance only, never bytes. There is deliberately no column or endpoint for file content;
  SpecR never dereferences url. Transport stays with the DMS (ADR-014), keeping the platform
  content-neutral (ADR-019) and MIT-redistributable.

Scope (held)

Out of scope per the issue and ADRs, and not built: storing/serving file bytes, any DMS
transport, scheduling. No update/PATCH endpoint (YAGNI).

Testing

• Unit tests pass (pnpm test — 988/988)
• Integration tests pass (pnpm test:integration — 534 pass; the only 2 failures are
  pre-existing & unrelated src/api/docs.integration.test.ts Scalar/openapi-serving cases that
  fail at the branch base too)
• Lint clean (pnpm lint — eslint + tsc strict + prettier)
• openapi contract gate green (route↔spec bidirectional + response-schema)
• Acceptance 1 — associate a datasheet to a Part 2 paragraph, visible via REST + MCP
  (src/api/associations.integration.test.ts, src/mcp/associations.integration.test.ts)
• Acceptance 2 — association survives spec regeneration, keyed on paragraph UUID
  (src/db/queries/associations.integration.test.ts)
• CI green

🤖 Co-authored by Claude Opus 4.8. Closes #109.

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

Summary by CodeRabbit

Release Notes

• New Features
  • Users can now create, list, and delete external content associations linked to paragraphs within specifications.
  • Associations support direct URLs or external provider references with optional content hashing for validation.
  • Associated external content metadata is preserved across specification regenerations.
  • Associations are surfaced in API responses for spec trees and paragraph retrieval operations.

[coderabbitai]

📝 Walkthrough

Walkthrough

Implements External Content Association (#109): adds a paragraph_associations PostgreSQL table, a full DB query module with CRUD operations, read-path enrichment of getSpecTree and getParagraphWithAncestors, three REST endpoints under /specs/:id/paragraphs/:nodeId/associations, OpenAPI contract updates, and an MCP integration test confirming association visibility via handleGetParagraph.

Changes

External Content Association

Layer / File(s)	Summary
Data model, migration, and Zod schema src/db/migrations/032_create_paragraph_associations.ts, src/ast/types.ts, src/ast/association-schemas.ts, src/ast/associations.test.ts, src/ast/index.ts	Defines the ParagraphAssociation interface and extends SpecNodeMeta.associations; creates the paragraph_associations table migration with FK/check constraints and indexes; introduces CreateAssociationBodySchema with dual-mode identity refinement (DMS pair or URL), unit tests, and barrel re-exports.
DB query module: CRUD, error types, barrel export src/db/queries/associations.ts, src/db/queries/associations.integration.test.ts, src/db/index.ts	Adds AssociationParagraphNotFoundError, CreateAssociationInput, row-to-DTO mapper, resolveSpecId, createAssociation, listAssociationsForParagraph, listAssociationsForSpec, and deleteAssociation; re-exports from src/db/index.ts; integration tests cover CRUD, error paths, durability across spec regeneration, and correct grouping by paragraph.
Read-path enrichment: getSpecTree and getParagraphWithAncestors src/db/queries/specs.ts, src/db/queries/paragraphs.ts	Adds attachAssociations recursive helper to getSpecTree to populate meta.associations from a spec-wide association map; updates getParagraphWithAncestors to fetch and attach associations to the leaf node only; extends ParagraphRow with optional associations.
REST handlers, OpenAPI contract, router, and integration tests src/api/associations.ts, src/api/router.ts, openapi.yaml, src/api/contract.integration.test.ts, src/api/associations.integration.test.ts	Adds createAssociationHandler, listAssociationsHandler, and deleteAssociationHandler with a resolveIds spec-ownership guard; registers three routes under /specs/:id/paragraphs/:nodeId/associations; adds ParagraphAssociation/CreateAssociation OpenAPI schemas, AssociationId parameter, and SpecNode.meta.associations; updates contract allowlist; exercises all endpoints with HTTP integration tests including cross-spec 404 and delete idempotency.
MCP integration test src/mcp/associations.integration.test.ts	Seeds a spec, paragraph, and association in the database; invokes handleGetParagraph; asserts the returned JSON payload contains the expected association label.
Implementation plan documentation docs/superpowers/plans/2026-06-23-external-content-association.md	Documents the full end-to-end implementation plan covering architecture, DB schema, query layer, REST contract, MCP/regeneration acceptance criteria, and a final checklist.

Sequence Diagram(s)

sequenceDiagram
  rect rgba(173, 216, 230, 0.5)
    Note over HTTPClient,DB: Create Association
    participant HTTPClient as HTTP Client
    participant Router as Express Router
    participant createAssociationHandler
    participant resolveIds
    participant DB as DB Query Layer
    HTTPClient->>Router: POST /specs/:id/paragraphs/:nodeId/associations
    Router->>createAssociationHandler: req, res
    createAssociationHandler->>resolveIds: specId, nodeId UUIDs
    resolveIds->>DB: SELECT spec_id FROM paragraphs WHERE id=nodeId
    DB-->>resolveIds: spec_id or null
    alt paragraph not owned by spec
      resolveIds-->>HTTPClient: 404
    end
    createAssociationHandler->>DB: createAssociation(nodeId, body)
    DB-->>createAssociationHandler: ParagraphAssociation
    createAssociationHandler-->>HTTPClient: 201 { success, data }
  end

  rect rgba(144, 238, 144, 0.5)
    Note over Consumer,DB: Read Associations via Spec Tree
    participant Consumer
    participant getSpecTree
    participant listAssociationsForSpec
    participant attachAssociations
    Consumer->>getSpecTree: specId
    getSpecTree->>listAssociationsForSpec: specId
    listAssociationsForSpec-->>getSpecTree: Map paragraphId → ParagraphAssociation[]
    getSpecTree->>attachAssociations: SpecNode[], associationMap
    attachAssociations-->>getSpecTree: SpecNode[] with meta.associations
    getSpecTree-->>Consumer: enriched SpecTree
  end

  rect rgba(255, 223, 186, 0.5)
    Note over MCP,DB: MCP get_paragraph includes associations
    participant MCP as MCP Tool
    participant handleGetParagraph
    participant getParagraphWithAncestors
    participant listAssociationsForParagraph
    MCP->>handleGetParagraph: { paragraphId }
    handleGetParagraph->>getParagraphWithAncestors: paragraphId
    getParagraphWithAncestors->>listAssociationsForParagraph: paragraphId
    listAssociationsForParagraph-->>getParagraphWithAncestors: ParagraphAssociation[]
    getParagraphWithAncestors-->>handleGetParagraph: node with associations
    handleGetParagraph-->>MCP: JSON with association label
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• wrzonance/SpecR#55: Implements the MCP get_paragraph tool on top of getParagraphWithAncestors, the same DB query this PR extends to attach meta.associations.
• wrzonance/SpecR#187: Modifies getSpecTree and SpecNode.meta to attach per-paragraph metadata (sourceFacts), the same enrichment pattern this PR applies for associations.
• wrzonance/SpecR#205: Extends getSpecTree in src/db/queries/specs.ts to populate additional SpecNodeMeta fields, the same code path this PR modifies to attach meta.associations.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly summarizes the main feature—external content association enabling paragraph-document references—and directly matches the primary objective (issue #109).
Linked Issues check	✅ Passed	All coding requirements from issue #109 are met: dual-identity model (DMS connector or URL+hash), paragraph UUID keying for regeneration survival, REST endpoints, MCP visibility, and association storage with provenance-only semantics per ADR-014.
Out of Scope Changes check	✅ Passed	All changes directly support the external content association feature. No unrelated modifications to unscoped areas (file byte storage, DMS transport, scheduling) are present.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/issue-109

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 6

🧹 Nitpick comments (3)

docs/superpowers/plans/2026-06-23-external-content-association.md (2)

26-40: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Smooth out repetitive sentence starters in the file structure section.

Lines 26–40 contain many consecutive bullet points starting with "Create" or "Modify". While the content is clear and actionable, LanguageTool flags the repetition as a style issue. Consider grouping by category or varying the phrasing (e.g., "Introduce", "Extend", "Wire") to improve readability without losing clarity.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-06-23-external-content-association.md` around
lines 26 - 40, The bullet points in the file structure section contain
repetitive sentence starters with "Create" and "Modify" appearing consecutively
throughout lines 26–40, which impacts readability. To fix this, vary the
phrasing by replacing some instances of "Create" with alternatives like
"Introduce" or "Set up", and replace some "Modify" instances with "Extend",
"Wire", or "Update" depending on the context of each task. Consider grouping
related tasks (e.g., all new file creation together, then all modifications
together) to naturally reduce consecutive repetition while maintaining clarity
and actionability of the checklist.

Source: Linters/SAST tools

---

943-951: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Clarify OpenAPI schema generation for ParagraphAssociation and CreateAssociation schemas.

Task 5, Step 5 (lines 943–951) defers the exact OpenAPI schema definitions to "follow existing components.schemas entries" but does not provide the complete YAML structure. While the DTO shape is fully specified in the locked row schema (lines 44–61) and Task 2 (ParagraphAssociation interface), consider adding a brief YAML snippet showing the ParagraphAssociation and CreateAssociation schemas as examples, so implementers know precisely which fields are required and optional, array nesting, etc.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-06-23-external-content-association.md` around
lines 943 - 951, The OpenAPI schema definitions for ParagraphAssociation and
CreateAssociation in Step 5 lack concrete YAML structure examples, leaving
ambiguity about required vs optional fields and proper formatting. Add complete
YAML schema snippets under Step 5 (around lines 943-951) showing the
ParagraphAssociation schema with required fields (id, label, externalMetadata,
createdAt) and optional fields (externalProvider, externalId, url, contentHash),
and the CreateAssociation schema mirroring the DTO structure from lines 44-61.
Include explicit field types, descriptions, and required array specifications so
implementers have precise guidance on schema structure.

src/api/contract.integration.test.ts (1)

81-82: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Prefer explicit response-schema assertions for new association endpoints instead of allowlisting.

Allowlisting these new JSON operations reduces contract guardrails for response-shape drift. Add explicit assertResponse(...) checks in src/api/associations.integration.test.ts and remove these allowlist entries once covered.

As per coding guidelines, src/api/**/*.ts: openapi.yaml is the live, authoritative API contract, so response conformance should stay directly tested.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/api/contract.integration.test.ts` around lines 81 - 82, Remove the two
allowlist entries for 'get /specs/{}/paragraphs/{}/associations' and 'post
/specs/{}/paragraphs/{}/associations' from the allowlist in
src/api/contract.integration.test.ts, then add explicit assertResponse(...)
checks in src/api/associations.integration.test.ts to validate the response
schemas for these two endpoints against the openapi.yaml contract. This ensures
proper response-shape validation instead of bypassing it through allowlisting.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openapi.yaml`:
- Around line 272-290: Add a 400 response reference to the GET and DELETE
association operations in the OpenAPI specification to reflect that these
endpoints return HTTP 400 for invalid UUID path parameters. After the existing
404 and 500 response definitions in both operations (around the areas at lines
272-290 and 337-343), insert a new 400 response entry that references an
appropriate BadRequest error response component, ensuring the OpenAPI contract
matches the actual behavior of the association endpoints in
src/api/associations.ts.
- Around line 4238-4265: The ParagraphAssociation and CreateAssociation schemas
currently allow any combination of optional properties but the server enforces a
constraint that either url must be present, or both externalProvider and
externalId must be present together. Update both schema definitions to encode
this invariant using OpenAPI's oneOf construct to create mutually exclusive
groups: one group requiring url, and another group requiring both
externalProvider and externalId together. This ensures the schema accurately
reflects the server's validation logic and prevents clients from creating
invalid requests.

In `@src/ast/association-schemas.ts`:
- Around line 19-29: The current validation in the .check() method only rejects
when both hasDmsPair and hasUrl are false, but it permits partial DMS identity
(externalProvider without externalId or vice versa) whenever url is present.
Update the validation logic to ensure that externalProvider and externalId are
both defined or both undefined regardless of whether url exists. Add an
additional condition to check for mismatched DMS fields (where exactly one of
externalProvider or externalId is defined) and push an error to ctx.issues when
this partial state is detected, preventing mixed identity states from passing
validation.

In `@src/ast/associations.test.ts`:
- Around line 23-29: Add a new test case in the associations.test.ts file to
cover the regression scenario where a url is provided along with an incomplete
DMS pair (either externalProvider without externalId, or externalId without
externalProvider). The test should use CreateAssociationBodySchema.safeParse()
to verify that such combinations are properly rejected, similar to the existing
test structure, but with a url field included to ensure the schema rejects this
specific invalid combination of partially present identity credentials.

In `@src/db/migrations/032_create_paragraph_associations.ts`:
- Around line 25-27: The CHECK constraint in the pgm.addConstraint call for
paragraph_associations_identity_check is logically incomplete. It currently
allows partial external provider/id configurations when url is set, violating
the identity invariant. Modify the check predicate to enforce that either both
external_provider AND external_id are NOT NULL together, or both
external_provider AND external_id are NULL with url NOT NULL. The corrected
constraint should be: (external_provider IS NOT NULL AND external_id IS NOT
NULL) OR (external_provider IS NULL AND external_id IS NULL AND url IS NOT
NULL).

In `@src/db/queries/associations.ts`:
- Around line 67-83: The createAssociation function has a race condition where
resolveSpecId checks if the paragraph exists, then performs a separate INSERT
operation, allowing the paragraph to be deleted between these steps and
resulting in a generic 500 error instead of a proper 404. Replace the separate
resolveSpecId call and INSERT statement with an atomic INSERT...SELECT query
that joins against the spec table in a single operation, eliminating the race
condition. When the INSERT returns no rows (meaning the paragraph was not
found), throw AssociationParagraphNotFoundError instead of falling through to
generic error handling. Apply this same atomic pattern to the other similar
INSERT operation referenced in the range 88-93.

---

Nitpick comments:
In `@docs/superpowers/plans/2026-06-23-external-content-association.md`:
- Around line 26-40: The bullet points in the file structure section contain
repetitive sentence starters with "Create" and "Modify" appearing consecutively
throughout lines 26–40, which impacts readability. To fix this, vary the
phrasing by replacing some instances of "Create" with alternatives like
"Introduce" or "Set up", and replace some "Modify" instances with "Extend",
"Wire", or "Update" depending on the context of each task. Consider grouping
related tasks (e.g., all new file creation together, then all modifications
together) to naturally reduce consecutive repetition while maintaining clarity
and actionability of the checklist.
- Around line 943-951: The OpenAPI schema definitions for ParagraphAssociation
and CreateAssociation in Step 5 lack concrete YAML structure examples, leaving
ambiguity about required vs optional fields and proper formatting. Add complete
YAML schema snippets under Step 5 (around lines 943-951) showing the
ParagraphAssociation schema with required fields (id, label, externalMetadata,
createdAt) and optional fields (externalProvider, externalId, url, contentHash),
and the CreateAssociation schema mirroring the DTO structure from lines 44-61.
Include explicit field types, descriptions, and required array specifications so
implementers have precise guidance on schema structure.

In `@src/api/contract.integration.test.ts`:
- Around line 81-82: Remove the two allowlist entries for 'get
/specs/{}/paragraphs/{}/associations' and 'post
/specs/{}/paragraphs/{}/associations' from the allowlist in
src/api/contract.integration.test.ts, then add explicit assertResponse(...)
checks in src/api/associations.integration.test.ts to validate the response
schemas for these two endpoints against the openapi.yaml contract. This ensures
proper response-shape validation instead of bypassing it through allowlisting.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1e0f0e2c-d354-49c5-afb2-1708422224cf

📥 Commits

Reviewing files that changed from the base of the PR and between 4eff546 and f813311.

📒 Files selected for processing (17)

• docs/superpowers/plans/2026-06-23-external-content-association.md
• openapi.yaml
• src/api/associations.integration.test.ts
• src/api/associations.ts
• src/api/contract.integration.test.ts
• src/api/router.ts
• src/ast/association-schemas.ts
• src/ast/associations.test.ts
• src/ast/index.ts
• src/ast/types.ts
• src/db/index.ts
• src/db/migrations/032_create_paragraph_associations.ts
• src/db/queries/associations.integration.test.ts
• src/db/queries/associations.ts
• src/db/queries/paragraphs.ts
• src/db/queries/specs.ts
• src/mcp/associations.integration.test.ts