Skip to content

feat(catalog): screaming service layer + LLM integrations page (closes #71)#84

Merged
danielnaab merged 36 commits intomainfrom
story-71/llm-integrations-catalog
Apr 19, 2026
Merged

feat(catalog): screaming service layer + LLM integrations page (closes #71)#84
danielnaab merged 36 commits intomainfrom
story-71/llm-integrations-catalog

Conversation

@danielnaab
Copy link
Copy Markdown
Member

Summary

  • Every service at src/services/<name>/ now exposes a single public-interface index.ts. External callers import from services/<name> only — never a deeper path. Enforced by an extended test/architecture/dependency-rule.test.ts that also catches type-only imports.
  • Six orphan files migrated (storage.tsstorage/, user-store.tsauth/, project-service.ts + form-project-repo.ts → new projects/, strategy-registry.ts + errors.tsshared/). src/services/ now reads as 12 intent-named folders with no loose files.
  • New catalog page catalog/architecture/llm-integrations.md enumerates every LLM call site (extraction, shaping, filling, evaluation, future RAG) with src: permalinks that resolve to GitHub at the exact deployed commit. Supporting infrastructure: src/shared/build-info.ts, src/services/content/github-permalink.ts, markdown-it src: URL rewriter, and NixOS deploy wiring for BUILD_GIT_SHA.
  • New navigation page catalog/architecture/navigation.md documents the convention with llm-integrations.md as the worked example. software-architecture.md, data-model.md, and CLAUDE.md refreshed to match the new layout.

Story

Closes #71

Acceptance Criteria

  • Catalog page enumerates every LLM touchpoint with deep links to relevant GitHub source lines — catalog/architecture/llm-integrations.md
  • Each service has a single, obvious public-interface entrypoint — 12 services/<name>/index.ts files
  • software-architecture.md describes the "service public interface" convention — new subsection at lines 119-130
  • Internal helpers are clearly internal; imports route only through each service's public entrypoint — enforced by test/architecture/dependency-rule.test.ts (includes type-only imports)
  • Catalog pages link to the service's public entrypoint on GitHub, permalinked — via src: scheme pinned to deployed commit SHA
  • "How to navigate this codebase" section names the convention and points at LLM-integrations page — catalog/architecture/navigation.md
  • No behavior change; bun run check passes — 1269 tests pass post-merge

Test Plan

  • bun run check passes (1269 tests, 0 fails)
  • test/architecture/dependency-rule.test.ts passes, including the new cross-service rule
  • Sanity check: inserting a deliberate deep cross-service import fails the test with a helpful file:line message
  • After merge/deploy: verify /srv/forms-lab/main/.env contains BUILD_GIT_SHA=<sha>
  • After merge/deploy: systemctl show forms-lab-homepage -p EnvironmentFiles shows /srv/forms-lab/main/.env
  • After merge/deploy: load /catalog/architecture/llm-integrations and click several src: links — each should open GitHub at the expected file + line range, pinned to the deployed commit
  • After merge/deploy: load /catalog/architecture/navigation — renders cleanly, cross-links resolve

Review Notes

  • Branch history: 33 commits. Grouped as one-index.ts-per-service, per-target-service import rewrites, public-API trimming passes, permalink infrastructure, deploy wiring, dep-rule test extension, catalog pages, and a merge commit. See notes/story-71-llm-integrations-catalog/review.md for the full breakdown.
  • Test-only exports: During Task 10 review, found ~28 symbols were being added to index.ts files just so tests could import through the "public" API. Removed them; tests now deep-import internals. The convention is documented in navigation.md: production code goes through index.ts; tests unit-testing internals may deep-import.
  • Type-only imports: The new dep-rule deliberately includes them (the existing P2 test excludes them). Rationale: the public interface is about intent visibility, not just runtime coupling.
  • Merge with main: Main had 8 upstream commits during the story (3 infra PRs + 5 feature PRs, including a new services/rag/ service and shaping-CLI). Merged cleanly; conflicts in 3 files resolved to route main's new code through the public APIs. The dep-rule test caught two violations main introduced — fixed in the merge commit.
  • Follow-ups flagged but out of scope: dataCollectionSpecSchema may belong in data-collection/ rather than form-documents/ (P3); projects/project-service.ts is 822 lines with natural seams; storage/index.ts contains inline implementation rather than pure re-exports. None are blocking.

Related

  • Design: notes/story-71-llm-integrations-catalog/design.md
  • Plan: notes/story-71-llm-integrations-catalog/plan.md
  • Review: notes/story-71-llm-integrations-catalog/review.md

Captures the brainstormed design: service public-interface convention
via services/<name>/index.ts, orphan-file migration plan, build-time
commit SHA + githubPermalink helper, new catalog pages for LLM
integrations and codebase navigation, and extended dependency-rule
test to enforce cross-service import boundaries.
Thirteen bite-sized tasks covering: public-interface index.ts per service
(Task 1), orphan file moves (2-5), build-info + permalink helper (6-7),
markdown src: rewriter (8), NixOS BUILD_GIT_SHA wiring (9), import
rewrites (10), dependency-rule test extension (11), and two new catalog
pages (12-13).
Each service now exposes its public API via index.ts with a top-of-file
comment stating that external imports must route through this file.
Consumers are not yet rewritten — that happens in a later commit.
StrategyRegistry is a generic Map<key, factory> pattern with no domain
content. It belongs in shared/ per the architecture principles.
…ce + form-project-repo)

Form-project handling is a real domain boundary. Moving these two files
into their own folder with an index.ts public interface makes the
services/ directory read as a list of intents.
Resolves the git ref the app was built from so catalog permalinks can
point at the exact deployed code. Prefers BUILD_GIT_SHA env var (set
by deploy script), falls back to git rev-parse. Dirty worktrees use a
'dev-<branch>' marker.
Builds a GitHub blob URL pinned to the current build's git ref, with
optional line number or range anchor.
Catalog markdown can now write [label](src:src/path/file.ts#L42-L88)
and the renderer substitutes a GitHub permalink pinned to the current
build's git ref at render time. Non-src: links pass through untouched.
renderMarkdown now takes a RenderOptions argument; all 11 catalog route
callers updated.
The deploy script captures the commit SHA of each deployed worktree
and writes it into the per-branch .env, which the app service already
reads via EnvironmentFile. The homepage service now also loads main's
.env so getBuildInfo() can resolve the running commit without shelling
out to git at runtime. Ignoring the file when absent keeps first-boot
working via a leading '-' on the EnvironmentFile path.
These symbols are only consumed by tests. Tests now deep-import them
directly. The public index.ts is reserved for symbols with non-test
consumers.
These symbols are only consumed by tests. Tests now deep-import them
directly. The public index.ts is reserved for symbols with non-test
consumers.
These symbols are only consumed by tests. Tests now deep-import them
directly. The public index.ts is reserved for symbols with non-test
consumers.
Seven types had no callers anywhere; three more had only test consumers.
The tests now deep-import these symbols. The public index.ts now reflects
the evaluation service's real external surface.
…x.ts

Task 10 routed runtime imports through each service's index.ts but missed
type-only cross-service imports (the P2 test exempts them). The new
service public interface rule in Task 11 is stricter — type-only imports
are covered too, because the public interface is about intent visibility,
not runtime coupling.

Also exports `createToolUsePdfExtractor` from form-documents/index.ts so
that extraction/registry.ts can import it through the public entrypoint.
Extends the dependency-rule test with a new rule: imports from outside
service A into service A must resolve to A's index.ts (i.e. `services/A`
or `services/A/index`). Intra-service imports are unrestricted.
Type-only imports are included — the public interface is about intent
visibility, not just runtime coupling.
Enumerates every LLM call site in the codebase — extraction, shaping,
filling, evaluation, and a placeholder for future RAG — with src:
permalinks that pin to the deployed commit. Acts as the 'where every
LLM call lives' map for the codebase.
Navigation.md explains the service public interface convention with
LLM integrations as the worked example. software-architecture.md
gains a 'Service public interface' subsection under Structure,
and system-overview.md links to navigation.md.
AppError and subclasses have no domain content — they're HTTP-status-
coded error types used across all services. They belong in shared/
alongside the other pure utilities.
software-architecture.md, data-model.md, and CLAUDE.md's Project
Structure list all carried references to paths that no longer exist
(services/ingestion/, services/storage.ts, services/user-store.ts,
services/form-project-repo.ts, services/errors.ts). Updated to match
the current layout and name the new services and conventions.
Resolved conflicts:
- src/entrypoints/cli/commands/evaluate.ts: routed main's new evaluation
  imports (fixtureProjectState, shapingIntentFixtures, shapingCommandsKind,
  RunResult, FormShaper) through the public services/evaluation and
  services/forms APIs; moved strategy-registry import to shared/ per this
  branch's refactor.
- src/entrypoints/webhook/main.ts: kept main's teardownBranch import;
  routed createGitHubClient through services/deployment.
- src/services/form-documents/extraction.ts: routed main's new
  PolicyRetriever/PolicyChunk imports through services/rag; routed
  ExtractionExemplar through services/extraction.
- src/services/form-documents/hybrid-extraction-prompt.ts: routed
  ExtractionExemplar through services/extraction.
- src/design-system/components/flex-spec-{browser,diff-browser}/examples.tsx:
  routed type imports through services/{data-collection,forms} public APIs.
- src/services/evaluation/index.ts: re-added fixtureProjectState,
  shapingIntentFixtures, shapingCommandsKind, RunResult to the public
  interface — they were removed earlier as test-only but main's new CLI
  evaluation code uses them as genuine public API.
- test/evaluate-shaping-cli.test.ts: services/strategy-registry ->
  shared/strategy-registry per earlier refactor.
- catalog/architecture/software-architecture.md: de-duplicated forms/
  entry created by auto-merge.
@danielnaab danielnaab had a problem deploying to story-71-llm-integrations-catalog April 19, 2026 18:49 Failure
…undle buildable

Problem: the forms service barrel re-exports SqliteFormSessionGateway,
SqliteSubmissionGateway, and BedrockFillingAgent, which import bun:sqlite
and @aws-sdk/credential-providers. When a design-system client.ts file
does 'import { executeBatch } from services/forms' at runtime, Bun's
browser-target build walks the whole barrel and errors on those server-
only imports before tree-shaking can drop the unused classes. This broke
CI's Build step after the merge.

Fix:
- client.ts files in src/design-system/components/ deep-import from
  specific sub-modules (e.g. services/forms/shaping/commands) so the
  browser bundle only pulls server-safe code. This is a physical
  constraint of browser bundling, not an architecture failure — the
  existing P2 rule already treats client.ts as entrypoint-level for the
  same reason.
- dependency-rule.test.ts's cross-service rule now exempts client.ts
  files explicitly, documenting the constraint in a comment.
- Removed an unused node:path import in webhook/recovery.ts (main merge
  leftover) that was biome's error-level noise.

Browser bundle now builds clean. 1269 tests pass.
@danielnaab danielnaab temporarily deployed to story-71-llm-integrations-catalog April 19, 2026 18:55 Inactive
@danielnaab danielnaab merged commit 29bbc6f into main Apr 19, 2026
4 checks passed
@danielnaab danielnaab deleted the story-71/llm-integrations-catalog branch April 19, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Developer navigates LLM integrations via a screaming service layer

1 participant