Skip to content

Fail loud on missing local-embedding peer dep + ship -local Docker image (#88)#131

Merged
jpr5 merged 3 commits into
mainfrom
fix/issue-88-local-embeddings-closeout
Jun 25, 2026
Merged

Fail loud on missing local-embedding peer dep + ship -local Docker image (#88)#131
jpr5 merged 3 commits into
mainfrom
fix/issue-88-local-embeddings-closeout

Conversation

@jpr5

@jpr5 jpr5 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Closes the local-embeddings half of #88. (The schema half shipped in v1.16.0.)

@xenova/transformers is an optional peer dep absent from the published Docker image, so provider: local failed — and failed silently in the common path. This PR makes it fail loudly and ships a supported Docker path. Three independent units; the authoritative design spec is included in the branch (docs/superpowers/specs/2026-06-25-local-embeddings-88-closeout-design.md).

Unit 1 — fail loud + fix validate semantics

  • Startup guard: serve now eagerly resolves @xenova/transformers at boot when embedding.provider === "local". If absent, the process exits non-zero immediately with an actionable message (install the peer or use ghcr.io/copilotkit/pathfinder:latest-local). Previously it booted fine and threw lazily at first embed.
  • validate semantics: the optional-dep check now routes to a new result.warnings[] field instead of result.errors[], so validate exits 0 when only warnings exist (it's a static linter; the startup guard is the real enforcement). The "Optional Dependencies" display block and the schema early-return are unchanged.
  • A single shared resolveLocalEmbeddingDep() / assertLocalEmbeddingDepForProvider() helper (with one LOCAL_EMBEDDING_DEP_MESSAGE) backs both the startup guard and validate.
  • Unit tests added: startup guard (local+absent→fail, local+present→ok, non-local+absent→ok) and validate exit-0-with-warning.

Unit 2 — -local Docker image variant

  • Dockerfile: added ARG INCLUDE_LOCAL_EMBEDDINGS=false to the prod stage; after npm ci --omit=dev, conditionally npm install @xenova/transformers only when the arg is true. Default build stays slim.
  • publish-docker.yml: the single build is now two variants in the same run from the same Dockerfile with disjoint tags — default (slim) → :latest, :<ref>; local (INCLUDE_LOCAL_EMBEDDINGS=true) → :latest-local, :<ref>-local. Both multi-arch, shared type=gha cache, one Slack notify summarizing both. The workflow_call/workflow_dispatch/push:tags triggers and tag-resolution step from Fix broken release→Docker publish pipeline (stale :latest at v1.15.0) #130 are preserved.

Unit 3 — docs

  • docs/deploy: notes :latest-local ships @xenova/transformers preinstalled and that the default image now fails loudly at startup for provider: local.
  • docs/config: cross-references the -local image vs npm install for the local provider.
  • README.md: one-line pointer to the -local image.

Local RED→GREEN proof (real surfaces)

All runs used a schema-valid provider: local config with @xenova/transformers genuinely ABSENT from node_modules.

Unit 1 — startup guard (serve)

  • RED (origin/main code, rebuilt to dist): serve boots past the embedding config with no mention of @xenova/transformers (grep count 0) and only dies later on the DB connection (ECONNREFUSED) — i.e. with a reachable DB it would boot a healthy-looking server that explodes at first embed.
  • GREEN (this branch): serve exits 1 at boot:
    [startup] fatal: Error: embedding.provider is "local" but the optional peer dependency
    @xenova/transformers is not installed. Either install it (npm install @xenova/transformers)
    or use the prebuilt image ghcr.io/copilotkit/pathfinder:latest-local, which ships it preinstalled.
    

Unit 1 — validate exit code

  • RED (origin/main): validate on the schema-valid local config prints Result: 1 optional dependency warning(s), no hard errors. yet exits 1 (the self-contradiction).
  • GREEN (this branch): same config, same printed line, exits 0.

Unit 2 — Docker variants (built locally before CI)

  • docker buildx build --build-arg INCLUDE_LOCAL_EMBEDDINGS=true --target prod -t pf:local .docker run --rm pf:local node -e "require.resolve('@xenova/transformers')…"PRESENT
  • docker buildx build --target prod -t pf:slim . → same probe → ABSENT

Test suite

Full npx vitest run (excluding **/.claude/**): 171 files / 3192 tests passed, 0 failures. Prettier --check clean, tsc build clean, tsc -p tsconfig.scripts.json clean, version-sync in sync. No as any.


A follow-up release ships the -local image and the startup guard (the matrix docker build only runs on release/dispatch, not on this PR — so PR CI here is Static Quality only). Do not merge as a release commit.

🤖 Generated with Claude Code

jpr5 added 3 commits June 25, 2026 15:56
…emantics

Issue #88 local-embeddings closeout, Unit 1.

- Add a shared resolveLocalEmbeddingDep() / assertLocalEmbeddingDepForProvider()
  helper in config.ts and a single LOCAL_EMBEDDING_DEP_MESSAGE.
- serve startup now eagerly resolves @xenova/transformers when
  embedding.provider is "local" and exits non-zero at boot with an actionable
  message (install the peer OR use ghcr.io/copilotkit/pathfinder:latest-local),
  instead of booting fine and throwing lazily at first embed.
- validate: route the optional-dep check to a new result.warnings[] field so the
  command exits 0 when only warnings exist (previously it pushed to errors[] and
  exited 1 while printing "no hard errors"). The Optional Dependencies display
  block is preserved; the schema early-return is untouched.
- Tests: startup guard (local+absent fail, local+present ok, non-local+absent ok)
  and validate exit-0-with-warning; existing validate tests updated for warnings[].

Includes the authoritative design spec.
Issue #88 local-embeddings closeout, Unit 2.

- Dockerfile prod stage: add ARG INCLUDE_LOCAL_EMBEDDINGS=false; after
  npm ci --omit=dev, conditionally npm install @xenova/transformers only when
  the arg is true. Default build stays slim.
- publish-docker.yml: build two variants in the same run from the same
  Dockerfile with disjoint tags — default (slim) -> :latest, :<ref>; local
  (arg=true) -> :latest-local, :<ref>-local. Both multi-arch, shared gha cache,
  one Slack notify summarizing both. Triggers and tag-resolution from #130
  preserved.

Verified locally: --build-arg INCLUDE_LOCAL_EMBEDDINGS=true -> require.resolve
PRESENT; default build -> ABSENT.
Issue #88 local-embeddings closeout, Unit 3.

- docs/deploy: note :latest-local ships @xenova/transformers preinstalled and
  that the default image now fails loudly at startup for provider: local.
- docs/config: cross-reference the -local image vs npm install for the local
  embedding provider.
- README: one-line pointer to the -local image for local embeddings in Docker.
@jpr5
jpr5 merged commit ec6c6bb into main Jun 25, 2026
9 checks passed
@jpr5
jpr5 deleted the fix/issue-88-local-embeddings-closeout branch June 25, 2026 23:00
jpr5 added a commit that referenced this pull request Jun 26, 2026
Removes
`docs/superpowers/specs/2026-06-25-local-embeddings-88-closeout-design.md`,
which was erroneously committed via #131. Design specs belong on Notion,
never in the repository.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant