mcp-data-platform-v1.61.11

Highlights

This release replaces the api-gateway toolkit's in-process embedding cache and background warmer with a persisted pgvector store. Per-operation vectors are now computed once at spec-upsert time, reused across every connection that mounts the same catalog, and survive process restart.

Before this change, every connection registration triggered a fresh warm-up pass through the embedding provider. N connections against the same catalog produced N identical embedding passes against the same operations. A pod restart threw all vectors away and re-warmed everything. The first api_list_endpoints(ranking=semantic) call after a restart often timed out on cold-start cost.

After this change, vectors live in the new api_catalog_operation_embeddings table. The admin handler computes them at spec write; the toolkit reads them at connection registration. Two connections that mount the same catalog share one vector set. A restarted pod re-reads the same rows and returns ranked semantic results on the first call without a warm-up window.

Closes #419.

What's new

Persisted operation embeddings (pgvector)

A new migration (000044_api_catalog_operation_embeddings) creates a vector(768) column keyed on (catalog_id, spec_name, operation_id), with ON DELETE CASCADE from api_catalog_specs. Each row carries a SHA-256 hash of the operation's source text plus the model identifier so a spec refresh that reuses operation text can skip the embed call for unchanged rows, and a model swap forces a clean re-embed (the dedup predicate compares text hash, dimension, and model identity together).

POST /api/v1/admin/api-catalogs/{id}/specs/{spec}/reembed

New admin endpoint that wipes and recomputes vectors for one spec. Use it when:

• The spec was written before an embedding provider was configured.
• A transient embedding-provider outage left the spec without vectors.
• An operator wants to refresh vectors after swapping providers.

The portal surfaces this as a Re-embed button next to each spec row. The catalog panel also shows an embeddings: N indexed badge (green) or not indexed (amber) so operators can tell at a glance which specs back semantic ranking.

Cleaner toolkit internals

The api-gateway toolkit no longer carries a warmer goroutine, a multi-state in-flight readiness check, or panic-recovery scaffolding around the embed cache. checkEmbeddingsReady is now a row-existence check on the pre-loaded vector map. Three of the previous five embedding-state sentinels are gone (errEmbeddingsWarming, errEmbeddingsNotStarted, errEmbeddingsFailed); a new errEmbeddingsNotIndexed covers the "spec was written without an embedder" case.

For operations without an explicit operationId, the synthesized id now uses the spec-relative rawPath rather than the basePath-prefixed runtime path. This is what lets a single set of stored vectors serve every connection regardless of which connBaseURL it resolves to.

Upgrade notes

• Migration 000044 runs automatically when the platform boots against a Postgres DB. It requires the vector extension, which migration 000031 (memory records) already installs. The new migration re-runs CREATE EXTENSION IF NOT EXISTS vector defensively so deployments that disable the memory layer still work.
• Existing catalogs are not auto-embedded. Specs written before this release have no vectors in the new table. Semantic and hybrid ranking on api_list_endpoints fall back to lexical with an errEmbeddingsNotIndexed note until the operator does one of: re-save the spec (any edit triggers an embed pass), or call the new re-embed admin endpoint, or click Re-embed in the portal.
• No embedding provider configured? Spec writes still succeed; the embedding table just stays empty for those specs. The ranking path falls back to lexical and prints a clear note. Wiring an Ollama provider via embedding.ollama.url and re-embedding fills in vectors at any time.
• Multi-connection deployments benefit immediately. If you have two or more api-kind connections pointing at the same catalog (sandbox vs prod, multiple tenants), the embedding provider load drops by a factor of N after this release. Verify by re-saving one spec and watching the embedding provider's request count.
• No breaking changes to the model-facing surface. api_list_endpoints, api_get_endpoint_schema, and api_invoke_endpoint keep the same input and output shapes. The only externally visible change in OperationSummary.OperationID is for ops without an explicit operationId: the synthesized id no longer carries the spec's base path. Specs that supply operation IDs (the common case) are unaffected.

Installation

Homebrew (macOS)

brew install txn2/tap/mcp-data-platform

Claude Code CLI

claude mcp add mcp-data-platform -- mcp-data-platform

Docker

docker pull ghcr.io/txn2/mcp-data-platform:v1.61.11

Verification

All release artifacts are signed with Cosign. Verify with:

cosign verify-blob --bundle mcp-data-platform_1.61.11_linux_amd64.tar.gz.sigstore.json \
  mcp-data-platform_1.61.11_linux_amd64.tar.gz

Commits

• fd2f6b5: feat(apigateway): persist operation embeddings in pgvector, keyed on spec (#420) (@cjimti)