mcp-data-platform-v1.81.0

v1.81.0 - First-class prompts and relevance search across every surface

This release closes the prompt-library epic (#525) and completes the "semantic search on every surface" arc. Prompts graduate from a single-owner convenience into a governed, shareable, discoverable library with a real lifecycle, and relevance ranking (vector + lexical, with an automatic lexical-only fallback) now spans captured insights, prompts, saved assets, and collections. Everything is additive and backward compatible: the five new migrations run automatically on startup, and with no embedding provider configured every new search degrades cleanly to keyword ranking.

Highlights

• Prompt library epic #525 is complete across all four phases: lifecycle and tags, admin promotion with a review queue, native user-to-user sharing, and semantic discovery.
• Relevance search everywhere: a new recall_insight tool, ranked prompt discovery, and ranked search over saved assets and collections, all reusing one shared hybrid ranking and the index-jobs embedding framework.
• Native MCP prompt visibility: prompts/list and prompts/get are now scoped per caller, matching the access model the REST API and manage_prompt already enforced.
• Two portal reliability fixes, including one that could blank the asset library on a keystroke.

---

The prompt library is now first-class (epic #525)

Prompts previously carried only an enabled flag, were globally unique by name, were visible to every MCP client regardless of owner, and "sharing" silently degraded a prompt into a markdown asset snapshot. This release fixes all of that.

Phase 1 - Lifecycle, tags, and scope-aware naming (#558)

• A draft -> approved -> deprecated -> superseded(by) lifecycle with validated transitions, mirroring the knowledge/insights state machine.
• Multi-valued tags[] (GIN-indexed) alongside the existing single category.
• Name uniqueness is now scope-aware: personal prompt names are unique per owner, while global and persona names stay globally unique. Database-backed prompts are served per caller with scope prefixes computed at request time (personal-<name>, <persona>-<name>, global-<name>).
• Lifecycle and tags are surfaced through manage_prompt, the admin API, and the portal.

Phase 2 - Admin promotion and review queue (#559)

• An owner can request promotion of a personal prompt to their persona or to global scope. This folds into the existing manage_prompt update (via requested_scope / requested_personas); there is no new tool or command.
• Admins see requests in a review queue and approve or reject. Approval sets the new scope and status=approved. Promotion to a shared scope stays admin-only; requesting it is the owner action.

Phase 3 - Native user-to-user sharing (#560)

• An owner can share a personal prompt directly with another user by email. The recipient gets a real, runnable MCP prompt (served as shared-<name>) with its arguments[] intact, plus a portal entry under Shared With Me.
• This replaces the prior degradation path that produced a markdown asset snapshot. Markdown export remains available as a separate, explicitly labeled action.
• Sharing reuses the existing portal_shares table that already backs asset and collection shares, rather than forking a prompt-specific table.

Phase 4 - Semantic discovery (#561)

• Approved, enabled prompts are embedded off the request path on the shared index-jobs framework (title + description + body + tags).
• manage_prompt list query=... (MCP) and the portal prompt-library search (GET /api/v1/portal/prompts/search) rank by similarity. Visibility and persona scoping are applied before ranking, so you can never find a prompt you cannot read.
• Editing a prompt's indexed text clears its vector so it re-embeds against the new content.

Native MCP prompt visibility (#553)

• prompts/list and prompts/get are now scoped to the calling user by a new visibility middleware, matching the REST API and manage_prompt. Previously every database-backed prompt was returned to every client and served by name, so personal and persona-scoped prompts were visible across users on the MCP surface even though the same data was correctly scoped everywhere else.

---

Relevance search across every surface

All of these reuse one weighted hybrid ranking (vector similarity fused with a lexical signal) and the same hybrid-vs-lexical decision, so no surface can drift from the others. When an embedding provider is configured they rank semantically; with none configured (or on an embedder error or a zero vector) they fall back automatically to lexical-only ranking.

• Captured insights - recall_insight (#551, closes #442). A new knowledge-toolkit tool that answers "what did we learn about X" by relevance instead of list-and-read. Insights are knowledge-dimension memory records, so it reuses the search primitives from #516 rather than adding a separate index. Owner-scoped on the caller's email.
• Prompts (#561). Covered above.
• Saved assets and collections - manage_artifact action=search and the portal (#562, closes #550, supersedes #443). Ranked search over saved assets (MCP tool and GET /api/v1/portal/assets/search) and collections (GET /api/v1/portal/collections/search), each returning a score per row and a ranking field. Collection section titles and descriptions are indexed via a denormalized, backfilled column so they feed both ranking arms. Search is scoped server-side by owner_id - the same ownership key the asset library list and the update/delete checks use - so results match exactly what the caller can see, and it fails closed without an identity.

---

UI reliability fixes

• #549 - Searching the portal asset library could throw Cannot read properties of undefined (reading 'toLowerCase') and blank the entire page when an asset had no description. The client-side filter now guards name, description, and tags.
• #552 - The portal prompt forms (view/edit, user create, admin) now validate the name field against the backend pattern up front, surface duplicate-name conflicts clearly, and use the markdown editor consistent with the rest of the portal.

---

Database migrations

All migrations run automatically on startup (golang-migrate). They are additive; existing rows are backfilled.

Migration	What it does
000059_prompts_lifecycle	Adds tags[] (GIN), status, approved_by/approved_at, deprecated_at, superseded_by to prompts; backfills existing rows to approved; scope-aware name uniqueness.
000060_prompt_promotion_request	Adds review_requested, requested_scope, requested_personas with a partial index backing the admin review queue.
000061_portal_shares_prompts	Adds prompt_id to portal_shares and updates the one-target constraint so exactly one of asset/collection/prompt is set per share.
000062_prompt_embeddings	Inline embedding / embedding_model / embedding_text_hash on prompts, plus an HNSW ANN index and a GIN full-text index.
000063_portal_asset_collection_embeddings	The same embedding columns and indexes on portal_assets and portal_collections, plus a backfilled sections_text column on collections.

Embeddings are populated off the request path by the index-jobs reconciler; they require a configured embedding provider. Until they are populated (or if no embedder is configured) the corresponding search runs lexical-only.

---

New and changed surfaces

MCP tools

• recall_insight (knowledge toolkit) - new tool, relevance search over the caller's captured insights.
• manage_prompt - list query=... semantic ranking; status and tags on create/update; promotion requests via requested_scope / requested_personas on update.
• manage_artifact - new action=search (ranked asset search with score and ranking).

REST endpoints

• GET /api/v1/portal/prompts/search
• GET /api/v1/portal/assets/search
• GET /api/v1/portal/collections/search
• Prompt share endpoints (create/list shares, list prompts shared with me) and the admin prompt-promotion review queue.

---

Behavior changes to know about

• Native MCP prompt visibility tightened (#553). prompts/list and prompts/get now return only prompts the caller may see. Clients that relied on every registered prompt being visible to every caller will see a smaller, correctly scoped set.
• Prompt sharing is now native (#560). Sharing a prompt produces a real runnable prompt for the recipient (shared-<name>), not a markdown asset. The markdown export is still available as a separate action.
• Prompt name uniqueness is now scope-aware (#558). Personal names are unique per owner rather than globally.
• Semantic ranking requires an embedding provider. Configure memory.embedding.provider: ollama (with nomic-embed-text) to enable vector ranking. Without it, prompt, insight, asset, and collection search all return correct lexical-ranked results.

Upgrade notes

• No breaking configuration changes. Migrations 000059 through 000063 apply automatically on first start; they are additive and backfill existing data.
• To light up semantic ranking, point memory.embedding.provider at an Ollama instance serving nomic-embed-text; embeddings then backfill in the background with no further action. The admin Indexing dashboard shows per-kind coverage (prompts, portal-assets, portal-collections, memory, and others).
• pgvector is required for the embedding columns and is already part of the platform's database image.

---

Changelog

Features

• #551 feat(knowledge): recall_insight tool for relevance search over captured insights (closes #442)
• #553 feat(prompts): scope native MCP prompts/list and prompts/get to the caller
• #558 feat(prompts): lifecycle + tags + per-owner naming foundation (epic #525 phase 1)
• #559 feat(prompts): admin promotion + review queue (epic #525 phase 2)
• #560 feat(prompts): native user-to-user prompt sharing (epic #525 phase 3)
• #561 feat(prompts): semantic discovery for the prompt library (epic #525 phase 4)
• #562 feat(portal+tools): semantic + lexical search over saved assets and collections (closes #550)

Bug fixes

• #549 fix(ui): guard asset search filters against undefined description
• #552 fix(ui): validate prompt name and use the markdown editor in prompt forms

Epic #525 is complete with this release.

---

Installation

Homebrew (macOS)

brew install txn2/tap/mcp-data-platform

Claude Code CLI

claude mcp add mcp-data-platform -- mcp-data-platform

Docker

docker pull ghcr.io/txn2/mcp-data-platform:v1.81.0

Verification

All release artifacts are signed with Cosign. Verify with:

cosign verify-blob --bundle mcp-data-platform_1.81.0_linux_amd64.tar.gz.sigstore.json \
  mcp-data-platform_1.81.0_linux_amd64.tar.gz