Skip to content

feat: items 7 + 9 bridge surfaces — 8 cartridges + boj_vector + boj_multimodal#103

Merged
hyperpolymath merged 1 commit into
mainfrom
feat/items-7-9-vector-multimodal
May 20, 2026
Merged

feat: items 7 + 9 bridge surfaces — 8 cartridges + boj_vector + boj_multimodal#103
hyperpolymath merged 1 commit into
mainfrom
feat/items-7-9-vector-multimodal

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

Summary

Closes the bridge-surface portion of epic #87 items 7 and 9. Backend dispatch (Elixir/Zig) tracked separately in tracker issues #100 (vector) and #101 (multi-modal).

Eight new cartridges in two domains; two new umbrella bridge tools.

Item 7 — Vector databases (4 cartridges)

Cartridge Provider strength
`pinecone-mcp` Hosted serverless indexes; dimension + metric required at create-time
`weaviate-mcp` Hybrid (vector + BM25) search; schema-driven classes; modular vectorisers
`qdrant-mcp` Rust-native; payload filtering; sparse + dense vectors; self-host or cloud
`chromadb-mcp` Embedded or client/server; LLM-app-focused; document storage alongside vectors

All four share a 7-operation manifest: `authenticate` / `list_collections` / `create_collection` / `delete_collection` / `upsert` / `query` / `delete`. Provider-specific extras live under `params` (Pinecone namespace, Weaviate alpha-hybrid weight, Qdrant score_threshold, Chroma embedding_function).

Item 9 — Multi-modal (4 cartridges)

Cartridge Role Worker-compat?
`whisper-mcp` Speech-to-text (OpenAI API + local whisper.cpp backends) ✅ HTTP API
`elevenlabs-mcp` Text-to-speech, voice cloning (premium), multilingual ✅ HTTP API
`replicate-mcp` Image/video/audio generation; async prediction model ✅ HTTP API
`ffmpeg-mcp` Local transcoding (probe/extract/concat/trim) Local-only per ADR-0013

ffmpeg-mcp is the glue: extract audio from video → whisper transcribe; extract frames → replicate vision model; etc.

Bridge tools (2 new)

Both follow the `boj_search` umbrella pattern from PR #99:

```
boj_vector {operation, provider ∈ {pinecone|weaviate|qdrant|chromadb}, ...}
boj_multimodal {operation, provider ∈ {whisper|elevenlabs|replicate|ffmpeg}, ...}
```

Each dispatches via `invokeCartridge` to the right cartridge. `hardeningGate` requires `operation`. Domain classifications added: `vector`, `multimodal`.

Offline-menu

All 8 cartridges added to `mcp-bridge/lib/offline-menu.js` (Teranga tier) so they surface in `boj_menu` even when the REST backend is offline.

Tool count

42 (after `boj_search`) → 44 (after `boj_vector` + `boj_multimodal`). Per the umbrella-tool design choice in #100/#101, adding 8 cartridges added only 2 bridge tools — not 8.

Tests

  • ✅ 15/15 existing tests pass (no regressions)
  • ✅ `tools/list` advertises both new tools end-to-end (verified via `deno run`)

What's NOT in this PR

  • Idris2 ABI, Zig FFI, Deno adapter for any of the 8 cartridges — just the manifest + bridge wiring
  • Tests against live provider APIs — surface captured; contract testing comes with backend impl
  • Backend dispatch in the Elixir/Zig layer — per-cartridge follow-up work (~1-2 days each per the trackers)

Sequencing

This closes the mechanical portion of items 7 and 9. The trackers (#100, #101) update on merge — backends remain open for follow-up sessions.

Epic #87 status after this lands:

  • Tier A: items 1, 8, 13 done in code; items 7, 9 surfaces done (backends pending); item 14 RFC only
  • Tier B: 6/6 RFCs done
  • Tier C: items 11, 12 untouched

🤖 Generated with Claude Code

…bridge tools

Closes the bridge-surface portion of epic #87 items 7 and 9. Backend
implementations (Elixir/Zig dispatch) tracked separately in issues
#100 and #101.

== Item 7 — Vector databases (4 cartridges) ==

Surface for four providers behind a single bridge tool:

- pinecone-mcp  — hosted serverless indexes; dimension + metric required
- weaviate-mcp  — hybrid (vector + BM25) + schema-driven classes
- qdrant-mcp    — Rust-native; payload filtering; sparse + dense vectors
- chromadb-mcp  — embedded or client/server; LLM-app-focused

All four share a 7-op manifest (authenticate / list_collections /
create_collection / delete_collection / upsert / query / delete) with
provider-specific extras (Pinecone namespace, Weaviate alpha-hybrid,
Qdrant score_threshold, Chroma embedding_function).

== Item 9 — Multi-modal (4 cartridges) ==

- whisper-mcp    — STT (OpenAI API + local whisper.cpp backends)
- elevenlabs-mcp — TTS, voice cloning (premium), multilingual
- replicate-mcp  — image/video/audio generation; async prediction model
- ffmpeg-mcp     — local transcoding; **local-only**, Worker-incompatible
                   (per ADR-0013 cartridge-compat matrix)

ffmpeg-mcp is the glue: extract audio from video → whisper transcribe;
extract frames → replicate vision model; etc.

== Bridge surface ==

Two umbrella tools (mirroring search-mcp's boj_search pattern, per the
recommendation in issues #100 + #101):

- boj_vector       — provider ∈ {pinecone, weaviate, qdrant, chromadb}
- boj_multimodal   — provider ∈ {whisper, elevenlabs, replicate, ffmpeg}

Each takes {operation, provider, ...} and dispatches via
invokeCartridge to the right cartridge. hardeningGate requires
operation; domain classification: vector / multimodal.

All 8 cartridges added to offline-menu.js (Teranga tier) so they
surface in boj_menu even when the REST backend is offline.

== Tests ==

- 15/15 existing tests pass (no regressions)
- tools/list advertises boj_vector + boj_multimodal — 44 tools total
  (was 42 after boj_search; was 41 before that)

== What's NOT in this PR ==

- Idris2 ABI, Zig FFI, Deno adapter — all 8 cartridges ship as
  cartridge.json + offline-menu entry only. Backend dispatch in the
  Elixir/Zig layer is per-cartridge follow-up work (estimated 1-2
  days each per the tracker issues)
- Tests against live provider APIs — surface is captured; provider
  contract testing comes with backend implementation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 33 issues detected

Severity Count
🔴 Critical 19
🟠 High 5
🟡 Medium 9

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Stale AI session file -- delete",
    "type": "stale",
    "file": "GEMINI.md",
    "action": "delete",
    "rule_module": "root_hygiene",
    "severity": "medium"
  },
  {
    "reason": "Issue in quality.yml",
    "type": "missing_workflow",
    "file": "quality.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Issue in security-policy.yml",
    "type": "missing_workflow",
    "file": "security-policy.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/cache@v4 needs attention",
    "type": "unpinned_action",
    "file": "abi-drift.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Python file detected -- banned language",
    "type": "banned_language_file",
    "file": "/home/runner/work/boj-server/boj-server/.github/scripts/validate-eclexiaiser.py",
    "action": "flag",
    "rule_module": "cicd_rules",
    "severity": "critical"
  },
  {
    "reason": "TypeScript file detected -- banned language",
    "type": "banned_language_file",
    "file": "/home/runner/work/boj-server/boj-server/cartridges/sanctify-mcp/adapter/mod.ts",
    "action": "flag",
    "rule_module": "cicd_rules",
    "severity": "critical"
  },
  {
    "reason": "TypeScript file detected -- banned language",
    "type": "banned_language_file",
    "file": "/home/runner/work/boj-server/boj-server/cartridges/academic-workflow-mcp/adapter/mod.ts",
    "action": "flag",
    "rule_module": "cicd_rules",
    "severity": "critical"
  },
  {
    "reason": "TypeScript file detected -- banned language",
    "type": "banned_language_file",
    "file": "/home/runner/work/boj-server/boj-server/cartridges/fireflag-mcp/adapter/mod.ts",
    "action": "flag",
    "rule_module": "cicd_rules",
    "severity": "critical"
  },
  {
    "reason": "TypeScript file detected -- banned language",
    "type": "banned_language_file",
    "file": "/home/runner/work/boj-server/boj-server/cartridges/ephapax-mcp/adapter/mod.ts",
    "action": "flag",
    "rule_module": "cicd_rules",
    "severity": "critical"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath merged commit 8a8f419 into main May 20, 2026
15 of 23 checks passed
@hyperpolymath hyperpolymath deleted the feat/items-7-9-vector-multimodal branch May 20, 2026 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant