Optimizer is unavailable to Modern (2026-07-28) clients

[JAORMX]

The product gap

Any go-sdk v1.7+ client connecting to an optimizer-enabled vMCP silently receives the full raw aggregated tool set instead of find_tool/call_tool, and tools/call find_tool fails with -32603 not found.

This is default client behaviour, not an exotic configuration: go-sdk v1.7's Connect is Modern-first — it probes server/discover before initialize and upgrades to 2026-07-28 whenever the server advertises it.

First observed as e2e failures on #6033's earlier heads: vmcp_optimizer_test.go, vmcp_cli_features_test.go:485, virtualmcp_optimizer_composite_test.go:201, virtualmcp_optimizer_multibackend_test.go.

Mechanism

The optimizer is Serve-layer and session-scoped (pkg/vmcp/server/serve_optimizer.go): at session registration a per-session optimizer is built over the core's advertised set, upserting tools into a shared FTS5 store, and the session advertises exactly find_tool/call_tool in place of the raw tools. That store is transport/session state and is deliberately not in the stateless core.

The Modern dispatch path has no session: dispatchModernToolsList/dispatchModernToolCall serve straight from core.ListTools/core.CallTool, so the optimizer transformation never runs.

Interim state

#6033 adds a capability gate (pkg/vmcp/server/modern_gate.go, modernDispatchBlockers) that keeps optimizer-enabled instances Legacy-only: server/discover falls through to the SDK and advertises a Legacy-only version list — steering Modern-first clients onto the Legacy handshake with no error surfaced — and any other well-formed Modern request is refused with a conformant 400 + -32022 listing the Legacy version.

So the feature works for every client, but an optimizer-enabled instance cannot serve the Modern revision at all. This issue is that trade being made explicit rather than left silent.

What parity needs

1. An index not keyed by session. The advertised set is identity-filtered (core.ListTools(ctx, identity)), so per-identity is the natural scope — the session index today piggybacks on the session↔identity binding. An instance-scoped index over the unfiltered set with identity-filtered search results is the alternative. Either way the FTS5 store's upsert-by-tool-name semantics need a scoping key.
2. Advertisement: when the optimizer is enabled, dispatchModernToolsList returns the two meta-tools (pagination is trivial at n=2).
3. Dispatch: dispatchModernToolCall recognises find_tool/call_tool and routes through an optimizer built over the caller identity's core tools. call_tool's inner invocation must dispatch through core.CallTool so the core admission seam authorizes the inner target by its real name — the same design constraint optimizerSessionTools satisfies on the Legacy path.
4. A cost model. The session path indexes once per registration; a naive Modern implementation would rebuild per request. Needs identity-scoped caching with a TTL (cf. capabilityCacheTTL = 30s) or a shared store with per-identity filtering. When embeddings are configured each upsert does an embedding round-trip per tool, so caching is mandatory, not an optimization.

Definition of done

Delete the "optimizer" entry in modernDispatchBlockers. The gate is pinned by TestModernDispatchBlockers, TestClassifyingHandler_ModernCapabilityGate, and the full-handler pair in modern_gate_integration_test.go — removing the entry flips those tests, which then assert the new behaviour (optimizer instance advertises Modern; Modern tools/list returns the meta-tools). The e2e optimizer suites then cover Modern clients end-to-end.

Refs #6033, #5959, #5743.