🚀 Toolhive v0.41.0 is live!
This release delivers first-class support for the new MCP 2026-07-28 ("Modern") specification revision across every ToolHive surface — the transport proxies, the transparent proxy, and Virtual MCP now recognize, serve, and bridge both the session-based 2025-11-25 revision and the new stateless revision, including mixed client×backend combinations. It also ships a reproducible project-skills workflow (thv skill sync/upgrade with a lock file and Sigstore groundwork), an opt-in Envoy network-isolation backend, and RFC 8693 token exchange with full delegation-chain auditing.
⚠️ Breaking Changes
- Storage version migrator is now enabled by default in the operator Helm chart — namespace-scoped installs (
operator.rbac.scope=namespace) failhelm upgradeat render time unless they setoperator.features.storageVersionMigrator: false; cluster-scoped installs need no action (#5603) - JSON-RPC batch requests are now rejected — top-level arrays get HTTP 400 /
-32600instead of being executed; batches previously bypassed authorization, tool filtering, and audit, and MCP removed batching in 2025-06-18, so send individual requests (#5931) - Rate-limit JSON-RPC error code moved from
-32029to429— the MCP 2026-07-28 spec reserves-32020..-32099; clients branching onerror.code == -32029must match429(the HTTP 429 status,Retry-Afterheader, anddata.retryAfterSecondsare unchanged) (#6120)
Migration guide: storage version migrator default
The chart now enables the StorageVersionMigrator controller by default (operator.features.storageVersionMigrator: true), and a new chart validation rejects that combination with operator.rbac.scope=namespace — the controller cannot sync its cluster-scoped CRD informer under namespace RBAC. Affected users see helm install/helm upgrade fail with:
operator.features.storageVersionMigrator requires operator.rbac.scope=cluster
Cluster-scoped installs (the chart default) need no action: the operator pod restarts once with the migrator enabled and begins automatically trimming status.storedVersions on ToolHive CRDs — the precondition for a future release to drop deprecated API versions. No new pods or RBAC objects are created. To opt out anyway, set operator.features.storageVersionMigrator: false.
Namespace-scoped installs must opt out explicitly:
Before
operator:
rbac:
scope: namespaceAfter
operator:
rbac:
scope: namespace
features:
storageVersionMigrator: falseMigration steps
- Check whether you are affected:
helm get values <release> -n <ns>— you are affected ifoperator.rbac.scopeisnamespaceandoperator.features.storageVersionMigratoris unset ortrue. - Add
operator.features.storageVersionMigrator: falseto your values (or pass--set operator.features.storageVersionMigrator=false). - Run
helm upgradeas usual. - Since namespace-scoped installs cannot run the migrator, plan to clean CRD
status.storedVersionsby other means (e.g. a one-off run of kube-storage-version-migrator) before any future release drops a deprecated CRD version. Seedocs/operator/storage-version-migration.md.
PR: #5603
Migration guide: JSON-RPC batch rejection and rate-limit error code
Batch requests (#5931) — affects only clients sending JSON-RPC batches (removed from MCP in 2025-06-18); no conformant 2025-11-25 or 2026-07-28 client emits them.
Before
[{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{...}},
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{...}}]After
Each request must be its own POST; a batch now returns:
{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"Invalid Request: batch requests are not supported"}}Rate-limit error code (#6120) — affects only clients branching on the JSON-RPC error.code for backoff.
Before
{"jsonrpc":"2.0","error":{"code":-32029,"message":"rate limited","data":{"retryAfterSeconds":30}}}After
{"jsonrpc":"2.0","error":{"code":429,"message":"rate limited","data":{"retryAfterSeconds":30}}}Migration steps
- Replace any JSON-RPC batch sends with sequential or concurrent single requests.
- Update rate-limit handling to match
error.code == 429— or, better, key off the HTTP 429 status ordata.retryAfterSeconds, which are stable across versions.
Upgrade notes (not breaking, worth knowing)
- HMAC secrets are now used byte-for-byte (#6067): if your
HMACSecretRefssecret file carried leading/trailing whitespace bytes, previously-minted authorization codes and refresh tokens fail validation once after upgrade and clients re-authenticate; ensure the mounted secret is exactly the raw random bytes (no trailing newline). - Denial/error responses are now spec-conformant JSON-RPC (#5944, #6055, #6066, #6068): correct envelope, standard error codes instead of HTTP statuses (500→
-32603),idomitted rather thannull, generic denial messages, and filtered tool calls answered with-32602over HTTP 200 instead of a bodyless 400 — monitoring or scripts keyed to the old malformed shapes need updating.
🆕 New Features
MCP 2026-07-28 ("Modern") spec support
- ToolHive now speaks the MCP 2026-07-28 stateless revision end to end, negotiating both revisions per backend via go-sdk v1.7 (#5993)
- The streamable and transparent proxies classify and serve Modern stateless requests alongside Legacy session traffic (#5839, #5884)
- Virtual MCP serves Modern clients directly (#5953), classifies request revisions at the client edge (#5913), and resolves each backend's revision independently (885e8b1)
- Era-mismatched client×backend combinations are bridged in vMCP, so a Modern client can use Legacy backends and vice versa (#6006)
- Modern client-facing dispatch is complete with listen-stream support and pagination (#6050), gated per capability instead of a global kill-switch (#6033)
- vMCP falls back to Legacy when a backend's revision probe is inconclusive, keeping health checks and aggregation resilient (#6001)
- Mid-call elicitation/sampling refusals to Modern clients are classified per spec with the new error codes (#6061), and the typed
input_requiredseam lands as groundwork for MRTR (SEP-2322) (#6074) - Reserved
io.modelcontextprotocol/*keys are stripped from backend response_metaso backends cannot spoof protocol metadata (#6024) - W3C trace context now propagates through outbound MCP
_meta(SEP-414), joining backend spans to the client→proxy→server trace (#5964) - Tool definitions carrying invalid
x-mcp-headerannotations (SEP-2243) are rejected as the spec requires (#6013) - The readiness probe sends the current MCP protocol version instead of a hardcoded 2024-11-05 (#5940)
- Opt-in strict
MCP-Protocol-Versionheader validation for the streamable proxy (#5957)
Virtual MCP conformance and aggregation
- vMCP is now MCP-conformant: completions, resource templates, subscriptions, and mid-call server→client forwarding all work (#5875)
- The streamable HTTP proxy supports GET listen streams and routes server-to-client messages per session, so progress, sampling, and elicitation reach clients (#5934)
- Backend
list_changednotifications are consumed and propagated to clients for tools (#5965) and for resources and prompts (#5971) - The aggregator resolves cross-backend name conflicts for resources, resource templates, and prompts, not just tools (#6075), and drops ambiguous prompt names instead of failing aggregation (#6099)
- tools/list pagination completeness is guaranteed for >1000-tool aggregated sets (#6021)
Reproducible project skills
- Project-scoped skill installs are pinned in a
toolhive.lock.yamllock file (#5892, #5893, #5894) thv skill syncrestores a project's pinned skill set on any machine and verifies on-disk content in CI (#5895)thv skill upgradere-resolves pinned skills to newer content without silent lock drift (#5896)- Sync and upgrade gain typed exit codes and a pre-install confirmation gate for scriptable CI use (#5897)
- Sigstore signing groundwork for skill OCI artifacts: provenance fields in the lock schema (#6015), a signer package (#6023), and commit-signature plumbing with bundle storage (#6084)
Authentication and token exchange
- RFC 8693 token exchange is wired into the embedded authorization server's token endpoint (#5881), with delegated token audience bounded by the subject token (#5882)
- Subject tokens from trusted external OIDC issuers (Keycloak, Entra, Okta) can be validated for token exchange (#5995)
- Audit logs capture the RFC 8693
actclaim and full delegation chain, so "agent X acting for Alice" is distinguishable from Alice (#6046, refined in #6096 and adopted as the toolhive-core canonical schema in #6107) - Malformed prior
actclaims are rejected at token exchange per RFC 8693 §4.1 (#6114) - Dynamically registered OAuth clients renew expiring client secrets automatically (RFC 7591/7592) (#5377)
- Multi-valued JWT claims can be normalized to canonical space-delimited form for Cedar policies, unifying Entra/Keycloak/Okta scope shapes (#5880)
- Authentication failures, webhook denials, and rate-limit rejections now produce audit events (#5874, #6005)
Network isolation
- An opt-in Envoy network-proxy backend (
TOOLHIVE_NETWORK_PROXY=envoy) consolidates egress and ingress into one container (#5907, seam extracted in #5906) AllowPortpermissions translate into Envoy egress RBAC policy, matching Squid's port ACLs (#5927)- The Envoy backend is hardened with a digest-pinned image and disabled admin interface (#5949), fixed for Linux Docker Engine readiness (#5926), and covered by e2e tests (#5918)
Other
thv runandthv buildgain a repeatable--build-withflag to constrain build-time dependencies (e.g.--build-with 'mcp<2'foruvx://packages) (#6111, #6116)- The operator Helm chart gains an opt-in image-discovery Deployment so air-gapped installs can statically discover runtime images (#6047)
🐛 Bug Fixes
- The authz response filter can no longer be bypassed via media-type or status-code tricks by a malicious backend (#6092)
- SSE responses are parsed per event rather than per line, closing a tool-list smuggling vector (#6088); filter failures are framed as proper SSE events instead of being silently ignored (#6087); and raw SSE buffers are no longer spliced into error text (#6079)
- Cedar authorization no longer denies everything when an upstream IdP issues JWT access tokens without referenced identity claims like
email(#6022) - VirtualMCPServer now honors
MCPOIDCConfig.caBundleRef, fixing OIDC discovery against self-signed issuers (#4923) - Denial and error paths emit conformant JSON-RPC 2.0 with standard error codes, scrubbed error text, and omitted (not null) ids (#6055, #6066, #6068, #6086)
- Filtered tool calls return a JSON-RPC error over HTTP 200 instead of a bodyless HTTP 400 (#5944), and session-not-found 404s echo the request id (#6031)
- Streamable-http SSE frames carry
event: message, unbreaking spec-lenient clients such as@ai-sdk/mcp(#5954) - Binary HMAC secrets are used as-is instead of being whitespace-trimmed (#6067)
- Digest-pinned image pulls (
tag@digest) no longer fail at container create (#5978) thv listno longer poisons workload status when transient Docker errors occur (#6076), the ingress proxy port is no longer derived from a fixed upstream port (#6069), and concurrent isolated-workload startups no longer race on network creation and ingress DNS (#6071, #6083)- Ingress Squid warms upstream connections at startup, removing cold-start latency skew (#5887)
- TUF outages no longer hard-fail
thv runin warn mode (#6063) - Backend tool schemas keep top-level
oneOf/anyOf/enumkeywords through vMCP ingestion (#5990)
🧹 Misc
- New architecture documentation for vMCP backend revision classification and the negotiate-down rationale (#6016, #6017, #6025)
- Extensive regression pins guarding the mcp-go → go-sdk migration: per-session security, TTL/DNS-rebind/bridge/schema behaviors, and Redis-backed mixed-era coverage (#5974, #5977, #5981, #6056)
- Large e2e reliability and speed investment: parallel Ginkgo procs, reliable readiness waits, and deflaked forwarding/group/dual-era suites (#6070, #6012, #6063, and others)
- Deterministic swagger docgen and regenerated OpenAPI docs (9ef27fa, #5970)
- Shared source-dispatch skeleton for skill sources and preserved inner error codes in the skills API (#5967, #5966)
- WorkflowAuditor event prologue extracted to remove eight-way duplication (#6122)
📦 Dependencies
| Module | Version |
|---|---|
google.golang.org/grpc |
v1.82.1 (security) |
github.com/klauspost/compress |
v1.18.7 (security, #6041) |
github.com/stacklok/toolhive-catalog |
v0.20260727.0 |
actions/checkout |
digest d23441a |
github/codeql-action |
digest e4fba86 |
golang/govulncheck-action |
digest 032d455 |
anthropics/claude-code-action |
digest be7b93b |
👋 Welcome to our newest contributors: @amir-rezaei and @stantheman0128 🎉
Full commit log
What's Changed
- Add stateless routing to streamable MCP proxy by @jhrozek in #5839
- Normalize opt-in multi-valued JWT claims for Cedar by @jhrozek in #5880
- Bound delegated token audience by subject token by @jhrozek in #5882
- Wire token exchange handler into fosite by @jhrozek in #5881
- Implement Client Secret Expiry and Renewal for Dynamic Client Registration by @Sanskarzz in #5377
- Serve MCP 2026-07-28 stateless requests through the transparent proxy by @jhrozek in #5884
- Warm ingress Squid peer connections with standby=2 by @jhrozek in #5887
- Ignore flaky tools-call-sampling in conformance CI by @jhrozek in #5888
- Make vMCP MCP-conformant: completions, templates, subscribe, forwarding by @JAORMX in #5875
- Extract a swappable networkProxy seam from deployOps by @ChrisJBurns in #5906
- Isolate HTTP transport in vmcp backend tests by @JAORMX in #5919
- Add Envoy network-proxy backend behind TOOLHIVE_NETWORK_PROXY=envoy by @ChrisJBurns in #5907
- Add Envoy network-isolation e2e tests by @ChrisJBurns in #5918
- Default storage version migrator to enabled in operator chart by @ChrisJBurns in #5603
- Wire MCP 2026-07-28 revision classification into vMCP by @jhrozek in #5913
- Reject JSON-RPC batches to close authz blind spot by @JAORMX in #5931
- Update batch e2e test to expect rejection by @JAORMX in #5933
- Add project-level skills lock file schema package by @samuv in #5892
- Fix Envoy isolated-server readiness and enable e2e tests by @ChrisJBurns in #5926
- Deflake vMCP forwarding notification wait by @JAORMX in #5941
- Send current MCP protocol version in readiness probe by @JAORMX in #5940
- Translate AllowPort into Envoy egress RBAC policy by @ChrisJBurns in #5927
- Route server-to-client MCP messages per session by @JAORMX in #5934
- Return JSON-RPC error for filtered tool calls by @JAORMX in #5944
- Update standalone-SSE test for filtered-tool error by @JAORMX in #5958
- Add opt-in strict MCP-Protocol-Version validation by @JAORMX in #5957
- Add SkillLockService interface and managed install flag by @samuv in #5893
- Drain backend notifications before per-call client close by @JAORMX in #5960
- Deflake real-backend forwarding tests by @JAORMX in #5963
- Record project-scope skill installs in the lock file by @samuv in #5894
- Add thv skill sync: restore project skills from the lock file by @samuv in #5895
- Propagate W3C trace context in outbound MCP _meta by @JAORMX in #5964
- Add thv skill upgrade: re-resolve pinned skills to newer content by @samuv in #5896
- Add typed exit codes and confirmation gate to skill sync/upgrade by @samuv in #5897
- Consume backend list_changed in vMCP and propagate to clients by @JAORMX in #5965
- Regenerate swagger docs to drop duplicate enums by @JAORMX in #5970
- Propagate MCPOIDCConfig CA bundle to vmcp OIDC config by @ChrisJBurns in #4923
- Harden Envoy backend: pin image digest, disable admin interface by @ChrisJBurns in #5949
- Serve MCP 2026-07-28 Modern stateless requests through vMCP by @jhrozek in #5953
- Propagate backend resources/prompts list_changed in vMCP by @JAORMX in #5971
- Add vMCP per-session security regression tests by @JAORMX in #5974
- Fix digest-pinned image pulls failing at container create by @ChrisJBurns in #5978
- Add regression pins for TTL, DNS-rebind, bridge, schema by @JAORMX in #5977
- Update module google.golang.org/grpc to v1.82.1 [SECURITY] by @renovate[bot] in #5943
- Update golang/govulncheck-action digest to 032d455 by @renovate[bot] in #5869
- Update anthropics/claude-code-action digest to be7b93b by @renovate[bot] in #5983
- Make swagger docgen deterministic (dedupe swag's doubled enums) by @jhrozek in #5987
- Handle per-backend MCP revision in vMCP (client-side dual protocol) by @jhrozek in #5980
- Bump toolhive-core for tool-schema ingestion fix by @JAORMX in #5990
- Unblock main CI: fix vMCP nil-registry panic and CRD docs drift by @jhrozek in #5999
- Fall back to Legacy on an inconclusive vMCP revision probe by @jhrozek in #6001
- Preserve inner error codes through the skills lock-state wrap by @samuv in #5966
- Extract shared source-dispatch skeleton for skill sources by @samuv in #5967
- Add live e2e tests for dual-era stateless proxy behavior by @jhrozek in #5981
- Validate subject tokens from external OIDC issuers by @jhrozek in #5995
- Support MCP 2026-07-28 spec (go-sdk v1.7 via toolhive-core) by @JAORMX in #5993
- Unify e2e yardstick backend on 1.2.0 by @jhrozek in #6004
- Update module github.com/stacklok/toolhive-catalog to v0.20260727.0 by @renovate[bot] in #6003
- Audit authentication failures and webhook denials by @JAORMX in #5874
- Audit stream opens on flush and pre-write panics by @JAORMX in #6005
- Pin forwarding tools/call against a stale-Legacy stateless backend by @JAORMX in #6010
- Make e2e workload readiness waits reliable by @JAORMX in #6012
- Add provenance and unsigned fields to the skills lock schema by @samuv in #6015
- Reject backend tools with invalid x-mcp-header by @JAORMX in #6013
- Document vMCP backend revision classification by @JAORMX in #6016
- Consolidate the negotiate-down rationale by @JAORMX in #6017
- Strip reserved io.modelcontextprotocol/* keys from responses by @jhrozek in #6024
- Bridge era-mismatched client×backend cells in vMCP by @jhrozek in #6006
- Correct two references in the canonical rationale by @JAORMX in #6025
- Pin tools/list pagination completeness for >1000-tool vMCP sets by @JAORMX in #6021
- Fix bridge_regression_test build after reserved-meta API rename by @JAORMX in #6029
- Fix Cedar deny-all when upstream JWT lacks identity claims by @JAORMX in #6022
- Bump klauspost/compress to v1.18.7 by @JAORMX in #6041
- Fix contradictory upstream ID token exp contract by @JAORMX in #6036
- Echo the request JSON-RPC id in session-not-found 404s by @JAORMX in #6031
- Bind the vMCP test server's port directly by @JAORMX in #6044
- Harden group e2e workload naming and waits by @JAORMX in #6043
- Speed up e2e(core) suite with Ginkgo procs and concurrent waits by @ChrisJBurns in #6070
- Stop deriving the ingress proxy port from a fixed upstream port by @jhrozek in #6069
- Fix concurrent isolated-startup races (network create + ingress DNS latch) by @ChrisJBurns in #6071
- Stop trimming binary HMAC secrets before use by @jhrozek in #6067
- Classify Modern mid-call capability refusals per spec by @JAORMX in #6061
- Gate dual-era e2e on the full replica count by @jhrozek in #6057
- Omit absent JSON-RPC ids instead of emitting null by @jhrozek in #6068
- Deflake CI: TUF warn mode, readiness diagnostics, test races by @JAORMX in #6063
- Make forwarding tests' Legacy dependency explicit by @JAORMX in #6051
- Complete Modern client-facing dispatch: listen + pagination by @JAORMX in #6050
- Update github/codeql-action digest to e4fba86 by @renovate[bot] in #5988
- Update actions/checkout digest to d23441a by @renovate[bot] in #5982
- Add Redis-backed mixed-era vMCP e2e coverage by @jhrozek in #6056
- Emit conformant JSON-RPC on denial and error paths by @jhrozek in #6055
- Use standard JSON-RPC error codes and scrub leaked error text by @jhrozek in #6066
- Stop splicing whole SSE buffers into error text by @jhrozek in #6079
- Add Sigstore signer package for skill OCI artifacts by @samuv in #6023
- Fix workload status poisoning and deflake Envoy AllowPort by @JAORMX in #6076
- Recreate the shared external network on a lost teardown race by @jhrozek in #6083
- Retry dual-era backend startup on a lost port-bind race by @jhrozek in #6080
- Capture RFC 8693 act claim in audit logs by @JAORMX in #6046
- Share one JSON-RPC error writer across HTTP denial paths by @JAORMX in #6086
- Close authz response filter bypasses via media type and status by @JAORMX in #6092
- Resolve aggregator conflicts for resources, templates and prompts by @JAORMX in #6075
- Frame SSE filter-failure errors as SSE events by @jhrozek in #6087
- Design MRTR for vMCP and land the typed input_required seam by @JAORMX in #6074
- Drop ambiguous prompt names instead of failing aggregation by @JAORMX in #6099
- Fix review findings in the act claim audit capture by @jhrozek in #6096
- operator helm: add opt-in image-discovery Deployment for offline installs by @reyortiz3 in #6047
- Replace vMCP Modern kill-switch with a capability gate by @JAORMX in #6033
- Parse SSE responses per event, not per line by @jhrozek in #6088
- docs: fix missing trailing periods in CONTRIBUTING.md by @amir-rezaei in #5961
- test(e2e): quarantine uvx arxiv test broken by Python mcp 2.0.0 by @JAORMX in #6109
- Adopt toolhive-core canonical delegation-chain schema by @JAORMX in #6107
- Deflake TestWorkflowEngine_ParallelExecution by @ChrisJBurns in #6112
- Add --uv-with constraint support for uvx:// builds; unskip arxiv e2e by @JAORMX in #6111
- Reject malformed prior act claims at token exchange by @jhrozek in #6114
- Emit event: message on streamable-http SSE frames by @stantheman0128 in #5954
- Rename --uv-with to --build-with; reject constraints on unsupported ecosystems by @JAORMX in #6116
- Add commit-signature plumbing and sigstore bundle storage by @samuv in #6084
- Reallocate rate-limit error code to 429 by @JAORMX in #6120
- Extract shared event prologue in WorkflowAuditor by @JAORMX in #6122
- Release v0.41.0 by @toolhive-release-app[bot] in #6125
New Contributors
- @amir-rezaei made their first contribution in #5961
- @stantheman0128 made their first contribution in #5954
Full Changelog: v0.40.1...v0.41.0
🔗 Full changelog: v0.40.1...v0.41.0