v1.3.2
[1.3.2] — 2026-07-21
Disclosure and durability. Upstream error text is filtered before it leaves the
process, a crashed MCP subprocess is now noticed instead of being advertised as
healthy, admin config changes and their audit entries are recorded atomically,
and MCP gains the metrics needed to alert on a server that never came up.
Default behaviour is unchanged. No request that was accepted is now refused, no
response shape changes, no status code changes, and no configuration becomes
required. The one opt-in is mcp_servers[].required, which defaults to false.
Fixed
- Provider error text could carry a credential to clients, logs, streams, and
observability backends. A provider controls the body of its own error
responses, and some echo the value of theAuthorizationorapi-keyheader
they were sent. That text became the error message the gateway reported, so
the gateway's own credential could reach an end user, a log aggregator, an SSE
frame, or an exporter. Error messages are now filtered at the points every
caller passes through — the OpenAI-compatible error writer, the admin error
writer, the streaming error frame, the request-failure event, and the failure
log lines. Only the message text changes; status,type, andcodeare
written exactly as before. Filtering is best-effort pattern matching and
covers the key formats that carry a recognisable prefix; a provider whose keys
have no distinguishing shape cannot be matched, so it reduces exposure rather
than eliminating it. - A crashed MCP stdio subprocess was never noticed. A server was marked ready
once and nothing ever cleared it, so after its subprocess died the gateway kept
advertising that server's tools to the model, kept resolving them to a dead
transport, and failed every call. Child exit is now detected two ways — the
subprocess closing its error stream, and a tool call failing with a closed
transport or a broken pipe — and the server's tools are withdrawn from the
model until it comes back. A server closing its error stream is confirmed with
a ping before anything is withdrawn, so one that does so while still running
keeps serving, and a server that closes it at startup is still covered later.
Detection applies to stdio servers; an HTTP server that becomes unreachable
after a successful handshake is not currently detected. Recovery on the next
configuration reload is unchanged. - An MCP server that could not be built was missing from readiness. A server
whoseheadersorenvreferenced an undefined variable was resolved before
its transport existed, so it never reached the registry that readiness reports
from. It was absent frommcp_serversrather than listed as down, and one
markedrequiredleft/readyzanswering ready while the server it depends on
had never been attempted. Such a server is now reported unready with its
requiredflag intact, and reads 0 ongateway_mcp_server_uprather than
being absent from/metrics. - A configuration reload could leave
gateway_mcp_server_upreading down for a
server that was up. The gauge is labelled by server name, so the retiring
registry and its replacement wrote the same series; a reload with a request
still in flight let the old one publish 0 after the new one had already
reported the server ready, and nothing wrote 1 again. Teardown now writes only
for servers the retiring registry still owns, and drops the series for a server
that has left the configuration instead of pinning it at 0 forever. - An MCP tool that reported failure was recorded as a success. Servers signal
a failed call by returning an error result rather than a transport error. That
signal was not read, so a failing tool incremented the success counter, was
audited as successful, and produced a successful span. It is now recorded as an
error in all three. What the model and the client receive is unchanged. - Admin config changes and their audit entries could disagree. Applying a
config and recording its history entry were separate steps under separate
locks, so two concurrent admin writes could interleave and leave the newest
history entry describing a config that was not the active one — with a rollback
then targeting the wrong version. The same gap existed one level lower, between
persisting a config and applying it, where the disagreement survived a restart.
Each admin config mutation is now applied and recorded as one serialized
operation. History reads are unaffected and never wait on a config apply. - An unknown MCP tool name became an unbounded metric label. The tool name on
the unknown-tool counter came from the model's output, so an invented name
minted a new time series. Names that do not resolve to a registered tool now
collapse to a single bounded label, matching how unroutable model names are
already handled.
Added
mcp_servers[].required— marks an MCP server whose availability gates
/readyz. Defaults tofalse, so an existing configuration behaves exactly as
before. See Upgrade notes.- MCP server state on
/readyz. The readiness body now reports each
configured MCP server's name, readiness, and whether it is required, so MCP
health is observable without gating on it. The failure reason is deliberately
omitted — the endpoint is unauthenticated and a failure can quote a server URL,
an authorization header, or a subprocess command line. It is logged instead. gateway_mcp_server_up— per-server gauge,1when a server completed its
handshake and its transport is alive,0otherwise.gateway_mcp_server_init_failures_total— per-server counter incremented
when a server fails to initialize. Initialization failures were previously only
logged, which left a fully dead MCP fleet indistinguishable from an idle one.
Alert on any non-zero value.- Tracing for MCP startup. Server initialization and tool discovery now emit
spans, so the phase where MCP most often fails is visible in a trace rather
than only in logs.
Changed
- The stored config history query now reads a bounded number of the most recent
entries instead of the whole table. This is a read bound only; no history row
is ever deleted.
Upgrade notes
/readyzbehaviour is unchanged unless you opt in. Setting
required: trueon an MCP server means that if the server is unavailable the
instance reports not-ready and an orchestrator will take it out of rotation —
stopping all traffic through it, including requests that use no tools. Set it
only for a server the deployment genuinely cannot serve without.- Deployments that already configure
mcp_serverswill see a newmcp_servers
array in the/readyzresponse body. It is additive; existing fields are
unchanged. - Error messages returned to clients, written to logs, and sent to observability
exporters are now filtered. Text that matches a credential pattern is masked,
so tooling that matches on exact upstream error strings may need adjusting. - MCP tool calls that return an error result now increment
ferrogw_mcp_tool_calls_totalwithstatus="error"rather thanstatus="ok".
Dashboards that treated the MCP success rate as near-100% will show the real
rate. Label sets on existing metrics are unchanged.
Contributors
Thanks to everyone who shipped this release:
- fix: disclosure, durability, and MCP readiness (v1.3.2) (#359) — @MitulShah1
Full changelog: https://github.com/ferro-labs/ai-gateway/blob/v1.3.2/CHANGELOG.md