otel: suppress /livez and /readyz spans, fix tools/call span name#44
otel: suppress /livez and /readyz spans, fix tools/call span name#44
Conversation
Health-check probe endpoints (/livez, /readyz) are called at high
frequency by Kubernetes and generate noisy, low-value spans in the
tracing backend. Add an otelhttp.WithFilter predicate to exclude those
two paths from trace creation.
Also update mcpOTelMiddleware to name MCP spans following the semconv
recommendation:
{mcp.method.name} {target}
where target is gen_ai.tool.name when available. For tools/call this
means spans are now named e.g. "tools/call search_projects" instead of
just "tools/call", making Datadog flame charts immediately readable
without having to expand span attributes.
Ref: https://opentelemetry.io/docs/specs/semconv/gen-ai/mcp/#server
🤖 Generated with [GitHub Copilot](https://github.com/features/copilot) (via Zed)
Signed-off-by: Eric Searcy <eric@linuxfoundation.org>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
There was a problem hiding this comment.
Pull request overview
Improves OpenTelemetry instrumentation in the MCP server by reducing trace noise from Kubernetes health probes and making tools/call spans more informative/readable in tracing backends.
Changes:
- Suppress tracing for high-frequency
/livezand/readyzHTTP endpoints viaotelhttp.WithFilter. - Update MCP middleware span naming for
tools/callto include the tool name (e.g.,tools/call search_projects) per MCP semconv guidance.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| spanName := method | ||
| if method == "tools/call" { | ||
| if params, ok := req.GetParams().(*mcp.CallToolParamsRaw); ok && params.Name != "" { | ||
| spanName = method + " " + params.Name | ||
| } | ||
| } |
There was a problem hiding this comment.
The tool name is extracted from req.GetParams() here and then extracted again later when setting gen_ai.tool.name. Consider extracting params/toolName once inside the method == "tools/call" branch and reusing it for both the span name and span attributes to avoid duplicated type assertions and keep the logic in sync.
| spanName := method | |
| if method == "tools/call" { | |
| if params, ok := req.GetParams().(*mcp.CallToolParamsRaw); ok && params.Name != "" { | |
| spanName = method + " " + params.Name | |
| } | |
| } | |
| toolName := "" | |
| if method == "tools/call" { | |
| if params, ok := req.GetParams().(*mcp.CallToolParamsRaw); ok && params.Name != "" { | |
| toolName = params.Name | |
| } | |
| } | |
| spanName := method | |
| if toolName != "" { | |
| spanName = method + " " + toolName | |
| } |
Summary
Two related observability improvements to the OTel instrumentation.
ARCH-374 — Suppress health-check probe spans
Kubernetes health-check probes (
/livez,/readyz) are called at highfrequency and generate noisy, low-value spans in the tracing backend.
An
otelhttp.WithFilterpredicate is added to exclude those two pathsfrom trace creation entirely.
Bonus — Fix
tools/callspan nameThe MCP semconv spec recommends span names in the format:
where
targetshould matchgen_ai.tool.namewhen applicable.Previously all
tools/callinvocations shared the same span name(
"tools/call"), meaning Datadog flame charts required expandingattributes to see which tool was actually called.
mcpOTelMiddlewarenow sets the span name to"tools/call <tool_name>"(e.g."tools/call search_projects"),making flame charts immediately readable.
Ref: https://opentelemetry.io/docs/specs/semconv/gen-ai/mcp/#server
Jira: ARCH-374