example/shoppingcart: server spans are named POST / and GET /, not after the route, because MaxBytesHandler hands openapi a copy of the request

[Zaba505]

What happens

In the API walkthrough, example/shoppingcart, the server spans of both services are named after the method and /, not after the operation's route. An add is POST / on the cart, and the catalog's answer is GET /. ServeHTTP in openapi/v3 says the spans are named after the route, such as GET /carts/{cartId}, and httpserver's doc says the same when the handler sets Request.Pattern.

This was found while writing the walkthrough's observing page in #1120 (PR #1240). The page documents the spans as the code names them, and its Tempo query finds an add by span.url.path instead of by its name.

What's known

This hasn't been triaged. What follows comes from reading the code on main at 992e51b:

• Both services serve the API through http.MaxBytesHandler, under a ServeMux. The cart does mux.Handle("/", http.MaxBytesHandler(api, maxBodyBytes)) (app.go#L138), and the catalog does the same (app.go#L126).
• MaxBytesHandler hands the API a shallow copy of the request. The ServeMux has already set Pattern to / on the request otelhttp passed down. The API sets the route it found on the copy (api.go#L224-L232), and otelhttp, which renames the span from its own request after the handler returns, sees /.
• httpserver may lose the pattern too. stripScenarios passes r.WithContext(ctx) on when it strips an untrusted scenario (strip.go#L39-L46), so a pattern set below it would land on that copy. Whether that names a span wrongly hasn't been checked.
• The page quotes the wrong names. observing.md lists POST / and GET / and queries by span.url.path.

Not yet known: whether it reproduces in a test, whether the fix belongs in the walkthrough, in openapi/v3 or in httpserver, and whether other walkthroughs that serve HTTP are affected.

Expected

A request the API serves is in a server span named after its method and the operation's route, such as POST /carts/{cartId}/items and GET /products/{productId}, as openapi/v3 and httpserver document. The observing page names the spans that way.

References

• (*API).ServeHTTP in openapi/v3/api.go
• the span naming in httpserver/service.go#L68-L80 and httpserver/doc.go#L121-L131
• PR #1240, the observing page's story, where it was found

[Zaba505]

Triage

Verdict: a reproducible bug in how httpserver names spans, not in the walkthrough alone. Any handler between otelhttp and the one that sets Request.Pattern that passes on a copy of the request hides the route. That includes http.MaxBytesHandler, which openapi/v3 tells end users to use, http.TimeoutHandler, and httpserver's own scenario strip. The metric http.server.request.duration loses its http.route the same way. #1295 is filed to fix it.

Is this a misuse of humus?

No. httpserver owns the span: it wraps the end user's handler in otelhttp, and its doc names the span after the route the handler sets. The end user owns the handler they pass, and the walkthrough's handler does set the route.

The strongest case for misuse is that the walkthrough wraps the API in a handler that copies the request. But humus asks for that wrapping. #760, revision 1 says the API has "no size limit of the package's own: behind http.MaxBytesHandler, a body over the limit is a 413". openapi/v3's doc.go answers MaxBytesHandler's error with a 413. Nothing says the wrapping costs the span its name.

The strip case has no end user code in it at all. httpserver's stripScenarios passes on r.WithContext(ctx), and a ServeMux below it sets the pattern on that copy.

An end user can't defend against either with anything humus gives them. Trusting every caller would keep the strip from copying, but that turns failure injection's protection off.

Is it a feature request?

No. It contradicts three promises:

• openapi/v3's doc.go: "httpserver gives every request a server span, named after the operation's route, such as GET /carts/{cartId}."
• (*API).ServeHTTP: "It then sets the pattern chi found on r, so that httpserver's spans are named after the route."
• httpserver's doc.go: "The span is named after the method and the route pattern, such as "GET /hello/{name}", when the handler sets Request.Pattern, as an http.ServeMux does." In the strip case the handler is a ServeMux, and the span is named GET.

A fix restores these promises. It changes no exported name, signature, Settings field or default. observing.md documents the bug, and gets corrected.

Is it reproducible?

Yes. I ran each test 4 times under -race at 867c7f8, with Go 1.27.0, otelhttp v0.72.0, chi v5.5.0 and the OpenTelemetry SDK v1.47.0. Every failing case below failed in 4 runs out of 4, and every passing case passed in 4 out of 4. No run reported a race. The tests use humustest and httpservertest, in the packages' own harness.

Scenario	Span	http.route on http.server.request.duration	Expected
The walkthrough's add, on the cart	POST /	not checked	POST /carts/{cartId}/items
The walkthrough's add, on the catalog	GET /	not checked	GET /products/{productId}
The API under a ServeMux, behind MaxBytesHandler	POST /	/	POST /carts/{cartId}/items
The API behind MaxBytesHandler alone	POST	empty	POST /carts/{cartId}/items
The API behind TimeoutHandler alone	POST	empty	POST /carts/{cartId}/items
The API under a ServeMux alone (passes)	POST /carts/{cartId}/items	/carts/{cartId}/items	the same
A ServeMux's GET /orders, its scenario stripped (two cases: nil TrustScenario, and one that refuses)	GET	not checked	GET /orders
A ServeMux's GET /orders, with no scenario or a trusted one (passes)	GET /orders	not checked	the same

The existing TestInit_traces_an_add_across_the_cart_and_the_catalog passed in 4 runs out of 4. It finds spans by url.path, so it never checked their names.

The walkthrough: the add is POST /, the catalog's answer GET /. Each service serves mux.Handle("/", http.MaxBytesHandler(api, maxBodyBytes)) through httpserver.

1. otelhttp starts the span and passes its own copy of the request, rCtx, down.
2. The ServeMux sets rCtx.Pattern to /.
3. MaxBytesHandler passes on a copy, r2.
4. The API routes the add with chi and sets r2.Pattern to /carts/{cartId}/items.
5. otelhttp reads rCtx.Pattern, which is still /, and renames the span POST /. The test logs the add's span "POST /", the catalog's span "GET /".

A stripped scenario: the span is named GET. httpserver serves a ServeMux with GET /orders, and the request carries a scenario the caller isn't trusted with.

1. stripScenarios strips it and passes on r.WithContext(ctx). The span gets one humus.fit.strip event.
2. The ServeMux sets the copy's pattern to GET /orders.
3. otelhttp reads its own request, whose pattern is empty, and names the span after the method alone. The test logs span "GET", 1 strip events.

This corrects one claim of the report. It says whether the strip "names a span wrongly hasn't been checked". It does, with or without openapi/v3.

Why

otelhttp names the span from its own copy of the request, after the handler returns. At v0.72.0 it makes the copy at handler.go#L183, serves it at L194, and renames the span only if that copy has a pattern, at L196-L198. The metric's attributes come from the same copy, at L212-L218, whose route is read from its pattern at server.go#L392-L394. Reproduced: the table above shows the span and the route lost together.

A ServeMux sets the pattern on the request it was given, at server.go#L2901, and so does the API. So otelhttp sees the pattern only if every handler above the one that sets it passes the same request on. These don't:

• http.MaxBytesHandler copies the request to replace its body. Reproduced.
• http.TimeoutHandler passes on r.WithContext(ctx). Reproduced.
• httpserver's stripScenarios passes on r.WithContext(ctx) when it strips. Reproduced.
• auth.NewHandler's handler passes on r.WithContext(res.Context) once it authenticates a request, at handler.go#L125. A ServeMux it wraps would lose its pattern the same way. I got that from the code and didn't reproduce it.

(*API).ServeHTTP already handles one copy, chi's own, by setting the pattern back on the request it was given. #760 revision 1 records that, without it, "every span is named GET". It can't reach a copy made above it.

In the walkthrough, both effects stack. The cart strips the scenario of a caller its FAULT_TRUST_SCOPE doesn't trust, so such an add's span would be POST, not POST /. I got that from the code and didn't reproduce it. The shoppingcart is the only walkthrough that serves HTTP through httpserver, and observing.md is the only page that quotes these names.

#1295 is filed to fix it:

• an API behind MaxBytesHandler or TimeoutHandler, under a ServeMux or not, has its span and http.route named after the operation's route;
• a request whose scenario is stripped keeps its route's name;
• the walkthrough's spans are POST /carts/{cartId}/items and GET /products/{productId}, and the fix holds whichever handler on the way copies the request;
• the triage's tests, in openapi/v3, httpserver and the walkthrough, show it;
• observing.md, both packages' doc.go and the comment on ServeHTTP are corrected.

[Zaba505]

Fixed by #1295 in #1296, merged as d98170c.

The openapi API now records the route it finds in the request's context, which every copy of the request shares. A new layer right below otelhttp in httpserver sets that route on otelhttp's own request. So the server span and the http.route of http.server.request.duration are named after the operation's route behind http.MaxBytesHandler, http.TimeoutHandler or any handler that copies the request. httpserver's scenario strip and auth's handler also set a pattern their handler sets on their copy back on the request they were given, so a stripped request keeps GET /orders. The walkthrough's add is now POST /carts/{cartId}/items on the cart and GET /products/{productId} on the catalog, and observing.md finds it by name.