Otel tracing MVP

[gotgelf]

This PR represents the first step in the incremental integration of OpenTelemetry into our application. Following the recommendations from the review of PR #3701, this PR focuses on a minimal viable product for OpenTelemetry instrumentation.

[corhere]

Thanks for taking this on!

Are you comfortable with rewriting Git history? I would really appreciate if you could move the vendoring changes into their own separate commit so I can skip over it when reviewing the changes. GitHub's web interface lags horribly and eats nearly a gig of RAM just for this one browser tab when I try to review this PR.

[corhere]

Plumb your contexts through!

Suggested change

	exp, err := autoexport.NewSpanExporter(context.Background())
	exp, err := autoexport.NewSpanExporter(ctx)

[corhere]

Constructing a new handler on each request is more expensive than necessary. Given that this is a code path which is executed for every incoming HTTP request, I feel a little optimization is not premature.

Peeking under the hood of otelhttp, the operation parameter is only used as an input to the span name formatter function, along with the request being handled. The default formatter just passes through the operation name, which means that this otelHandler is just formatting the span name from the request in a very roundabout way.

Suggested change

	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	// Create a custom operation name
	operationName := r.Method + " " + r.URL.Path
	otelHandler := otelhttp.NewHandler(next, operationName)
	otelHandler.ServeHTTP(w, r)
	})
	return otelhttp.NewHandler(next, "",
	otelhttp.WithSpanNameFormatter(func(_ string, r *http.Request) string { return r.Method + " " + r.URL.Path }))

[gotgelf]

Are you comfortable with rewriting Git history?

Yes, done.

[corhere]

Thanks!