fix: separate OTel provider provisioning from span generation#3880
Merged
fix: separate OTel provider provisioning from span generation#3880
Conversation
0d31195 to
6bf593e
Compare
eddea5b to
31b350b
Compare
zhming0
approved these changes
May 4, 2026
Contributor
zhming0
left a comment
There was a problem hiding this comment.
LGTM ✅ with some nits. Well done 👍🏿
0758991 to
ca8ddc5
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Problem
setupLoggerAndConfigonly initialised the OpenTelemetry TracerProvider whenTRACEPARENTwas set in the environment. This was unreliable:TRACEPARENTis injected into every subprocess the agent spawns (git, bootstrap, hooks) so its presence is not a reliable signal that the current process is a traced subcommand.This also caused a double-initialisation bug in the bootstrap flow:
setupLoggerAndConfigcreated one provider, andstartTracingOpenTelemetrycreated a second, loosing the first one.Fix
Separate the two concerns:
setupLoggerAndConfigalways initialises the provider whenBUILDKITE_TRACING_BACKEND=opentelemetry, and owns its lifecycle (ForceFlush+Shutdownindone())startTracingOpenTelemetryusesotel.GetTracerProvider()to obtain the already-registered provider and only creates the root job span, adding job-specific attributes (pipeline slug, job ID, etc.) at span level rather than resource level since they are not available at provider init timeContext
Changes
Testing
go test ./...). Buildkite employees may check this if the pipeline has run automatically.go tool gofumpt -extra -w .)screenshots on grafana to show the subcommands:

Disclosures / Credits