-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[3.5] Backport add sampling rate to fix distributed tracing #16951
[3.5] Backport add sampling rate to fix distributed tracing #16951
Conversation
Skipping CI for Draft Pull Request. |
/test all |
ExperimentalDistributedTracingSamplingRatePerMillion is the number of samples to collect per million spans. Defaults to 0. Signed-off-by: James Blair <mail@jamesblair.net>
Signed-off-by: James Blair <mail@jamesblair.net>
efeef90
to
5a564d5
Compare
Signed-off-by: James Blair <mail@jamesblair.net>
Ready for review. With this branch we can once more see tracing spans in jaeger for etcd 3.5: For reviewers, the testing procedure can be: # Clone and compile pr branch
git clone -b fix-distributed-tracing https://github.com/jmhbnz/etcd.git
cd etcd && make build
# Start jaeger instance
docker run -d --name jaeger -e COLLECTOR_OTLP_ENABLED=true -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 -p 5775:5775/udp -p 6831:6831/udp -p 6832:6832/udp -p 5778:5778 -p 16686:16686 -p 14250:14250 -p 14268:14268 -p 14269:14269 -p 4317:4317 -p 4318:4318 -p 9411:9411 jaegertracing/all-in-one:1.51
# Start etcd instance
./bin/etcd --experimental-enable-distributed-tracing=true --experimental-distributed-tracing-address=localhost:4317 --experimental-distributed-tracing-service-name=etcd --experimental-distributed-tracing-instance-id=hello --experimental-distributed-tracing-sampling-rate=999999&
# Generate some spans
sleep 10s
./bin/etcdctl put foo bar
./bin/etcdctl get foo
sleep 10s
# Open jaeger ui in browser and select etcd in dropdown
echo "http://localhost:16686"
# Clean up etcd and jaeger instances started earlier
# killall etcd
# docker rm --force jaeger |
cc @dashpole |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ExperimentalDistributedTracingSamplingRatePerMillion
is the number of samples to collect per million spans.Defaults to 0.
Without this flag tracing is effectively broken / unusable in
release-3.5
as the default sample rate was changed to0
in 3.5.5 via 4c013c9.Backports:
Fixes #16935