Repository navigation
Does the Go agent re-sample incoming sampled traces before applying upstream references? #14201
|
Hi, I noticed a possible ordering issue in the Go agent's sampling logic and would like to confirm whether this is expected behavior. Versions inspected
ScenarioFor a service chain Service A samples a request and propagates a valid Code observationsIn However, ds := NewDefaultSpan(t, parent)
// Resolve the local parent span...
isForceSample := len(ds.Refs) > 0
if parentSpan == nil && !isForceSample {
isSampled := t.Sampler.IsSampled(ds.OperationName)
if !isSampled {
return newNoopSpan(t), true, nil
}
}
// The upstream reference is applied only here.
for _, opt := range coreOpts {
opt.(tracing.SpanOption).Apply(ds)
}
Minimal verificationI tested the core API with:
The returned span was a With a downstream sampling rate of Questions
For comparison, the Java agent appears to check for a valid upstream carrier and pass Thanks for helping clarify this behavior. |
Replies: 1 comment 4 replies
|
Thanks for the detailed report. You're right, this is a bug, and it is still present on the latest 1. Is inheriting the upstream sampling decision intended? Yes. When the entry has a valid upstream context, the trace must be force-sampled and the local sampler must not run. That is what 2. Should Yes. When the plugin API was added (apache/skywalking-go#9), The fix is to move the The same reorder also fixes two related problems:
The effect is also worse than losing B's segment. Once B drops the trace, its exit span is a noop and injects no 3. Is there another mechanism that prevents the re-sampling? No. The only workaround today is to set Would you be interested in raising a PR? It would need:
If you'd rather not, let me know and we'll take care of it. |
Thanks for the detailed report. You're right, this is a bug, and it is still present on the latest
main(apache/skywalking-go@7c664b5). I reproduced it the same way: a validsw8header plusNewConstSampler(false)on the downstream side gives aNoopSpan, and nothing is reported.1. Is inheriting the upstream sampling decision intended?
Yes. When the entry has a valid upstream context, the trace must be force-sampled and the local sampler must not run. That is what
isForceSampleincreateSpan0is for, and it matches the Java agent:ContextManager#createEntrySpanforce-samples whenevercarrier.isValid().2. Should
coreOptsbe applied beforeisForceSampleis evaluated?Yes. When the plugin AP…