Undefined method `update' for #<Datadog::RateSampler>.

[BabyGroot]

I have seen quite a lot of log level errors popping up in our application with the following message:

ERROR -- ddtrace: [ddtrace] (/usr/local/bundle/gems/ddtrace-0.32.0/lib/ddtrace/workers.rb:58:in `rescue in callback_traces') Error during traces flush: dropped 4 items. Cause: undefined method `update' for #<Datadog::RateSampler:0x000055baf400c530>

I went through the code in this repo and can confirm the rescue block but I couldn't determine why dd_trace was throwing an undefined method update for a RateSampler instance.

Is this a known issue?

[delner]

Thanks for the report @BabyGroot

We recently released a totally new RuleSampler in 0.32.0 via #917. update is typically invoked by the transport when it updates the sampling rates on a PrioritySampler, so there's a fair chance that wires might be getting crossed such that it's attempting to update a RateSampler which does not respond to update.

I've not seen this issue yet; can you provide any more details that'd help me reproduce it?

[delner]

Looking at the code, RuleSampler will receive update, and forward it to its default sampler, whatever that's configured to. That default sampler is usually a RateByServiceSampler, but it looks like it can be a RateSampler if a default sample rate is provided:

@default_sampler = if default_sampler
                     default_sampler
                   elsif default_sample_rate
                     # We want to allow 0.0 to drop all traces, but \RateSampler
                     # considers 0.0 an invalid rate and falls back to 100% sampling.
                     #
                     # We address that here by not setting the rate in the constructor,
                     # but using the setter method.
                     #
                     # We don't want to make this change directly to \RateSampler
                     # because it breaks its current contract to existing users.
                     Datadog::RateSampler.new.tap { |s| s.sample_rate = default_sample_rate }
                   else
                     RateByServiceSampler.new(1.0, env: -> { Datadog.tracer.tags[:env] })
                   end

@BabyGroot in your application, have you set DD_TRACE_SAMPLE_RATE or configured Datadog.configuration.sampling.default_rate? If so, that might explain the error.

If this is the cause, then we should change RuleSampler to only forward update if the default sampler responds to update.

[delner]

@BabyGroot I opened a PR that addresses the #update problem I could see in the RuleSampler. Can you give it a try, let me know if it fixes the problem you were seeing?

[BabyGroot]

@delner thanks so much for the info!
From what I can see in our application config, we are using a sampling rate of 0.01 ie 1%
In datadog.rb:

Datadog.configure do |c|
...
c.sampling.default_rate = 0.01
...
end

And in our docker compose files, we set an Environment variable:

"DD_TRACE_SAMPLE_RATE=0.01"

[BabyGroot]

@BabyGroot I opened a PR that addresses the #update problem I could see in the RuleSampler. Can you give it a try, let me know if it fixes the problem you were seeing?

I can prioritise this, will have a look at your PR. Thanks!
For now we have just added an exclusion rule in our log indexing.

[delner]

Should be resolved with #950. Thanks!