Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs are misleading when transaction is sampled due to parent being sampled #1712

Closed
rmsy opened this issue Feb 5, 2022 · 5 comments 路 Fixed by #1716
Closed

Logs are misleading when transaction is sampled due to parent being sampled #1712

rmsy opened this issue Feb 5, 2022 · 5 comments 路 Fixed by #1716
Assignees
Projects
Milestone

Comments

@rmsy
Copy link

rmsy commented Feb 5, 2022

Issue Description

Hello 馃憢

If a transaction has a parent which is sampled, the tracing debug logs are slightly misleading. Due to the precedence of the conditionals below, the sample_rate may be false or 0.0 if a sampler proc is defined and returns one of them:

sample_rate =
if @traces_sampler.is_a?(Proc)
@traces_sampler.call(sampling_context)
elsif !sampling_context[:parent_sampled].nil?
sampling_context[:parent_sampled]
else
@traces_sample_rate
end

This results in the following log message being logged, indicating the transaction will be discarded:

if sample_rate == 0.0 || sample_rate == false
@sampled = false
log_debug("#{MESSAGE_PREFIX} Discarding #{transaction_description} because traces_sampler returned 0 or false")
return
end

However, if @parent_sampled is true for the transaction, it is still sent in finish:

unless @sampled || @parent_sampled
hub.current_client.transport.record_lost_event(:sample_rate, 'transaction')
return
end
event = hub.current_client.event_from_transaction(self)
hub.capture_event(event)

I assume the first two conditionals in the sample_rate assignment can just be reversed, but I wasn't 100% sure if doing that would have any other side-effects, so I opened this issue instead of a PR.

Reproduction Steps

  1. Configure Sentry to have a traces_sampler proc which returns false or 0.0
  2. Send a request to Rails with a Sentry-Trace header
  3. Observe the debug log message indicating the transaction will be discarded
  4. Observe that the transaction is still sent to Sentry

Expected Behavior

The log message should not indicate that the transaction will be discarded.

Actual Behavior

The log message indicates that the transaction will be discarded.

Ruby Version

2.7.4

SDK Version

4.8.2

Integration and Its Version

Rails

Sentry Config

No response

@st0012
Copy link
Collaborator

st0012 commented Feb 5, 2022

After digging into this a bit, I think this there's a conflict between 2 tracing requirements or my understanding on them.

  1. We should prioritize traces_sampler's decision over inherited decision. This follows sentry-python's implementation.
  2. Always send the transaction if its parent is sampled, which is the @parent_sampled in this line

unless @sampled || @parent_sampled

I think 2 is probably wrong as sentry-python doesn't implement this and it kind of disobeys the decision of set_initial_sample_decision. So removing the @parent_sampled part should be the right thing to do.

@sl0thentr0py Can you help me check if the above is true?

@st0012 st0012 added this to To do in 5.x via automation Feb 5, 2022
@st0012 st0012 added this to the 5.1.0 milestone Feb 5, 2022
@sl0thentr0py
Copy link
Member

@st0012, the official precedence order requirement is written down here, so I believe yes, 1. is correct - traces_sampler takes priority.

@st0012
Copy link
Collaborator

st0012 commented Feb 7, 2022

@sl0thentr0py sorry that I wasn't clear about what "the above" meant. I meant that my 2nd point was incorrect and the condition should be changed.

@sl0thentr0py
Copy link
Member

sl0thentr0py commented Feb 7, 2022

yes, 2. is incorrect and the condition should be changed. The transaction should be sent based on the sampling decision in 1. where traces_sampler takes priority over the parent.

@st0012
Copy link
Collaborator

st0012 commented Feb 7, 2022

@sl0thentr0py thanks for the confirmation!

@rmsy in this case, the logging logic is fine. it's the transaction should not be sent, which I'll fix shortly.

if you want to keep the current behavior (send unsampled transaction based on its parent's decision), you need prioritize the inherited sampling decision in your traces_sampler:

config.traces_sampler = lambda do |sampling_context|
  parent_sampled = sampling_context[:parent_sampled]

  if !parent_sampled.nil?
    parent_sampled
  else
    # the rest of sampling logic
  end
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
5.x
Done
Development

Successfully merging a pull request may close this issue.

3 participants