-
Notifications
You must be signed in to change notification settings - Fork 478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(api): Added error_sampler
option
#2456
Conversation
tests/test_client.py
Outdated
# Baseline test with issues_sampler only, both floats and bools | ||
(mock.MagicMock(return_value=1.0), None, 1), | ||
(mock.MagicMock(return_value=0.0), None, 0), | ||
(mock.MagicMock(return_value=True), None, 1), | ||
(mock.MagicMock(return_value=False), None, 0), | ||
# Baseline test with sample_rate only | ||
(None, 0.0, 0), | ||
(None, 1.0, 1), | ||
# issues_sampler takes precedence over sample_rate | ||
(mock.MagicMock(return_value=1.0), 0.0, 1), | ||
(mock.MagicMock(return_value=0.0), 1.0, 0), | ||
), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also test different rates for different errors. Like ZeroDivisionError: 1.0
, RuntimeError: 0.5
and such.
@@ -456,10 +456,11 @@ def _should_sample_error( | |||
event, # type: Event |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
event
contains information about the logged message via logging?
There is a hint
parameter in the before_send
that contains this information in log_record
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@saippuakauppias Could you please clarify your question? How exactly are you logging the message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are using the LoggingIntegration
, your log message should appear in the event
passed to this function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I am using LoggingIntegration
and logging all warning messages to sentry. I tried to discard some messages that are not needed and in debug mode found that the path to the file that triggers the message is in hint['log_record'].pathname
.
Actually, I need this as a minimum. I would like this feature to have a similar option to filter by module name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@saippuakauppias we decided to also pass the hint
into the events_sampler
, so you will be able to make your sampling decision based on information contained in the hint
.
@szokeasaurusrex Is it only errors passing through If it's only errors, calling the option More importantly, are the events from the original issue errors (as opposed to e.g. messages) and would they pass through the |
An issue in Sentry represents N events, so calling it |
Co-authored-by: Ivana Kellyerova <ivana.kellyerova@sentry.io>
The
I am unsure given the information I see in the original issue. @saippuakauppias, could you please provide the code where these errors/messages originate from, or provide us more detail with how these are being generated, so we can answer this question? |
Yes, looks like it's everything but transactions and checkins: sentry-python/sentry_sdk/client.py Lines 556 to 560 in 4d10edf
In that case
@cleptric Checkins and transactions also end up in |
Once we've settled on the API please don't forget to update the docs @szokeasaurusrex 📖 |
I discussed with @cleptric, and we now think we should pick a different name than The best idea we came up with was to just call the new sampler |
I'm using django, so it's going to be a bit of a specific example here, but I think it's reproducible. # settings/production.py
LOGGING = {
'version': 1,
'formatters': {
'simple': {
'format': '[%(asctime)s] %(levelname)s %(module)s %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
},
'handlers': {
'sentry': {
'level': 'WARNING',
'class': 'base.sentry.SentryHandler',
},
'streamlogger': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
},
'loggers': {
'warn.logger': {
'handlers': ['sentry', 'streamlogger'],
'level': 'WARNING',
'propagate': False
},
},
'root': {
'level': 'INFO',
'handlers': ['streamlogger'],
},
} # base/sentry.py
from sentry_sdk.integrations.atexit import AtexitIntegration
from sentry_sdk.integrations.dedupe import DedupeIntegration
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.excepthook import ExcepthookIntegration
from sentry_sdk.integrations.logging import EventHandler, LoggingIntegration
from sentry_sdk.integrations.modules import ModulesIntegration
from sentry_sdk.integrations.stdlib import StdlibIntegration
class SentryHandler(EventHandler):
...
def initialize(dsn, service, release, branch, debug: bool, breadcrumb_filters: list[dict] = None):
from django.conf import settings
sample_events_by_path = {
f'{settings.PROJECT_ROOT}/some/module.py': 0.0001,
}
def prepare_event(event, hint):
request_data = get_request_data()
tags = event.setdefault('tags', {})
# my hack for filtration here
is_sampled = False
if isinstance(hint, dict) and 'log_record' in hint:
pathname = getattr(hint['log_record'], 'pathname', '')
is_sampled = pathname in sample_events_by_path
sample_rate = sample_events_by_path.get(pathname, 1)
if sample_rate < 1.0 and random.random() < sample_rate:
return None
tags['is_sampled'] = is_sampled
return event
sentry_sdk.init(
dsn=dsn,
release=release or None,
before_send=prepare_event,
default_integrations=False,
integrations=[
StdlibIntegration(),
ExcepthookIntegration(),
DedupeIntegration(),
AtexitIntegration(),
ModulesIntegration(),
DjangoIntegration(),
LoggingIntegration(level=None, event_level=None),
],
) # some/module.py
import logging
import requests
logger = logging.getLogger('warn.logger')
def some_fun():
try:
resp = requsts.get('http://some.service.name')
resp.raise_for_status()
except Exception as e:
logger.warning('Error get value from service', exc_info=True) |
I'm not opposed to calling it FWIW I don't think there's an ideal solution here. But if I saw |
I discussed with @antonpirker this morning about naming the sampler as |
events_sampler
option
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, left one final suggestion.
sentry_sdk/client.py
Outdated
else ("sample_rate", "contains") | ||
) | ||
logger.warning( | ||
"The provided %s %s an invalid value. The value should be a float or a bool. Defaulting to sampling the event." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including the sample_rate
in the log message might make debugging easier.
events_sampler
optionerror_sampler
option
Thank you so much for implementing this idea! Can you please tell me if I can hope that soon you will release a new version of the package so that I can install it from PyPI? |
@saippuakauppias We typically release new versions of the Python SDK every few weeks, so hopefully you will be able to install it from PyPI soon |
Now, it is possible for Python SDK users to dynamically sample errors using the
error_sampler
option, similar how they are able to sample transactions dynamically with thetraces_sampler
option.In other words, the new
error_sampler
is to thesample_rate
as the existingtraces_sampler
is to thetraces_sample_rate
.Fixes #2440