SentryJsonProcessor breaks logging breadcrumbs

[hannseman]

When using SentryJsonProcessor no logging breadcrumbs show up in sentry.

More info: #15

[chiragjn]

Bump for breadcrumbs support. I have written some code of my own to let Sentry's LoggingIntegration take care of logs (both errors and breadcrumbs) from structlog instead of using this processor.
https://github.com/chiragjn/sentry_structlog_experiments/

But would love to start using SentryJsonProcessor if both error and breadcrumb log events could be taken care of.

[TomGoBravo]

When making the transition from the standard logging to structlog I tried structlog.configure(processors=[... SentryProcessor(level=logging.INFO) thinking that sentry would cleverly continue ingesting < error as breadcrumbs and >= error as events. Instead sentry creates an event for every call similar to structlog.get_logger().info('message', extra_value=foo). So now I need to decide if it is worth changing to level=logging.ERROR, which will lose the info formally in breadcrumbs, to make Sentry.io is less noisy.

[b0g3r]

I write this wrapper for fixing the breadcrumb issue:
https://gist.github.com/b0g3r/2a723e267e6f1e558d049e576a1c020b

TLDR: grab all events, compare level with breadcrumb_level, extract from event_dict breadcrumb object, save it as breadcrumb

[b0g3r]

@paveldedik We used wrapper for catching breadcrumbs a few months in our production. What do you think if I create a pull request with it into upstream SentryJsonProcessor?

[pySilver]

@b0g3r I guess the following config should work fine?

LOGGING = {
    "version": 1,
    "disable_existing_loggers": True,
    "formatters": {
        "json_formatter": {
            "()": structlog.stdlib.ProcessorFormatter,
            "processor": structlog.processors.JSONRenderer(),
        },
        "plain_console": {
            "()": structlog.stdlib.ProcessorFormatter,
            "processor": structlog.dev.ConsoleRenderer(),
        },
        "key_value": {
            "()": structlog.stdlib.ProcessorFormatter,
            "processor": structlog.processors.KeyValueRenderer(key_order=["timestamp", "level", "event", "logger"]),
        },
    },
    "handlers": {
        "console": {
            "level": "DEBUG",
            "class": "logging.StreamHandler",
            "formatter": "json_formatter",
        }
    },
    "root": {"level": "INFO", "handlers": ["console"]},
    "loggers": {
        "django.db.backends": {
            "level": "ERROR",
            "handlers": ["console"],
            "propagate": False,
        },
        # Errors logged by the SDK itself
        "sentry_sdk": {"level": "ERROR", "handlers": ["console"], "propagate": False},
        "django.security.DisallowedHost": {
            "level": "ERROR",
            "handlers": ["console"],
            "propagate": False,
        },
    },
}

structlog.reset_defaults()
structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        SentryBreadcrumbJsonProcessor(level=logging.ERROR),
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        structlog.processors.UnicodeDecoder(),
        structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
    ],
    context_class=structlog.threadlocal.wrap_dict(dict),
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

# Sentry
# ------------------------------------------------------------------------------
SENTRY_DSN = env("SENTRY_DSN")
SENTRY_LOG_LEVEL = env.int("DJANGO_SENTRY_LOG_LEVEL", logging.INFO)

sentry_logging = LoggingIntegration(
    level=SENTRY_LOG_LEVEL,  # Capture info and above as breadcrumbs
    event_level=logging.ERROR,  # Send errors as events
)
sentry_sdk.init(
    dsn=SENTRY_DSN,
    release=env("GIT_COMMIT", "unspecified"),
    integrations=[
        sentry_logging,
        DjangoIntegration(),
        CeleryIntegration(),
        RedisIntegration(),
    ],
    traces_sample_rate=env.float("SENTRY_TRACES_SAMPLE_RATE", 0.25),
)

[james-certn]

@b0g3r @paveldedik I think what you both have contributed above in your comments should be merged in, yes (documentation and code updates).

👍 from this random internet person

[aringot]

@aexvir I believe you are one of the last person that approved PRs "recently", maybe you can answer the last comments? We had a similar issue as well and used the workaround suggested by @b0g3r.

[paveldedik]

Hi, sorry for the late response, @aexvir gave me the permissions to merge PRs again, so I could take care of the issues.

I released a new BETA version of structlog-sentry 2.0.0b1 as it contains several breaking changes, for full list check this PR - #79

It still needs to be tested properly in production (I am currently testing it) so it would be nice if some of you could test it too.

Especially fixed breadcrumbs should be part of the new version, SentryJsonProcessor is also no longer needed, see https://github.com/kiwicom/structlog-sentry#logging-as-json

So please, if you have time to test 2.0.0b1 it would be great.

I'm going to close this issue now as it should be fixed, feel free to open a new issue or reopen this one if you still have issues after upgrading to the beta version.