Fix exc_info and race condition in ProcessorFormatter #109
I'm totally new to structlog so it's possible that I just missed something, but I think there's an issue with exception formatting and some sort of race condition in ProcessorFormatter. Let me illustrate that with a code example:
#!/usr/bin/env python3 import logging import structlog import sys structlog.configure( processors=[ structlog.stdlib.add_log_level, structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], logger_factory=structlog.stdlib.LoggerFactory(), wrapper_class=structlog.stdlib.BoundLogger, ) handler_structlog = logging.StreamHandler(stream=sys.stdout) handler_structlog.setFormatter(structlog.stdlib.ProcessorFormatter( processor=structlog.processors.JSONRenderer(), # processor=structlog.dev.ConsoleRenderer(colors=False), foreign_pre_chain=[ structlog.processors.format_exc_info, ], )) handler_classic = logging.StreamHandler(stream=sys.stdout) handler_classic.setFormatter(logging.Formatter( fmt="%(asctime)s - %(levelname)s - %(message)s", )) logger1 = logging.getLogger('L1') logger1.addHandler(handler_classic) logger1.addHandler(handler_structlog) logger2 = logging.getLogger('L2') logger2.addHandler(handler_structlog) logger2.addHandler(handler_classic) for logger in (logger1, logger2): try: raise RuntimeError("ohnoes") except Exception: logger.exception("the worst has happened") print("--") print()
Here I'm creating two stdlib handlers: one that's using ProcessorFormatter and one with the standard
If I run it with the latest release of structlog, I'll get the following output:
I see a couple of issues here:
I tried to pull together a patch and after the fix the output looks like I would expect it to:
@@ Coverage Diff @@ ## master #109 +/- ## ========================================== - Coverage 100% 99.61% -0.39% ========================================== Files 13 13 Lines 774 782 +8 Branches 96 98 +2 ========================================== + Hits 774 779 +5 - Misses 0 2 +2 - Partials 0 1 +1
Thank you very much for that detailed report/patch! The only big picture thing that comes to my mind is that we probably will have to make the unsetting of stack information on the log record an option since it’s possible, someone (there is always someone) may rely on it being there?
I could get talked into making the new behavior the default because it’s just very unlikely.