Implement async bound loggers for stdlib

[hynek]

This is so far only an experimental WiP/PoC. But unless I'm missing anything, adding truly async loggers shouldn't be much of a problem.

---

You can use it like this:

import structlog
import asyncio
import logging
import sys

logging.basicConfig(
    format="%(message)s",
    stream=sys.stdout,
    level=logging.INFO,
)


structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        structlog.processors.UnicodeDecoder(),
        structlog.processors.JSONRenderer()
    ],
    context_class=dict,
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.AsyncBoundLogger,  # <---
    cache_logger_on_first_use=True,
)

logger = structlog.get_logger()

async def main():
    l = logger.bind(foo="bar")

    coro = l.info("qux")
    w = asyncio.sleep(0.5)

    await l.info("baz", x="42")

    await asyncio.gather(coro, w)

asyncio.run(main())

[Aeron]

I’m personally not sure how I feel about the thread executor approach. I understand that for such a rich library as Structlog—with various processors, handlers, and loggers—it might be even the only solution, to wrap a blocking logger calls (and everything behind it) and execute them in threads.

And yet I wonder if a simplistic log record emitting is worth the cost of thread creation and existence. I mean, the downsides of threads are well known. So it’s a compromise.

On the other hand, an Asyncio’s StreamWriter (more natural) approach seems to be much more resource-effective, but it implies everything in a chain must be non-blocking as well, of course.

Nonetheless, I’m glad to see it’s a WIP already, good job 👍

[hynek]

And yet I wonder if a simplistic log record emitting is worth the cost of thread creation and existence.

There are two problems here:

1. You cannot predict where most of the time will be spent. Processors am can take quite a bit of computing power (eg serialization).
2. Using async write APIs would require to rewrite all of structlog and all processors to be async too. I guess your best call would be to play with different loggers and I believe logging comes with something. But again: your logger will block while you json.dumps.

[hynek]

@Aeron I've added a warning in a0d9d48 to point out the overhead. You're right that not everyone is gonna be aware of the trade-off.