Asynchronous HTTP Logging

Non-blocking HTTP handler for Python logging with local SQLite buffer/cache.

Documentation

Please refer to the project Wiki.

Installation

Virtual environment is highly recommended.

pip install http_logging

Basic usage

import logging

from http_logging import HttpHost
from http_logging.handler import AsyncHttpHandler

log_handler = AsyncHttpHandler(http_host=HttpHost(name='your-domain.com'))

logger = logging.getLogger()
logger.addHandler(log_handler)

# Works with simple log messages like:
logger.info('Some useful information...')

# Can also handle extra fields:
logger.warning('You\'ve been warned!', extra={'foo': 'bar'})

# And, of course, captures exception with full stack-trace
try:
    1/0
except Exception as exc:
    logger.error('Ooops!', exc_info=exc)

These log messages are cached in a local SQLite database and periodically delivered (asynchronously, in a separate thread) to your host in a POST request with a body similar to this one:

[
    {
        "type": "async-http-logging",
        "created": 1610393068.365492,
        "relative_created": 1505.5122375488281,
        "message": "Some useful information...",
        "level": {
            "number": 20,
            "name": "INFO"
        },
        "stack_trace": null,
        "sourcecode": {
            "pathname": "/path/to/your/python/script.py",
            "function": "function_name",
            "line": 123
        },
        "process": {
            "id": 1234,
            "name": "MainProcess"
        },
        "thread": {
            "id": 1234567890,
            "name": "MainThread"
        }
    },
    {
        "type": "async-http-logging",
        "created": 1610393068.3663092,
        "relative_created": 1506.3292980194092,
        "message": "You've been warned!",
        "level": {
            "number": 30,
            "name": "WARNING"
        },
        "stack_trace": null,
        "sourcecode": {
            "pathname": "/path/to/your/python/script.py",
            "function": "function_name",
            "line": 456
        },
        "process": {
            "id": 1234,
            "name": "MainProcess"
        },
        "thread": {
            "id": 1234567890,
            "name": "MainThread"
        }
    },
    {
        "type": "async-http-logging",
        "created": 1610393068.3663092,
        "relative_created": 1506.3292980194092,
        "message": "Ooops!",
        "level": {
            "number": 40,
            "name": "ERROR"
        },
        "stack_trace": "Traceback (most recent call last):\n  File \"/path/to/your/python/script.py\", line 17, in function_name\n    1/0\nZeroDivisionError: division by zero\n",
        "sourcecode": {
            "pathname": "/path/to/your/python/script.py",
            "function": "function_name",
            "line": 17
        },
        "process": {
            "id": 1234,
            "name": "MainProcess"
        },
        "thread": {
            "id": 1234567890,
            "name": "MainThread"
        }
    }
]

In your backend, you can funnel these logs to wherever suits you best: database, ElasticSearch index, third-party monitoring service, etc.

Learn more about these and other features in the project Wiki.