Example of filtering events to different logging handlers

[Zsailer]

After chatting with some collaborators last week, the following use-case came up. They want two "event sinks" that collect different sets of events (mostly non-overlapping).

The current API design allows them to route the same set of events to two different logs.

I've been sketching out a design for this and curious what others think. This introduces a EventRouter object that takes a list of EventLog objects. Each EventLog has a different set of allowed_schemas (a.k.a. events to record) and handlers. The EventRouter interface has the same methods as the EventLog, but loops through the EventLogs, recording events that are listed in the allowed_schemas.

Here's an example of what the API might look like:

import logging
from jupyter_telemetry import EventLog, EventRouter

router = EventRouter([
    EventLog(
        handlers=[
            logging.FileHandler('events_set1.log')
        ],
        allowed_schemas=[
            'my.events/set1/event1',       # all events are coming from set 1.
            'my.events/set1/event2',
        ]
    ),
    Eventlog(
        handlers=[
            logging.FileHandler('events_set2.log')
        ],
        allowed_schemas=[
            'my.events/set2/event1',       # all events are coming from set 2.
            'my.events/set2/event2',
        ]
    )
])

# Log an event to 'events_set1.log' but not 'events_set2.log'
router.record_event(
    'my.events/set1/event1', 1,
    {...}
)

[Zsailer]

I realized this is the wrong approach. We should just let users leverage Python's standard logging library tools (and probably help them with documentation).

The ideal approach is using logging.Filter objects.

For example,

import logging

schema_set_1 = [
    'my.events/set1/event1',       # all events are coming from set 1.
    'my.events/set1/event2'
]

schema_set_2 = [
    'my.events/set2/event1',       # all events are coming from set 2.
    'my.events/set2/event2',
]

class FilterSet1(logging.Filter):
    """Filter all events that occur in set 1."""
    def filter(self, record):
        if record.msg['__schema__'] in schema_set_1:
            return True
        return False 

handler_set_1 = logging.FileHandler('events_set_1.log')
handler_set_1.addFilter(FilterSet1())

class FilterSet2(logging.Filter):
    """Filter all events that occur in set 2."""
    def filter(self, record):
        if record.msg['__schema__'] in schema_set_2:
            return True
        return False 

handler_set_2 = logging.FileHandler('events_set_2.log')
handler_set_2.addFilter(FilterSet2())

eventlog = EventLog(
    handlers=[
        handler_set_1,
        handler_set_2
    ],
    allowed_schemas=schema_set_1 + schema_set_2
)