Clear buffer in MemoryHandler flush

[Penlect]

BPO	38781
Nosy	@vsajip, @Penlect
PRs	bpo-38781: Clear buffer in MemoryHandler flush #17132

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-11-13.22:20:29.565>
created_at = <Date 2019-11-12.21:43:12.763>
labels = ['type-feature', 'library', '3.9']
title = 'Clear buffer in MemoryHandler flush'
updated_at = <Date 2019-11-13.22:20:29.564>
user = 'https://github.com/Penlect'

bugs.python.org fields:

activity = <Date 2019-11-13.22:20:29.564>
actor = 'penlect'
assignee = 'none'
closed = True
closed_date = <Date 2019-11-13.22:20:29.565>
closer = 'penlect'
components = ['Library (Lib)']
creation = <Date 2019-11-12.21:43:12.763>
creator = 'penlect'
dependencies = []
files = []
hgrepos = []
issue_num = 38781
keywords = ['patch']
message_count = 2.0
messages = ['356489', '356563']
nosy_count = 2.0
nosy_names = ['vinay.sajip', 'penlect']
pr_nums = ['17132']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38781'
versions = ['Python 3.9']

[Penlect]

The logging.handlers.MemoryHandler has a method flush which clears the buffer by assigning an empty list literal:

self.buffer = []

This forces the buffer to be a list instance.
My suggestion is to clear the buffer like this instead:

self.buffer.clear()

In this way it would be possible to implement a custom buffer or use the collections.deque when subclassing MemoryHandler. At the moment you must copy-past the flush method and modify it accordingly in the subclass:

def flush(self):
    # (Docstring skipped)
    self.acquire()
    try:
        if self.target:
	    for record in self.buffer:
	        self.target.handle(record)
	    self.buffer = []  # <-- change to `self.buffer.clear()`
    finally:
        self.release()

Example where this change is useful
===================================
This example creates a DequeMemoryHandler which uses the collections.deque as a buffer. Only the latest capacity number of records will stored in the buffer. The buffer is then flushed if and only if a record has a level greater or equal to logging.ERROR.

import collections
import logging
from logging.handlers import MemoryHandler


class DequeMemoryHandler(MemoryHandler):

    def __init__(self, capacity, *args, **kwargs):
        super().__init__(capacity, *args, **kwargs)
        self.buffer = collections.deque(maxlen=capacity)

    def shouldFlush(self, record):
        return record.levelno >= self.flushLevel


handler = DequeMemoryHandler(capacity=5, target=logging.StreamHandler())
logging.basicConfig(level=logging.INFO, handlers=[handler])

for i in range(1, 21):
    logging.info('Spam %d', i)
    if i % 10 == 0:
        logging.error(f'---> Eggs {i}')

Desired output (with the change):
---------------------------------
Spam 7
Spam 8
Spam 9
Spam 10
---> Eggs 10
Spam 17
Spam 18
Spam 19
Spam 20
---> Eggs 20

Actual output (without the change):
-----------------------------------
Spam 7
Spam 8
Spam 9
Spam 10
---> Eggs 10
Spam 11
Spam 12
Spam 13
Spam 14
Spam 15
Spam 16
Spam 17
Spam 18
Spam 19
Spam 20
---> Eggs 20

[Penlect]

The suggested change has been merged. I'm closing this issue. Thank you Vinay Sajip for reviewing.