logging.handlers.BufferingHandler capacity is unclearly specified

[enrico]

BPO	32934
Nosy	@vsajip
PRs	bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and M… #14498 [3.8] bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and MemoryHandler. (GH-14498) #14507 [3.7] bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and MemoryHandler. (GH-14498) #14508

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vsajip'
closed_at = <Date 2019-07-01.12:12:59.144>
created_at = <Date 2018-02-24.10:24:49.630>
labels = ['3.7', '3.8', '3.9', 'docs']
title = 'logging.handlers.BufferingHandler capacity is unclearly specified'
updated_at = <Date 2019-07-01.12:12:59.144>
user = 'https://bugs.python.org/enrico'

bugs.python.org fields:

activity = <Date 2019-07-01.12:12:59.144>
actor = 'vinay.sajip'
assignee = 'vinay.sajip'
closed = True
closed_date = <Date 2019-07-01.12:12:59.144>
closer = 'vinay.sajip'
components = ['Documentation']
creation = <Date 2018-02-24.10:24:49.630>
creator = 'enrico'
dependencies = []
files = []
hgrepos = []
issue_num = 32934
keywords = ['patch']
message_count = 6.0
messages = ['312709', '346277', '346320', '347005', '347007', '347008']
nosy_count = 2.0
nosy_names = ['vinay.sajip', 'enrico']
pr_nums = ['14498', '14507', '14508']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue32934'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[enrico]

BufferingHandler's documentatio says "Initializes the handler with a buffer of the specified capacity." but it does not specify what capacity means. One would assume the intention is to give a bound to memory usage, and that capacity is bytes.

Looking at the source instead, the check is:

    return (len(self.buffer) >= self.capacity)

and self.buffer is initialised with an empty list, so capacity is a number of lines, which cannot be used to constrain memory usage, and for which I struggle to see a use case.

I believe that the current behaviour is counterintuitive enough to deserve, if not changing, at least documenting

[vsajip]

Python isn't a low-level language, and there isn't *in general* a particular intention "to give a bound to memory usage". When using "buffer" in a general sense, rather than a buffer object (which is more akin to a byte-array), it's usually understood to mean a Python list (used as an array) and the capacity refers to the number of elements.

You may struggle to see a use case for specifying a buffer capacity as a number of elements rather than a byte size, but that doesn't mean that such use cases don't exist.

Are you perhaps using MicroPython in a constrained-memory environment?

In the 17 years that this code has been in Python, it's the first time AFAIK that anyone has raised the term "capacity" as potentially confusing, so I don't think such confusion is common. However, I'll be happy to update the documentation to clarify that "capacity" means "number of records buffered".

[enrico]

Thanks! Clarifying "number of records buffered" would perfectly solve the problem for me.

[vsajip]

New changeset 84de34e by Vinay Sajip in branch 'master':
bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and MemoryHandler. (GH-14498)
84de34e

[vsajip]

New changeset 471d785 by Vinay Sajip (Miss Islington (bot)) in branch '3.7':
bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and MemoryHandler. (GH-14498) (GH-14508)
471d785

[vsajip]

New changeset 3db5c5c by Vinay Sajip (Miss Islington (bot)) in branch '3.8':
bpo-32934: Clarified meaning of 'capacity' for BufferingHandler and MemoryHandler. (GH-14498) (GH-14507)
3db5c5c