RotatingFileHandler formats the record twice

[serhiy-storchaka]

RotatingFileHandler calls self.format() twice for every record -- first time to determine if rollover should occur, second time to write it to the stream. It is not optimal. Even if the IO time dominates in common cases, the formatting time may be non-trivial -- this is why formatting is delayed.

It is not easy to fix without duplicating the code and introducing a coupling between several handler classes. The simplest way -- check the current file size in shouldRollover(), without adding the size of the new record. This will make the size of backups slightly larger than maxBytes, but this calculation is already inaccurate (see #64336, #69128). This will automatically solve #116263.

Linked PRs

• gh-116267: Do not check record size in RotatingFileHandler to avoid f… #116330
• gh-116267: Avoid formatting record twice in RotatingFileHandler. #122731

[Jason-Y-Z]

Thanks for raising it, the idea makes sense to me, but I guess we might need a wider consensus on this.
Nevertheless, I had a go at the implementation in #116330, which might help with the discussion