MemoryHandler can't be setup through a DictConfig if any other logger is setup after it #123239
Description
Bug report
Bug description:
This is the logging that's failing.
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"fine-logger1": {
'level': logging.INFO,
'formatter': 'verbose-console',
'class': 'logging.StreamHandler',
'stream': 'ext://sys.stdout',
},
"fine-logger2": {
"level": logging.INFO,
"class": "logging.FileHandler",
"filename": Path("logs/somefilename.log"),
"mode": "a",
"formatter": "verbose",
},
"fine-logger3": {
"level": logging.DEBUG,
"class": "logging.FileHandler",
"filename": Path("logs/errors/somefilename.log"),
"delay": True,
"formatter": "verbose",
},
"bugged-logger3": {
"level": logging.DEBUG,
"class": "logging.handlers.MemoryHandler",
"capacity": 9999999,
"flushLevel": logging.ERROR,
"target": "fine-logger3",
"flushOnClose": False,
},
},
"loggers": {
"bug-logger": {
"handlers": ["fine-logger1", "fine-logger2", "bugged-logger3"],
"level": logging.DEBUG,
"propagate": True
},
},
}After it's been setup, if any other handler is setup afterwards (in my case it's the copernicusmarine logger) the handler bugged-logger3 will have a None inside its target field, as every handler that's not incremental will delete existing unused handlers.
I've dug into the source code of logging and I think I've tracked down where this is happpening, but I'm not really sure on how to fix it.
In the config file python3.11/logging/config.py in the configure() function there's a call to _clearExistingHandlers():
else:
disable_existing = config.pop('disable_existing_loggers', True)
_clearExistingHandlers() <-----
# Do formatters first - they don't refer to anything else
formatters = config.get('formatters', EMPTY_DICT)
for name in formatters:
try:
formatters[name] = self.configure_formatter(
formatters[name])
If we comment this line, it works completely fine. But I guess this just keeps handlers loaded in memory and that's not a proper solution.
CPython versions tested on:
3.11
Operating systems tested on:
Linux