Force set --log-level for all modules (and/or allow configuring per module)

[mtvx]

What's the problem this feature will solve?

--log-level is nice. However, certain modules (like SQLAlchemy) default to certain log level, which doesn't get set/adjusted/overwritten via --log-level which only sets the level for root. From SQLAlchemy docs:

By default, the log level is set to logging.WARN within the entire sqlalchemy namespace so that no log operations occur, even within an application that has logging enabled otherwise.

You have to manually logging.getLogger("sqlalchemy").setLevel(logging.DEBUG) etc if you want to enable the logging.

Describe the solution you'd like

Add some flag, that will wipe the existing loggers/handlers/levels so it'll use the same level for everything?

Alternative Solutions

Do the same but also allow setting log level per module. This is maybe for flexible (but less simple). Something like:

log_level = debug
log_levels =
    pacakage1:info
    package2:warn
    package2.submodule:error

Additional context

It looks like SQLAlchemy sets the logging level when the module is loaded (so during the test discovery?):

rootlogger = logging.getLogger("sqlalchemy")
if rootlogger.level == logging.NOTSET:
    rootlogger.setLevel(logging.WARN)

Apparently pytest has not configured the logger at this point, because indeed the logging.WARNING gets set for sqlalchemy namespace. Rises a question of at what point does pytest configure the logging?

Looks like logging.root.manager.loggerDict allows to iterate (and clear/override) the configured loggers. For me, in a random environment, it shows as:

{'stack_data.serializing': <Logger stack_data.serializing (INFO)>,
 'stack_data': <logging.PlaceHolder at 0x7f105a72b8e0>,
 'concurrent.futures': <Logger concurrent.futures (INFO)>,
 'concurrent': <logging.PlaceHolder at 0x7f1059fc9630>,
 'asyncio': <Logger asyncio (INFO)>,
 'prompt_toolkit.buffer': <Logger prompt_toolkit.buffer (INFO)>,
 'prompt_toolkit': <logging.PlaceHolder at 0x7f1059b20c10>,
 'parso.python.diff': <Logger parso.python.diff (INFO)>,
 'parso.python': <logging.PlaceHolder at 0x7f1059471510>,
 'parso': <logging.PlaceHolder at 0x7f10594723e0>,
 'parso.cache': <Logger parso.cache (INFO)>,
 'TerminalIPythonApp': <Logger TerminalIPythonApp (DEBUG)>,
 'sqlalchemy': <Logger sqlalchemy (WARNING)>}

Could iterate over everything and wipe them out, leaving just the root logger?

A couple of SO questions about per-module configuration:

• https://stackoverflow.com/questions/71664685/configure-logging-for-pytest-on-a-per-module-basis
• https://stackoverflow.com/questions/52086974/pytest-selective-log-levels-on-a-per-module-basis