warnings: -W option and PYTHONWARNINGS now use the message as a regex

[vstinner]

BPO	43862
Nosy	@vstinner, @tirkarthi
PRs	bpo-43862: Enhance -W cmdline option documentation #25439
Superseder	bpo-34624: -W option and PYTHONWARNINGS env variable does not accept module regexes

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-12-13.23:28:26.708>
created_at = <Date 2021-04-16.13:06:47.083>
labels = ['library', '3.10']
title = 'warnings: -W option and PYTHONWARNINGS now use the message as a regex'
updated_at = <Date 2021-12-13.23:28:26.707>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-12-13.23:28:26.707>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-12-13.23:28:26.708>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2021-04-16.13:06:47.083>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43862
keywords = ['patch']
message_count = 7.0
messages = ['391190', '391193', '391194', '391198', '391233', '391588', '408497']
nosy_count = 2.0
nosy_names = ['vstinner', 'xtreak']
pr_nums = ['25439']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '34624'
type = None
url = 'https://bugs.python.org/issue43862'
versions = ['Python 3.10']

[vstinner]

I propose to change the -W command line option and the PYTHONWARNINGS environment variable to use the message as a regular expression in Python 3.10. Or does anyone have a reason to keep the current behavior as it is?

--

Python provides two ways to specify warnings filters:

• -W command line option: can be used multiple times
• PYTHONWARNINGS environment variable: can contain multiple options separated by commas

While the Python API warnings.filterwarnings(action, message="", ...) uses the message as a regular expression, -W and PYTHONWARNINGS require to match *exactly* the *whole* message.

For example, if you only want to ignore the new distutils deprecation warning, you must write exactly:

$ ./python -X dev -W 'ignore:The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives:DeprecationWarning' -c 'import distutils'

I use -X dev to show DeprecationWarning, or you can also use -Wdefault if you prefer.

If the deprecation warning changes in Python or if you have a single typo, the warning is not ignored. Example with a typo ("3.13" rather than "3.12"):

$ ./python -X dev -W 'ignore:The distutils package is deprecated and slated for removal in Python 3.13. Use setuptools or check PEP 632 for potential alternatives:DeprecationWarning' -c 'import distutils'
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

The PYTHONWARNINGS has another limitation: you cannot specify a message if it contains a comma (","). Hopefully, Python doesn't raise warnings containing comma, right? Well... Just one example:

Lib/distutils/sysconfig.py:554:        warnings.warn('SO is deprecated, use EXT_SUFFIX', DeprecationWarning, 2)

You cannot only ignore the message:

$ PYTHONWARNINGS='ignore:SO is deprecated, use EXT_SUFFIX:DeprecationWarning' ./python -c 'import sys; print(sys.warnoptions); print(len(sys.warnoptions))'
Invalid -W option ignored: invalid action: 'use EXT_SUFFIX'
['ignore:SO is deprecated', ' use EXT_SUFFIX:DeprecationWarning']
2

You can only try to use "module" and "lineno" parameters of a warning filter, which are more fragile and hard to use to use.

[tirkarthi]

See also https://bugs.python.org/issue34624

[vstinner]

Sadly, the current -W option documentation is wrong :-( I started by writing PR 25439 to fix the documentation, before changing its behavior.

[vstinner]

Sadly, the current -W option documentation is wrong :-( I started by writing PR 25439 to fix the documentation, before changing its behavior.

Hum, "wrong" may be too strong. I would say that it is at least misleading.

It starts with "file:line: category: message". I understood that it was the format the option, whereas it is not. Moreover, the full format is simply... not documented at all! Only the short -Waction format :-(

https://docs.python.org/dev/using/cmdline.html#cmdoption-W

[vstinner]

New changeset 62ec638 by Victor Stinner in branch 'master':
bpo-43862: Enhance -W cmdline option documentation (GH-25439)
62ec638

[vstinner]

I also created a thread on python-dev:
https://mail.python.org/archives/list/python-dev@python.org/thread/JMKLA4RUJLTORBPPTM4BWL76VNNMKYVG/

[vstinner]

I mark this issue as a duplicate of bpo-34624.