-W command line options and PYTHONWARNINGS environmental variable should not override -b / -bb command line options

[Arfrever]

BPO	20361
Nosy	@warsaw, @brettcannon, @ncoghlan, @pitrou, @vstinner, @larryhastings, @pjenvey, @jwilk, @ezio-melotti, @merwok, @briancurtin, @serhiy-storchaka, @jdufresne
PRs	bpo-32230, bpo-20361: Set sys.warnoptions for all command line options #4734 bpo-32230: Set sys.warnoptions with -X dev #4820 bpo-20361: Remove workaround for a now fixed bug #5283
Dependencies	bpo-20363: BytesWarnings triggerred by test suite
Files	python-bytes_warnings.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-12-12.22:01:13.847>
created_at = <Date 2014-01-23.09:19:49.506>
labels = ['interpreter-core', 'type-bug', '3.7']
title = '-W command line options and PYTHONWARNINGS environmental variable should not override -b / -bb command line options'
updated_at = <Date 2018-01-23.12:30:55.712>
user = 'https://bugs.python.org/Arfrever'

bugs.python.org fields:

activity = <Date 2018-01-23.12:30:55.712>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-12-12.22:01:13.847>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2014-01-23.09:19:49.506>
creator = 'Arfrever'
dependencies = ['20363']
files = ['33648']
hgrepos = []
issue_num = 20361
keywords = ['patch']
message_count = 8.0
messages = ['208894', '208898', '212095', '217439', '307792', '308157', '308158', '310500']
nosy_count = 14.0
nosy_names = ['barry', 'brett.cannon', 'ncoghlan', 'pitrou', 'vstinner', 'larry', 'pjenvey', 'jwilk', 'ezio.melotti', 'eric.araujo', 'Arfrever', 'brian.curtin', 'serhiy.storchaka', 'jdufresne']
pr_nums = ['4734', '4820', '5283']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20361'
versions = ['Python 3.7']

[Arfrever]

Currently -W command line options and PYTHONWARNINGS environmental variable non-intuitively override settings specified by -b / -bb command line options.

$ python3.4 -c 'print("" == b"")'
False
$ python3.4 -b -c 'print("" == b"")'
-c:1: BytesWarning: Comparison between bytes and string
False
$ python3.4 -bb -c 'print("" == b"")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
BytesWarning: Comparison between bytes and string
$ python3.4 -We -b -c 'print("" == b"")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
BytesWarning: Comparison between bytes and string
$ python3.4 -Wd -bb -c 'print("" == b"")'
-c:1: BytesWarning: Comparison between bytes and string
False
$ python3.4 -Wi -b -c 'print("" == b"")'
False
$ python3.4 -Wi -bb -c 'print("" == b"")'
False
$

I suggest to increase priority of -b / -bb command line options.
The attached patch should be applied after patch from issue bpo-20355.

(Test suite of Python triggers some BytesWarnings, so firstly they will have to be fixed.)

[Arfrever]

By the way, Tools/scripts/run_tests.py (used by 'test' target in Makefile) passes '-W default -bb', which currently fails work as intended (i.e. treating BytesWarnings as errors).

[serhiy-storchaka]

Final patch should remove a workaround in Tools/scripts/run_tests.py.

[pitrou]

The patch is a bit problematic, because Py_BytesWarningFlag may also be set by e.g. an application embedding Python, but then Python's main.c won't be executed.

[ncoghlan]

The approach I've taken in my PR is similar to the one that Arfrever proposed (albeit different in the details due to other changes in the startup code since that patch was written).

For the embedding case, I've simply noted in the What's New section of the porting guide that embedding applications need to configure a error::BytesWarning filter explicitly if they want to treat BytesWarning as errors (in addition to setting the flag to request that the warnings be emitted in the first place).

[vstinner]

New changeset 747f48e by Victor Stinner in branch 'master':
bpo-32230: Set sys.warnoptions with -X dev (bpo-4820)
747f48e

[vstinner]

Ok, I merged Nick's PR: -b and -bb options now have the highest priority. I close the issue.

[vstinner]

New changeset dc6b946 by Victor Stinner in branch 'master':
bpo-20361: Remove workaround for a now fixed bug (bpo-5283)
dc6b946