Add a default filter for DeprecationWarning in __main__

[ncoghlan]

BPO	31975
Nosy	@gvanrossum, @warsaw, @ncoghlan, @pitrou, @merwok, @alex, @bitdancer, @kevinoid
PRs	bpo-31975 (PEP 565): Show DeprecationWarning in __main__ #4458 bpo-42272: fix misleading warning filter message/module docs #23172

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2018-01-08.02:45:38.174>
created_at = <Date 2017-11-08.02:55:09.027>
labels = ['type-feature', '3.7']
title = 'Add a default filter for DeprecationWarning in __main__'
updated_at = <Date 2022-03-12.06:32:30.191>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2022-03-12.06:32:30.191>
actor = 'kevinoid'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2018-01-08.02:45:38.174>
closer = 'ncoghlan'
components = []
creation = <Date 2017-11-08.02:55:09.027>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 31975
keywords = ['patch']
message_count = 34.0
messages = ['305801', '305804', '305812', '305835', '305925', '305926', '305928', '305929', '305930', '305940', '305945', '305967', '305968', '306003', '306073', '306081', '306096', '306097', '306098', '306099', '306102', '306103', '306105', '306106', '306107', '306109', '306110', '306125', '306131', '306134', '306135', '306136', '309618', '309658']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'barry', 'ncoghlan', 'pitrou', 'eric.araujo', 'alex', 'r.david.murray', 'kevinoid']
pr_nums = ['4458', '23172']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31975'
versions = ['Python 3.7']

[ncoghlan]

As per the post at https://mail.python.org/pipermail/python-dev/2017-November/150366.html, this is an RFE to cover adding a new warning filter to the default set:

once::DeprecationWarning:__main__

This means that all deprecation warnings triggered directly by code in __main__ will start being reported again, while deprecation warnings triggered by imported modules will continue to be ignored by default.

Thus the following will start emitting DeprecationWarning by default again (as they did in 2.6 and earlier):

• experiments at the REPL
• directly executed scripts
• modules executed with the -m switch
• pkg.__main__ submodules executed with the -m switch
• __main__.py files in executed directories and zip archives

However, any code *imported* from these will not trigger warnings.

This means that the following still won't trigger any warnings by default:

• modules imported & functions called from entry point wrapper scripts
• modules imported & functions called from pkg.__main__ submodules
• modules imported & functions called from __main__.py files

The intent behind the change is that it should be much harder for developers and educators to miss seeing a deprecation warning at least once for a deprecated API, while still silencing deprecation warnings by default for deployed Python applications.

[merwok]

This can’t be backported, but could the docs of 2.7 and stable 3.x version gain an example of equivalent PYTHONWARNINGS envvar?

[pitrou]

Thus the following will start emitting DeprecationWarning by default again (as they did in 2.6 and earlier):

• experiments at the REPL

This is backwards. If there's one place where you *don't* want stderr polluted with warnings (or any kind of logging), it's an interactive prompt.

More generally, I think this entire proposal is unsound, and needs a PEP written to make its case clearly.

[bitdancer]

From the linked email:

That way ad hoc scripts and the REPL will get warnings by default,
while zipapps and packages can avoid warnings by keeping their
__main__.py simple, and importing a CLI helper function from another
module. Entry point wrapper scripts will implicitly have the same
effect for installed packages.

But a lot of non-ad-hoc scripts consist of a single __main__ module, and this will produce warnings in those. Weren't those kind of scripts one of the motivators for not enabling deprecation warnings by default? I seem to remember that's where they annoyed me most, but it has been a long time (thankfully).

When we had warnings by default we got lots of complaints. Since we turned them off and turned them back on in unittest (and other test packages followed suit), we have not had complaints that I remember hearing before this thread, except about them not appearing at the REPL. Are there real-world (as opposed to theoretical) instances of the current policy causing real problems?

[ncoghlan]

Yes - stuff just flat out breaks when Linux distros upgrade Python.

The response is "Yeah, python-dev decided years ago that they don't care about that".

[ncoghlan]

(And yes, I could push this as a downstream patch to Fedora's Python packages instead - if enough folks insist that springing breaking changes on people without warning them that those changes coming is fine, that's what I'll do)

[gvanrossum]

The response is "Yeah, python-dev decided years ago that they don't care about that".

That's a little harsh don't you think?

[ncoghlan]

Sure, it's harsh, but it's still what it looks like from the outside - there's no PEP explaining the reasoning for putting the interests of app developers that don't want to take responsibility for their user experience ahead of those of regular Python developers, and if you try to suggest that "Hey, maybe this was a mistake", you get a deluge of people yelling at you "No, we can't possibly revisit that decision".

And this is with *me* suggesting it - let alone if someone that wasn't already an experienced core dev bringing it up.

[bitdancer]

It was never about putting the interests of the app developers first, it was about putting the interests of the users first: not being pestered by seeing deprecation warnings they can't do anything about.

So yeah, maybe a PEP explaining the logic would be helpful, since there seems to be disagreement about what the logic is. I'm not in a position to write it, though.

Maybe it would indeed be worth it to push this change into Fedora/RedHat and see what the response is before doing it in core? Kind of the same principle as putting something on pypi before putting in the stdlib.

Also note that *no matter what you do* adding keywords is painful. That's why we try really really hard not to do it.

[ncoghlan]

I'm currently still too annoyed to write a PEP about this - it would probably come out in tone like the state of the unicode literals PEP before I edited it.

I just don't understand how this logic is hard: *our* primary responsibility for deprecation warnings is to ensure that Python developers (including us) can alert other *Python developers* as to upcoming API breakages.

Yet somehow this primary goal has been subordinated to "app developers are bad at managing whether or not deprecation warnings are shown to their users, so it's OK that we ended up breaking deprecation warnings in general for everyone that doesn't run a test suite, even though we didn't realise that's what we were doing at the time we made the decision".

And I'll repeat: I'm entirely happy with the "revert the change for __main__ only" compromise, since it covers all the cases I care about.

If folks are also running third party single file scripts, and those scripts are emitting warnings, and they really hate seeing those warnings, then they can put "PYTHONWARNINGS=ignore::DeprecationWarning" into their shell profile.

[pitrou]

And I'll repeat: I'm entirely happy with the "revert the change for __main__ only" compromise, since it covers all the cases I care about.

And that's what I'm unhappy with. We should either revert the change for all code, or not revert it at all.

Such a middle ground is just a wishy-washy decision-by-committee compromise. It will satisfy almost nobody (neither the people who don't want to see deprecation warnings, nor the ones who don't to miss them), is more complicated to remember, and will most certainly have to be revisited in a couple of years.

[gvanrossum]

Such a middle ground is just a wishy-washy decision-by-committee compromise.

Fine, but since *I* won't tolerate these warnings on for everything, this is the best you can do. If you don't like it, the status quo wins.

[pitrou]

Then let the statu quo wins!