Deprecation warnings for the future async and await keywords in Python 3.6 #70370

marco-buttu · 2016-01-22T19:06:07Z

BPO	26182
Nosy	@brettcannon, @giampaolo, @ned-deily, @1st1, @marco-buttu, @AnishShah
Files	async_await.patch: Patch and related test issue_26182.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2016-09-15.16:51:13.036>
created_at = <Date 2016-01-22.19:06:06.579>
labels = ['interpreter-core', 'type-feature', 'release-blocker', 'expert-asyncio']
title = 'Deprecation warnings for the future async and await keywords in Python 3.6'
updated_at = <Date 2017-11-08.19:58:50.497>
user = 'https://github.com/marco-buttu'

bugs.python.org fields:

activity = <Date 2017-11-08.19:58:50.497>
actor = 'serhiy.storchaka'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2016-09-15.16:51:13.036>
closer = 'yselivanov'
components = ['Interpreter Core', 'asyncio']
creation = <Date 2016-01-22.19:06:06.579>
creator = 'marco.buttu'
dependencies = []
files = ['41908', '44669']
hgrepos = []
issue_num = 26182
keywords = ['patch']
message_count = 24.0
messages = ['258833', '259038', '259394', '259409', '259411', '259412', '259413', '259534', '259887', '259888', '259892', '260097', '260118', '260177', '260225', '276091', '276097', '276118', '276493', '276575', '276576', '276577', '276585', '280355']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'giampaolo.rodola', 'ned.deily', 'python-dev', 'yselivanov', 'marco.buttu', 'anish.shah']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26182'
versions = ['Python 3.6']

marco-buttu · 2016-01-22T19:06:06Z

I saw that async and await will become keywords in Python 3.7 :

https://www.python.org/dev/peps/pep-0492/#deprecation-plans

I enabled the deprecation warnings in Python 3.5.1 and Python 3.6 dev, and I noticed that assigning to async or await does not issue any deprecation warning:

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.5.1 (default, Jan 21 2016, 19:59:28)
[GCC 4.8.4]

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.6.0a0 (default:4b434a4770a9, Jan 12 2016, 13:01:29)
[GCC 4.8.4]

brettcannon · 2016-01-27T18:00:08Z

If someone wants to try and fix this, I would look at how the warning for the 'with' statement was handled (it will either be in the compiler while generating bytecode or somewhere in the parser, but I'm fairly certain it's the compiler).

marco-buttu · 2016-02-02T14:59:30Z

The check for the 'with' statement was handled in the parser (parsetok.c), and in Python 2.5 the warnings were enabled by default.
I do not know how to check if the deprecation warning are enabled, otherwise I would have tried to fix the problem.

brettcannon · 2016-02-02T18:10:08Z

I'm not quite sure what you mean by "do not know how to check if the deprecation warning are enabled". Do you mean how do you make sure you don't emit a DeprecationWarning if someone hasn't turned warnings on? If that's the case then if you look at some example code and read https://docs.python.org/3/c-api/exceptions.html#issuing-warnings you will notice that when you call the warning-related functions, all you have to do is check if the return value is < 0, then return like there is an exception raised. Otherwise the warnings module handles whether something will be shown.

AnishShah · 2016-02-02T18:22:04Z

I would like to work on this, if it is okay with Marco?
I look at the history of parsetok.c file and I think I can solve this issue.

Also, I have a doubt - PEP-492 says that "async and await names will be softly deprecated in CPython 3.5 and 3.6".
What exactly does "softly deprecate" mean? is it just same as throwing a warning?

brettcannon · 2016-02-02T18:35:51Z

I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.

1st1 · 2016-02-02T18:41:25Z

I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.

I actually thought about emitting DeprecationWarnings in 3.6 for async/await NAME tokens. I think it's a reasonable thing to do to prepare for 3.7, where they will become proper keywords.

marco-buttu · 2016-02-04T06:38:24Z

Thank you Brett, your explanation (msg259409) and the c-api doc are really straightforward. I think I can fix it in a few days, just the time to read the devguide and be confident about the workflow.

1st1 · 2016-02-08T21:17:21Z

Assigning the issue to myself to make sure it won't be forgotten before it's too late. Anish or Marco, feel free to propose a patch.

vstinner · 2016-02-08T21:26:51Z

Can you please mention the python version in the title?

vstinner · 2016-02-08T22:16:21Z

Oh thank. I didn't understand if you wanted to change Python 3.6 or 3.7.

marco-buttu · 2016-02-11T11:55:15Z

I added the PyErr_WarnEx(PyExc_DeprecationWarning, ...) in Python/ast.c, right below the check for None, True and False as names. Running the following test the message is properly printed:

def test_async(self):
    with self.assertWarnsRegex(DeprecationWarning, "reserved keyword"):
        async = 33

However, the test does not pass, because the DeprecationWarning is not triggered. I am sorry but as a cpython beginner I can not figure out how to solve the problem, so I hope someone else can find the time to do it

brettcannon · 2016-02-11T17:54:30Z

You need to temporarily turn on warnings for it to work. For example:

  with warnings.catch_warnings():
      warnings.simplefilter('always')
      with self.assertWarnsRegex(DeprecationWarning, "reserved keyword"):
        exec('async = 33')

Do notice I used exec() as otherwise the warning will be triggered at import instead of when the test runs.

marco-buttu · 2016-02-12T13:53:24Z

Thank you Brett, the problem was the missed exec(). With the patch in attachment the tests pass, but it does not seem to me a good solution. Infact, changing the filter at runtime has no effect:

$ cat foo.py 
import warnings

warnings.simplefilter("always")
async = 33
await = 33

$ ./python foo.py 
$

Does this happen because, putting the PyErr_WarnEx() in Python/ast.c, the warning is issued before the runtime?

Furthermore, if I set the filter from the CL, then the warning is properly triggered, but the file name and line number are wrong:

$ ./python -Wd foo.py 
sys:1: DeprecationWarning: 'async' will become a reserved keyword in Python 3.7
sys:1: DeprecationWarning: 'await' will become a reserved keyword in Python 3.7

brettcannon · 2016-02-13T04:19:07Z

Because parsing is done before execution you can't flip on warnings during runtime in the file you to be affected.

As for the line number, that's because it's raise in C code that doesn't have a trigger in Python code. Try importing the code and you should get the line number of the import. Otherwise you will have to check if there is some function to specify a syntax warning that lets you set the line number explicitly (I don't think there is).

1st1 · 2016-09-12T19:05:41Z

Ned, would it be OK to commit this patch after b1?

async/await are scheduled to become real keywords in 3.7. Right now they are only keywords in 'async def' blocks, meaning that it's OK to have a class with 'async' or 'await' attributes.

So this will be a backwards compatibility breaking change in 3.7. This patch makes Python to emit a warning each time you use async or await as an attribute/variable/etc.

ned-deily · 2016-09-12T19:33:08Z

As long as Brett is also OK with it, it can go in for 360b2.

brettcannon · 2016-09-12T21:01:22Z

I'm fine with it being in b2 because IMO the warning really should make it in 3.6 and for stuff like this it's more critical to hit the RC for people's testing than the beta to work out semantic changes.

1st1 · 2016-09-14T22:31:44Z

I had to rewrite the patch to make sure it reports correct position and covers all cases where using async/await should trigger a warning.

Brett, could you please take a look at the patch?

1st1 · 2016-09-15T16:41:52Z

I'm going to commit the patch now (I'm going on vacation tomorrow, and I want to watch the buildbots).

python-dev · 2016-09-15T16:50:58Z

New changeset 82e6017dc841 by Yury Selivanov in branch '3.6':
Issue bpo-26182: Raise DeprecationWarning for improper use of async/await keywords
https://hg.python.org/cpython/rev/82e6017dc841

New changeset 3f8b75173543 by Yury Selivanov in branch 'default':
Merge 3.6 (issue bpo-26182)
https://hg.python.org/cpython/rev/3f8b75173543

1st1 · 2016-09-15T16:51:13Z

Merged.

brettcannon · 2016-09-15T18:09:48Z

Sorry I didn't get around to reviewing; I'm sick.

On Thu, Sep 15, 2016, 09:51 Yury Selivanov <report@bugs.python.org> wrote:

Yury Selivanov added the comment:

Merged.

----------
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue26182\>

python-dev · 2016-11-08T21:54:46Z

New changeset 7a996e826f83 by Yury Selivanov in branch '3.6':
Issue bpo-26182: Fix ia refleak in code that raises DeprecationWarning.
https://hg.python.org/cpython/rev/7a996e826f83

New changeset 7b0e79e7f567 by Yury Selivanov in branch 'default':
Merge 3.6 (issue bpo-26182)
https://hg.python.org/cpython/rev/7b0e79e7f567

marco-buttu mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Jan 22, 2016

SilentGhost mannequin added the topic-asyncio label Jan 23, 2016

1st1 self-assigned this Feb 8, 2016

brettcannon changed the title ~~Deprecation warnings for the future async and await keywords~~ Deprecation warnings for the future async and await keywords in Python 3.6 Feb 8, 2016

1st1 added the release-blocker label Sep 12, 2016

1st1 closed this as completed Sep 15, 2016

ezio-melotti transferred this issue from another repository Apr 10, 2022

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

marco-buttu mannequin commented Jan 22, 2016

marco-buttu mannequin commented Jan 22, 2016

brettcannon commented Jan 27, 2016

marco-buttu mannequin commented Feb 2, 2016

brettcannon commented Feb 2, 2016

AnishShah mannequin commented Feb 2, 2016

brettcannon commented Feb 2, 2016

1st1 commented Feb 2, 2016

marco-buttu mannequin commented Feb 4, 2016

1st1 commented Feb 8, 2016

vstinner commented Feb 8, 2016

vstinner commented Feb 8, 2016

marco-buttu mannequin commented Feb 11, 2016

brettcannon commented Feb 11, 2016

marco-buttu mannequin commented Feb 12, 2016

brettcannon commented Feb 13, 2016

1st1 commented Sep 12, 2016

ned-deily commented Sep 12, 2016

brettcannon commented Sep 12, 2016

1st1 commented Sep 14, 2016

1st1 commented Sep 15, 2016

python-dev mannequin commented Sep 15, 2016

1st1 commented Sep 15, 2016

brettcannon commented Sep 15, 2016

python-dev mannequin commented Nov 8, 2016

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

Comments

marco-buttu mannequin commented Jan 22, 2016

marco-buttu mannequin commented Jan 22, 2016

brettcannon commented Jan 27, 2016

marco-buttu mannequin commented Feb 2, 2016

brettcannon commented Feb 2, 2016

AnishShah mannequin commented Feb 2, 2016

brettcannon commented Feb 2, 2016

1st1 commented Feb 2, 2016

marco-buttu mannequin commented Feb 4, 2016

1st1 commented Feb 8, 2016

vstinner commented Feb 8, 2016

vstinner commented Feb 8, 2016

marco-buttu mannequin commented Feb 11, 2016

brettcannon commented Feb 11, 2016

marco-buttu mannequin commented Feb 12, 2016

brettcannon commented Feb 13, 2016

1st1 commented Sep 12, 2016

ned-deily commented Sep 12, 2016

brettcannon commented Sep 12, 2016

1st1 commented Sep 14, 2016

1st1 commented Sep 15, 2016

python-dev mannequin commented Sep 15, 2016

1st1 commented Sep 15, 2016

brettcannon commented Sep 15, 2016

python-dev mannequin commented Nov 8, 2016