keyword module missing async and await keywords

[timofurrer]

BPO	26264
Nosy	@rhettinger, @terryjreedy, @ncoghlan, @vstinner, @ezio-melotti, @vadmium, @1st1, @timofurrer
PRs	bpo-26264: Added new key words to keyword lib #1215

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-05-15.02:20:34.618>
created_at = <Date 2016-02-02.02:06:26.343>
labels = ['library']
title = 'keyword module missing async and await keywords'
updated_at = <Date 2018-05-15.02:20:34.574>
user = 'https://github.com/timofurrer'

bugs.python.org fields:

activity = <Date 2018-05-15.02:20:34.574>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2018-05-15.02:20:34.618>
closer = 'terry.reedy'
components = ['Library (Lib)']
creation = <Date 2016-02-02.02:06:26.343>
creator = 'tuxtimo'
dependencies = []
files = []
hgrepos = []
issue_num = 26264
keywords = []
message_count = 7.0
messages = ['259349', '259353', '259354', '259771', '259779', '261984', '316611']
nosy_count = 8.0
nosy_names = ['rhettinger', 'terry.reedy', 'ncoghlan', 'vstinner', 'ezio.melotti', 'martin.panter', 'yselivanov', 'tuxtimo']
pr_nums = ['1215']
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue26264'
versions = ['Python 3.5', 'Python 3.6']

[timofurrer]

I had a look at the *Lib/keyword.py* module. It seems like the auto generated *kwlist* is missing the *await* and *async* keywords.
At least according to https://www.python.org/dev/peps/pep-0492/ they are called *keywords*.

The keyword module generates the *kwlist* from *Python/graminit.c*.

[1st1]

async and await are only keywords in the context of an 'async def' function. It will be that way until Python 3.7. That said, I'm not sure what to do about the keywords module.

Nick, Victor, thoughts?

[vadmium]

What happened in cases like “with” and “yield”, when these names were turned into reserved keywords depending on a __future__ statement?

[rhettinger]

It shouldn't go into keywords.py until it is actually a keyword. Keywords is used for many purposes including syntax highlighting and code analysis tools.

[vadmium]

To answer my question, it looks like the keywords.py list is automatically generated (by running the script) from Python/graminit.c, in turn generated from Grammar/Grammar. “Yield” and “with” were always listed even when the __future__ statement was needed to enable them (2.2 and 2.5). But currently ASYNC and AWAIT are in Grammar as what I believe are special tokens, rather than normal keywords.

Depending on the purpose of the syntax highlighting or code analysis, it might be better for these to be listed as keywords now, rather than waiting until 3.7.

[ezio-melotti]

Depending on the purpose of the syntax highlighting or code analysis,
it might be better for these to be listed as keywords now, rather than
waiting until 3.7.

I agree, especially considering that it takes time before syntax highlighters update their keywords list and users update their syntax highlighters.

Would this be doable, considering that the list is generated automatically? Are there any other downsides in doing it?

[terryjreedy]

This is out-of-date for 3.5 and almost so for 3.6 and not an issue for 3.7. The PR has been closed. And I agree with Raymond that keywords.py should not anticipate the future. All this is aside from the auto-generation issue.