Compiler "'await' outside function" error message is unreachable

[tonybaloney]

BPO	30029
Nosy	@tonybaloney
PRs	bpo-30029: SyntaxError: 'await' outside function was unreachable #1069

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-04-10.01:04:13.935>
created_at = <Date 2017-04-09.23:26:16.886>
labels = ['3.7']
title = 'Compiler "\'await\' outside function" error message is unreachable'
updated_at = <Date 2017-04-10.01:04:13.934>
user = 'https://github.com/tonybaloney'

bugs.python.org fields:

activity = <Date 2017-04-10.01:04:13.934>
actor = 'anthonypjshaw'
assignee = 'none'
closed = True
closed_date = <Date 2017-04-10.01:04:13.935>
closer = 'anthonypjshaw'
components = []
creation = <Date 2017-04-09.23:26:16.886>
creator = 'anthonypjshaw'
dependencies = []
files = []
hgrepos = []
issue_num = 30029
keywords = []
message_count = 2.0
messages = ['291398', '291399']
nosy_count = 1.0
nosy_names = ['anthonypjshaw']
pr_nums = ['1069']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30029'
versions = ['Python 3.7']

[tonybaloney]

This is related to bpo-26188,

Using await in a simple statement (outside of an async def method) raises SyntaxError with the unhelpful message "invalid syntax".

It seems obvious once you've read PEP-492 in detail, but I think that as more and more developers use async/await this will stump lots of people.

I've been trying to pick apart where this constraint is raised to see whether I can help with a PR, I've been through Grammar, Parser and then the AST and Compiler.

Looking at https://github.com/python/cpython/blob/master/Python/compile.c#L4307-L4319 I can see there are helpful error messages, but during the tokenizer phase it checks that the syntax cannot be used unless you're within an async def method

cpython/Parser/tokenizer.c

Lines 1574 to 1583 in 6f0eb93

	if (ahead_tok_kind == NAME
	&& ahead_tok.cur - ahead_tok.start == 3
	&& memcmp(ahead_tok.start, "def", 3) == 0)
	{
	/* The next token is going to be 'def', so instead of
	returning 'async' NAME token, we return ASYNC. */
	tok->async_def_indent = tok->indent;
	tok->async_def = 1;
	return ASYNC;
	}

I can't reproduce this in a REPL for 3.7.0a0, it never gets any further than the grammar or tokenizer phase.

[tonybaloney]

yey! I figured it out!!