Async list comprehension (PEP 530) causes SyntaxError in Python 3.6.3

[cannedrag]

BPO	31832
Nosy	@zware, @1st1

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-10-20.23:31:13.562>
created_at = <Date 2017-10-20.23:08:38.275>
labels = ['invalid', 'expert-asyncio']
title = 'Async list comprehension (PEP 530) causes SyntaxError in Python 3.6.3'
updated_at = <Date 2017-10-21.03:58:39.008>
user = 'https://bugs.python.org/cannedrag'

bugs.python.org fields:

activity = <Date 2017-10-21.03:58:39.008>
actor = 'cannedrag'
assignee = 'none'
closed = True
closed_date = <Date 2017-10-20.23:31:13.562>
closer = 'zach.ware'
components = ['asyncio']
creation = <Date 2017-10-20.23:08:38.275>
creator = 'cannedrag'
dependencies = []
files = []
hgrepos = []
issue_num = 31832
keywords = []
message_count = 6.0
messages = ['304688', '304689', '304690', '304692', '304693', '304694']
nosy_count = 3.0
nosy_names = ['zach.ware', 'yselivanov', 'cannedrag']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue31832'
versions = ['Python 3.6']

[cannedrag]

$ python3
Python 3.6.3 (default, Oct  3 2017, 21:45:48) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> async def arange(n):
...     for i in range(n):
...         yield i
... 
>>> [i async for i in arange(10)]
  File "<stdin>", line 1
    [i async for i in arange(10)]
           ^
SyntaxError: invalid syntax
>>>

[zware]

Your async listcomp must also be defined within a coroutine to turn async into a keyword in 3.6. The following is far from best practice (don't do this, I don't know what I'm doing! :), but at least it compiles and shows that it works:

async def arange(n):
    for i in range(n):
        yield i

async def alistcomp():
    return [i async for i in arange(10)]

try:
next(alistcomp().__await__())
except StopIteration as e:
value = e.value

print(value)

[cannedrag]

Zachary,

Thank you for your response. I had the impression that async comprehensions are a bridge between async functions and non-async functions. Is there a wat to use async (and asyncio) and then go back to regular python? Or am I just wishful thinking? :-)

For example, it would be nice to start multiple processes through asyncio.create_subprocess_exec (or _shell) await twice and then process all the output in "traditional" (i.e., non-async) Python.

Cheers :),
Cannedrag

[cannedrag]

I have been thinking about my previous comment a bit more. For consistency there should at least be an await somewhere to move back from async land to non-async land.

For example:

#!/usr/bin/env python3

    import asyncio

    async def main():
        cmds = [['ssh', 'user@host', 'echo {}'.format(i)] for i in range(4)]
        creations = [asyncio.create_subprocess_exec(*cmd, 
                stdout=asyncio.subprocess.PIPE, 
                stderr=asyncio.subprocess.PIPE)
                    for cmd in cmds]
        processes = [await creation for creation in creations]
        outputs = [await process.communicate() for process in processes]
        print(outputs)
    
    if __name__ == '__main__':
        event_loop = asyncio.get_event_loop()
        event_loop.run_until_complete(main())

prints

[(b'0\n', b''), (b'1\n', b''), (b'2\n', b''), (b'3\n', b'')]

It would be nice if you could somehow return outputs from main() and process the results in a non-async function. There are no async concepts left in outputs after all. But I am not aware of a way you can do this in Python currently.

[zware]

That's already provided, just do return outputs instead of print(outputs), and outputs = event_loop.run_until_complete(main()). See https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.AbstractEventLoop.run_until_complete

[cannedrag]

Thank you!