Python 3.9 eval on list comprehension sometimes returns coroutines

[JonathanCrall]

BPO	40438
Nosy	@vstinner, @isidentical

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-29.15:12:56.500>
created_at = <Date 2020-04-29.15:02:05.023>
labels = ['interpreter-core', '3.9']
title = 'Python 3.9 eval on list comprehension sometimes returns coroutines'
updated_at = <Date 2020-04-29.17:56:14.424>
user = 'https://bugs.python.org/JonathanCrall'

bugs.python.org fields:

activity = <Date 2020-04-29.17:56:14.424>
actor = 'BTaskaya'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-29.15:12:56.500>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-04-29.15:02:05.023>
creator = 'Jonathan Crall'
dependencies = []
files = []
hgrepos = []
issue_num = 40438
keywords = []
message_count = 6.0
messages = ['367651', '367652', '367653', '367656', '367684', '367685']
nosy_count = 3.0
nosy_names = ['vstinner', 'BTaskaya', 'Jonathan Crall']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40438'
versions = ['Python 3.9']

[JonathanCrall]

I first noticed this when testing xdoctest on Python 3.9, and then again when using IPython.

I was finally able to generate a minimal working example in Python itself. The following code:

python -c "print(eval(compile('[i for i in range(3)]', mode='eval', filename='foo', flags=221184)))"

produces

[0, 1, 2]

in Python <= 3.8, but in 3.9 it produces:

<coroutine object <module> at 0x7fa336d40ec0>
<string>:1: RuntimeWarning: coroutine '<module>' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Is this an intended change? I can't find any notes in the CHANGELOG that seem to correspond to it.

[vstinner]

I first noticed this when testing xdoctest on Python 3.9, and then again when using IPython.

What is your Python 3.9 exact version number?

I cannot reproduce your issue with Python 3.9.0a6:

vstinner@apu$ ./python -c "print(eval(compile('[i for i in range(3)]', mode='eval', filename='foo', flags=221184)))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: compile(): unrecognised flags

vstinner@apu$ ./python -VV
Python 3.9.0a6+ (heads/master:9a8c1315c3, Apr 29 2020, 17:03:41)
[GCC 9.3.1 20200408 (Red Hat 9.3.1-2)]

[JonathanCrall]

Ah, sorry. I neglected all the important information.

I tested this using:

Python 3.9.0a5 (default, Apr 23 2020, 14:11:34)
[GCC 8.3.0]

Specifically, I ran in a docker container:

DOCKER_IMAGE=circleci/python:3.9-rc
docker pull $DOCKER_IMAGE
docker run --rm -it $DOCKER_IMAGE bash

And then in the bash shell in the docker image I ran:

python -c "print(eval(compile('[i for i in range(3)]', mode='eval', filename='foo', flags=221184)))"

[vstinner]

I close the issue: it's already fixed in 3.9.0a6. If it's not the case, feel free to reopen the issue ;-)

[JonathanCrall]

This can be closed, but for completeness, the test you ran didn't verify that the bug was fixed. This is because the hard coded compile flags I gave in my example seem to have changed in Python 3.9 (is this documented?).

In python3.8 the compile flags I specified correspond to division, print_function, unicode_literals, and absolute_import.

python3.8 -c "import __future__; print(future.print_function.compiler_flag | __future__.division.compiler_flag | __future__.unicode_literals.compiler_flag | __future__.absolute_import.compiler_flag)"

Results in: 221184

In Python 3.9 the same code results in: 3538944

I can modify the MWE to accommodate these changes:

./python -c "import __future__; print(eval(compile('[i for i in range(3)]', mode='eval', filename='fo', flags=future.print_function.compiler_flag | __future__.division.compiler_flag | __future__.unicode_literals.compiler_flag | __future__.absolute_import.compiler_flag)))"

Which does produce the correct output as expected. So, the issue can remain closed. I am curious what the bug in 3.9.0a5 was though if you have any speculations.

[isidentical]

This can be closed, but for completeness, the test you ran didn't verify that the bug was fixed. This is because the hard coded compile flags I gave in my example seem to have changed in Python 3.9 (is this documented?).

Yes, this is documented on What's New.

Which does produce the correct output as expected. So, the issue can remain closed. I am curious what the bug in 3.9.0a5 was though if you have any speculations.

See bpo-39562