Walrus operator support

[piotrkilczuk]

I wanted to try out the new assignment expressions aka walrus operator in Python 3.8.

It all worked fine in plain Python 3.8:

$ python
Python 3.8.0a1 (default, Feb 19 2019, 11:26:51) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> sample_data = [
...     {"userId": 1, "id": 1, "title": "delectus aut autem", "completed": False},
...     {"userId": 1, "id": 2, "title": "quis ut nam facilis", "completed": False},
...     {"userId": 1, "id": 3, "title": "fugiat veniam minus", "completed": False},
...     {"userId": 1, "id": 4, "title": "et porro tempora", "completed": True},
...     {"userId": 1, "id": 4, "title": None, "completed": True},
... ]
>>> 
>>> print("With Python 3.8 Walrus Operator:") 
With Python 3.8 Walrus Operator:
>>> for entry in sample_data: 
...     if title := entry.get("title"):
...         print(f'Found title: "{title}"')
... 
Found title: "delectus aut autem"
Found title: "quis ut nam facilis"
Found title: "fugiat veniam minus"
Found title: "et porro tempora"

But failed miserably in IPython:

$ ipython
Python 3.8.0a1 (default, Feb 19 2019, 11:26:51) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0.dev -- An enhanced Interactive Python. Type '?' for help.

In [1]: sample_data = [ 
   ...:     {"userId": 1, "id": 1, "title": "delectus aut autem", "completed": False}, 
   ...:     {"userId": 1, "id": 2, "title": "quis ut nam facilis", "completed": False}, 
   ...:     {"userId": 1, "id": 3, "title": "fugiat veniam minus", "completed": False}, 
   ...:     {"userId": 1, "id": 4, "title": "et porro tempora", "completed": True}, 
   ...:     {"userId": 1, "id": 4, "title": None, "completed": True}, 
   ...: ] 
   ...:  
   ...: print("With Python 3.8 Walrus Operator:")  
   ...: for entry in sample_data:  
   ...:     if title := entry.get("title"): 
   ...:         print(f'Found title: "{title}"') 
   ...:                                                                                                  
With Python 3.8 Walrus Operator:
---------------------------------------------------------------------------
SystemError                               Traceback (most recent call last)
/usr/local/lib/python3.8/codeop.py in __call__(self, source, filename, symbol)
    131 
    132     def __call__(self, source, filename, symbol):
--> 133         codeob = compile(source, filename, symbol, self.flags, 1)
    134         for feature in _features:
    135             if codeob.co_flags & feature.compiler_flag:

SystemError: unexpected expression

I was surpised to see that happen, as IPython 7.3.0+ should have Python 3.8 support.

$ python -c "import IPython; print(IPython.sys_info())"
{'commit_hash': '19b47fa',
 'commit_source': 'repository',
 'default_encoding': 'utf-8',
 'ipython_path': '/home/piotr/Personal/ipython/IPython',
 'ipython_version': '7.4.0.dev',
 'os_name': 'posix',
 'platform': 'Linux-4.15.0-45-generic-x86_64-with-glibc2.17',
 'sys_executable': '/home/piotr/.virtualenvs/ipython-KlOhEFsK/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.8.0a1 (default, Feb 19 2019, 11:26:51) \n'
                '[GCC 5.4.0 20160609]'}

[Carreau]

Damn, I didn't try that.

If your playing with 3.8 alpha, I'm going to assume you likely have enough skills to dig into this and find what the values of source, filename, symbol (and self.flags?) at that point. IPython do a number of source and ast transforms that are either incomplete or conflicting with 3.8.

I'm not sure I'll have time to install 3.8 and try to track that down myself.

[piotrkilczuk]

Hi @Carreau ,

I didn't know how to serialize the source to provide you with something meaningful...

source =  <_ast.Module object at 0x7f01bc3fbee0>
source.body =  [<_ast.For object at 0x7f01bc3fbc70>]
filename =  <ipython-input-4-8e5a09cadd60>
symbol =  exec
self.flags =  512

I'll see if I can find some time this week to understand the problem (I'm not familiar with ast).

[tomyun]

I also encountered this problem on IPython with Python 3.8.0a2. Seems like there is an issue with built-in compile() function can't handle NamedExpr in AST object used for representing assignment expression (a.k.a. Walrus operator).

Python 3.8.0a2+ (heads/master:06e1e68, Mar 17 2019, 14:27:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.3.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import ast

In [2]: compile("if a == 1:\n  True", '<string>', 'exec')
Out[2]: <code object <module> at 0x111075810, file "<string>", line 1>

In [3]: compile(ast.parse("if a == 1:\n  True"), '<string>', 'exec')
Out[3]: <code object <module> at 0x111075930, file "<string>", line 1>

In [4]: compile("if a := 1:\n  True", '<string>', 'exec')
Out[4]: <code object <module> at 0x1110759c0, file "<string>", line 1>

In [5]: compile(ast.parse("if a := 1:\n  True"), '<string>', 'exec')
---------------------------------------------------------------------------
SystemError                               Traceback (most recent call last)
<ipython-input-5-d255a1eb1b58> in <module>
----> 1 compile(ast.parse("if a := 1:\n  True"), '<string>', 'exec')

SystemError: unexpected expression

[tomyun]

Reported upstream: https://bugs.python.org/issue36332

[tirkarthi]

The upstream issue was fixed and the fix is available with Python 3.8 alpha 3. Verified the same on master. I think this can be closed.

(foo-venv) ⋊> ~/cpython on master ⨯ ipython                                                  18:59:03
Python 3.8.0a3+ (heads/master:2ea8099523, Apr  6 2019, 18:56:04)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: items = [{'name': 'foo'}, {}]

In [2]: for item in items:
   ...:     if name := item.get('name'):
   ...:         print(name)
   ...:
foo

[Carreau]

Closing as fixed upstream; thanks for taking care off the report and followup.

[Carreau]

Hi there, You seem to run IPython on Python 3.8; do you mind trying #11713 that tried to make use of the new 3.8b1 (not out yet) top-level await ?