It seems that the operator ':=' in python3.8 is not supported

[simon-zy]

For example, a file named as test.py is as follow:

import os
a=[1,2,3,4]
if (n:=len(a))>2:
    print()

when you use yapf test.py to format it, error occurs:

Traceback (most recent call last):
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 115, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/lib/python3.8/lib2to3/pgen2/driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/lib/python3.8/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/lib/python3.8/lib2to3/pgen2/parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=11, value=':', context=('', (4, 5))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/virtualenvs/limbov2/bin/yapf", line 8, in <module>
    sys.exit(run_main())
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/__init__.py", line 326, in run_main
    sys.exit(main(sys.argv))
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/__init__.py", line 204, in main
    changed = FormatFiles(
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/__init__.py", line 262, in FormatFiles
    changed |= _FormatFile(filename, lines, style_config, no_local_style,
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/__init__.py", line 282, in _FormatFile
    reformatted_code, encoding, has_change = yapf_api.FormatFile(
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/yapflib/yapf_api.py", line 85, in FormatFile
    reformatted_source, changed = FormatCode(
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/yapflib/yapf_api.py", line 129, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "/data/virtualenvs/limbov2/lib/python3.8/site-packages/yapf/yapflib/pytree_utils.py", line 121, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/lib/python3.8/lib2to3/pgen2/driver.py", line 103, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/lib/python3.8/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/lib/python3.8/lib2to3/pgen2/parse.py", line 162, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=11, value=':', context=('', (4, 5))

When will yapf support ':=' operator?

[bwendling]

YAPF uses lib2to3, which is shipped with Python. If that doesn't support something, then we can't. :-(

[gainskills]

YAPF uses lib2to3, which is shipped with Python. If that doesn't support something, then we can't. :-(

#856 (comment)

since the Change was merged on 2020-12-14 17:10:37, YAPF should work in the next version (Python).

[gainskills]

@HungChien r u using Py3.8.7?

if not, as
bpo-36541: Fixed lib2to3.pgen2 to be able to parse PEP-570 positional only argument syntax. already included in the version.
could you please try with it?
Thanks,

[kamahen]

There are a number of updates to lib2to3, depending on which version of Python you're using. This broke some of my code ... I've solved the versioning problem by taking a snapshot of lib2to3.pygram.Grammar.txt and checking its sha1 chksum:

# Verify that the Grammar.txt file matches what we expect.
# The check is generated by this command: sha1sum pygram/Grammar.txt
# (which is copied from /usr/lib/python3.7/lib2to3/Grammar.txt)
with open(pygram._GRAMMAR_FILE, 'rb') as grammar_file:
    if (hashlib.sha1(grammar_file.read()).hexdigest() !=
                '2713b32f014ddd6268ed4396ee95f960123e6e7d'):
        raise RuntimeError(
                'lib2to3 grammar file (%s) is an unexpected version' % pygram._GRAMMAR_FILE)

[Spitfire1900]

This does not reproduce as far back as at least 0.30.0. (tested on python 3.10) and I think this issue may be closed.

> yapf --diff .\yapf#894.py
--- .\yapf#894.py       (original)
+++ .\yapf#894.py       (reformatted)
@@ -1,4 +1,5 @@
 import os
-a=[1,2,3,4]
-if (n:=len(a))>2:
+
+a = [1, 2, 3, 4]
+if (n := len(a)) > 2:
     print()