Compilation fails on Python 3.11

[gsnedders]

With Python 3.11 (from CPython 9e05da62241):

  ERROR: Failed building wheel for typed-ast
  Running setup.py clean for typed-ast
Failed to build typed-ast
Installing collected packages: typing-extensions, toml, mypy-extensions, types-ujson, types-six, types-setuptools, types-requests, types-PyYAML, types-atomicwrites, typed-ast, mypy
    Running setup.py install for typed-ast: started
    Running setup.py install for typed-ast: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-n2i2a0we/typed-ast_17123afaf5934a17827f262904008b6d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-n2i2a0we/typed-ast_17123afaf5934a17827f262904008b6d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-oz_exif7/install-record.txt --single-version-externally-managed --compile --install-headers /mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include/site/python3.11/typed-ast
         cwd: /tmp/pip-install-n2i2a0we/typed-ast_17123afaf5934a17827f262904008b6d/
    Complete output (63 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.11
    creating build/lib.linux-x86_64-3.11/typed_ast
    copying typed_ast/conversions.py -> build/lib.linux-x86_64-3.11/typed_ast
    copying typed_ast/ast3.py -> build/lib.linux-x86_64-3.11/typed_ast
    copying typed_ast/ast27.py -> build/lib.linux-x86_64-3.11/typed_ast
    copying typed_ast/__init__.py -> build/lib.linux-x86_64-3.11/typed_ast
    package init file 'ast3/tests/__init__.py' not found (or not a regular file)
    creating build/lib.linux-x86_64-3.11/typed_ast/tests
    copying ast3/tests/test_basics.py -> build/lib.linux-x86_64-3.11/typed_ast/tests
    running build_ext
    building '_ast27' extension
    creating build/temp.linux-x86_64-3.11
    creating build/temp.linux-x86_64-3.11/ast27
    creating build/temp.linux-x86_64-3.11/ast27/Custom
    creating build/temp.linux-x86_64-3.11/ast27/Parser
    creating build/temp.linux-x86_64-3.11/ast27/Python
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Custom/typed_ast.c -o build/temp.linux-x86_64-3.11/ast27/Custom/typed_ast.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/acceler.c -o build/temp.linux-x86_64-3.11/ast27/Parser/acceler.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/bitset.c -o build/temp.linux-x86_64-3.11/ast27/Parser/bitset.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/grammar.c -o build/temp.linux-x86_64-3.11/ast27/Parser/grammar.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/grammar1.c -o build/temp.linux-x86_64-3.11/ast27/Parser/grammar1.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/node.c -o build/temp.linux-x86_64-3.11/ast27/Parser/node.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/parser.c -o build/temp.linux-x86_64-3.11/ast27/Parser/parser.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/parsetok.c -o build/temp.linux-x86_64-3.11/ast27/Parser/parsetok.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Parser/tokenizer.c -o build/temp.linux-x86_64-3.11/ast27/Parser/tokenizer.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Python/Python-ast.c -o build/temp.linux-x86_64-3.11/ast27/Python/Python-ast.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Python/asdl.c -o build/temp.linux-x86_64-3.11/ast27/Python/asdl.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Python/ast.c -o build/temp.linux-x86_64-3.11/ast27/Python/ast.o
    ast27/Python/ast.c: In function ‘ast_for_import_stmt’:
    ast27/Python/ast.c:2781:13: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
     2781 |             if (!import_alias)
          |             ^~
    In file included from ast27/Python/../Include/Python-ast.h:3,
                     from ast27/Python/ast.c:7:
    ast27/Python/../Include/../Include/asdl.h:46:31: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
       46 | #define asdl_seq_SET(S, I, V) (S)->elements[I] = (V)
          |                               ^
    ast27/Python/ast.c:2783:17: note: in expansion of macro ‘asdl_seq_SET’
     2783 |                 asdl_seq_SET(aliases, 0, import_alias);
          |                 ^~~~~~~~~~~~
    ast27/Python/ast.c:2788:17: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
     2788 |                 if (!import_alias)
          |                 ^~
    In file included from ast27/Python/../Include/Python-ast.h:3,
                     from ast27/Python/ast.c:7:
    ast27/Python/../Include/../Include/asdl.h:46:31: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
       46 | #define asdl_seq_SET(S, I, V) (S)->elements[I] = (V)
          |                               ^
    ast27/Python/ast.c:2790:21: note: in expansion of macro ‘asdl_seq_SET’
     2790 |                     asdl_seq_SET(aliases, i / 2, import_alias);
          |                     ^~~~~~~~~~~~
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Python/graminit.c -o build/temp.linux-x86_64-3.11/ast27/Python/graminit.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=native -O3 -pipe -fno-plt -march=x86-64 -mtune=native -O3 -pipe -fno-plt -fPIC -Iast27/Include -I/mnt/home-extra/gsnedders/Documents/other-projects/wpt/web-platform-tests/tools/.tox/py311-mypy/include -I/usr/include/python3.11 -c ast27/Python/mystrtoul.c -o build/temp.linux-x86_64-3.11/ast27/Python/mystrtoul.o
    ast27/Python/mystrtoul.c: In function ‘Ta27OS_strtoul’:
    ast27/Python/mystrtoul.c:112:21: error: ‘_PyLong_DigitValue’ undeclared (first use in this function)
      112 |                 if (_PyLong_DigitValue[Py_CHARMASK(str[1])] >= 16) {
          |                     ^~~~~~~~~~~~~~~~~~
    ast27/Python/mystrtoul.c:112:21: note: each undeclared identifier is reported only once for each function it appears in
    error: command '/usr/bin/gcc' failed with exit code 1

See python/cpython@5f09bb0 for this getting moved to an internal API

[srittau]

Is typed_ast still required for Python 3.11? As far as I'm aware, Python's ast module now has all the features of typed_ast, and this can be solved with a conditional import. To me, fixing typed_ast for Python 3.11 has no priority, although I'd sure consider a PR.

[ethanhs]

The only reason would be to type check/lint Python 2 code, but I doubt there are many use cases where someone a) wants to type check/lint Python 2 and b) wants to use Python 3.11 to do so