Add PyPy Support + tests by fohrloop · Pull Request #393 · wakepy/wakepy

fohrloop · 2024-09-20T06:54:12Z

Closes: #274

Add tests for PyPy in the GitHub Actions
Replaced time-machine with mocked datetime.now() to make it possible to run tests with PyPy
Update documentation

Note: It's likely that all versions of wakepy have been running well with PyPy. This is just the official rubber stamp as there's now also CI tests for PyPy.

fohrloop · 2024-09-20T07:00:52Z

Otherwise looks good but pypy3.7 is failing with

mypy: install_deps> failing with pypy3.7

mypy: install_deps> python -I -m pip install -r /home/runner/work/wakepy/wakepy/requirements/requirements-mypy.txt
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [258 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-pypy37
      creating build/lib.linux-x86_64-pypy37/typed_ast
      copying typed_ast/ast3.py -> build/lib.linux-x86_64-pypy37/typed_ast
      copying typed_ast/ast27.py -> build/lib.linux-x86_64-pypy37/typed_ast
      copying typed_ast/__init__.py -> build/lib.linux-x86_64-pypy37/typed_ast
      copying typed_ast/conversions.py -> build/lib.linux-x86_64-pypy37/typed_ast
      creating build/lib.linux-x86_64-pypy37/typed_ast/tests
      copying ast3/tests/test_basics.py -> build/lib.linux-x86_64-pypy37/typed_ast/tests
      running build_ext
      building '_ast27' extension
      creating build/temp.linux-x86_64-pypy37
      creating build/temp.linux-x86_64-pypy37/ast27
      creating build/temp.linux-x86_64-pypy37/ast27/Custom
      creating build/temp.linux-x86_64-pypy37/ast27/Parser
      creating build/temp.linux-x86_64-pypy37/ast27/Python
      gcc -pthread -DNDEBUG -O2 -fPIC -Iast27/Include -I/home/runner/work/wakepy/wakepy/.tox/mypy/include -I/opt/hostedtoolcache/PyPy/3.7.13/x64/include -c ast27/Custom/typed_ast.c -o build/temp.linux-x86_64-pypy37/ast27/Custom/typed_ast.o
      In file included from ast27/Custom/../Include/Python-ast.h:3,
                       from ast27/Custom/typed_ast.c:2:
      ast27/Custom/../Include/../Include/asdl.h:36:41: error: unknown type name ‘PyArena’
         36 | asdl_seq *asdl_seq_new(Py_ssize_t size, PyArena *arena);
            |                                         ^~~~~~~
      ast27/Custom/../Include/../Include/asdl.h:37:49: error: unknown type name ‘PyArena’
         37 | asdl_int_seq *asdl_int_seq_new(Py_ssize_t size, PyArena *arena);
            |                                                 ^~~~~~~
      In file included from ast27/Custom/typed_ast.c:2:
      ast27/Custom/../Include/Python-ast.h:398:63: error: unknown type name ‘PyArena’
        398 | mod_ty _Ta27_Module(asdl_seq * body, asdl_seq * type_ignores, PyArena *arena);
            |                                                               ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:400:43: error: unknown type name ‘PyArena’
        400 | mod_ty _Ta27_Interactive(asdl_seq * body, PyArena *arena);
            |                                           ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:402:39: error: unknown type name ‘PyArena’
        402 | mod_ty _Ta27_Expression(expr_ty body, PyArena *arena);
            |                                       ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:404:65: error: unknown type name ‘PyArena’
        404 | mod_ty _Ta27_FunctionType(asdl_seq * argtypes, expr_ty returns, PyArena *arena);
            |                                                                 ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:406:37: error: unknown type name ‘PyArena’
        406 | mod_ty _Ta27_Suite(asdl_seq * body, PyArena *arena);
            |                                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:409:92: error: unknown type name ‘PyArena’
        409 |                           decorator_list, string type_comment, int lineno, int col_offset, PyArena
            |                                                                                            ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:413:68: error: unknown type name ‘PyArena’
        413 |                        decorator_list, int lineno, int col_offset, PyArena *arena);
            |                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:415:65: error: unknown type name ‘PyArena’
        415 | stmt_ty _Ta27_Return(expr_ty value, int lineno, int col_offset, PyArena *arena);
            |                                                                 ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:417:70: error: unknown type name ‘PyArena’
        417 | stmt_ty _Ta27_Delete(asdl_seq * targets, int lineno, int col_offset, PyArena *arena);
            |                                                                      ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:420:34: error: unknown type name ‘PyArena’
        420 |                      col_offset, PyArena *arena);
            |                                  ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:423:25: error: unknown type name ‘PyArena’
        423 |                         PyArena *arena);
            |                         ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:425:91: error: unknown type name ‘PyArena’
        425 | stmt_ty _Ta27_Print(expr_ty dest, asdl_seq * values, bool nl, int lineno, int col_offset, PyArena
            |                                                                                           ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:429:61: error: unknown type name ‘PyArena’
        429 |                   type_comment, int lineno, int col_offset, PyArena *arena);
            |                                                             ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:432:21: error: unknown type name ‘PyArena’
        432 |                     PyArena *arena);
            |                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:435:18: error: unknown type name ‘PyArena’
        435 |                  PyArena *arena);
            |                  ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:438:62: error: unknown type name ‘PyArena’
        438 |                    type_comment, int lineno, int col_offset, PyArena *arena);
            |                                                              ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:440:92: error: unknown type name ‘PyArena’
        440 | stmt_ty _Ta27_Raise(expr_ty type, expr_ty inst, expr_ty tback, int lineno, int col_offset, PyArena
            |                                                                                            ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:444:37: error: unknown type name ‘PyArena’
        444 |                         col_offset, PyArena *arena);
            |                                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:446:93: error: unknown type name ‘PyArena’
        446 | stmt_ty _Ta27_TryFinally(asdl_seq * body, asdl_seq * finalbody, int lineno, int col_offset, PyArena
            |                                                                                             ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:449:77: error: unknown type name ‘PyArena’
        449 | stmt_ty _Ta27_Assert(expr_ty test, expr_ty msg, int lineno, int col_offset, PyArena *arena);
            |                                                                             ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:451:68: error: unknown type name ‘PyArena’
        451 | stmt_ty _Ta27_Import(asdl_seq * names, int lineno, int col_offset, PyArena *arena);
            |                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:454:38: error: unknown type name ‘PyArena’
        454 |                          col_offset, PyArena *arena);
            |                                      ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:457:20: error: unknown type name ‘PyArena’
        457 |                    PyArena *arena);
            |                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:459:68: error: unknown type name ‘PyArena’
        459 | stmt_ty _Ta27_Global(asdl_seq * names, int lineno, int col_offset, PyArena *arena);
            |                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:461:63: error: unknown type name ‘PyArena’
        461 | stmt_ty _Ta27_Expr(expr_ty value, int lineno, int col_offset, PyArena *arena);
            |                                                               ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:463:48: error: unknown type name ‘PyArena’
        463 | stmt_ty _Ta27_Pass(int lineno, int col_offset, PyArena *arena);
            |                                                ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:465:49: error: unknown type name ‘PyArena’
        465 | stmt_ty _Ta27_Break(int lineno, int col_offset, PyArena *arena);
            |                                                 ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:467:52: error: unknown type name ‘PyArena’
        467 | stmt_ty _Ta27_Continue(int lineno, int col_offset, PyArena *arena);
            |                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:469:83: error: unknown type name ‘PyArena’
        469 | expr_ty _Ta27_BoolOp(boolop_ty op, asdl_seq * values, int lineno, int col_offset, PyArena *arena);
            |                                                                                   ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:472:21: error: unknown type name ‘PyArena’
        472 |                     PyArena *arena);
            |                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:474:83: error: unknown type name ‘PyArena’
        474 | expr_ty _Ta27_UnaryOp(unaryop_ty op, expr_ty operand, int lineno, int col_offset, PyArena *arena);
            |                                                                                   ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:476:83: error: unknown type name ‘PyArena’
        476 | expr_ty _Ta27_Lambda(arguments_ty args, expr_ty body, int lineno, int col_offset, PyArena *arena);
            |                                                                                   ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:478:93: error: unknown type name ‘PyArena’
        478 | expr_ty _Ta27_IfExp(expr_ty test, expr_ty body, expr_ty orelse, int lineno, int col_offset, PyArena
            |                                                                                             ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:481:84: error: unknown type name ‘PyArena’
        481 | expr_ty _Ta27_Dict(asdl_seq * keys, asdl_seq * values, int lineno, int col_offset, PyArena *arena);
            |                                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:483:64: error: unknown type name ‘PyArena’
        483 | expr_ty _Ta27_Set(asdl_seq * elts, int lineno, int col_offset, PyArena *arena);
            |                                                                ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:485:88: error: unknown type name ‘PyArena’
        485 | expr_ty _Ta27_ListComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, PyArena
            |                                                                                        ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:488:87: error: unknown type name ‘PyArena’
        488 | expr_ty _Ta27_SetComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, PyArena
            |                                                                                       ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:492:36: error: unknown type name ‘PyArena’
        492 |                        col_offset, PyArena *arena);
            |                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:494:92: error: unknown type name ‘PyArena’
        494 | expr_ty _Ta27_GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, PyArena
            |                                                                                            ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:497:64: error: unknown type name ‘PyArena’
        497 | expr_ty _Ta27_Yield(expr_ty value, int lineno, int col_offset, PyArena *arena);
            |                                                                ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:500:35: error: unknown type name ‘PyArena’
        500 |                       col_offset, PyArena *arena);
            |                                   ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:503:56: error: unknown type name ‘PyArena’
        503 |                    kwargs, int lineno, int col_offset, PyArena *arena);
            |                                                        ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:505:63: error: unknown type name ‘PyArena’
        505 | expr_ty _Ta27_Repr(expr_ty value, int lineno, int col_offset, PyArena *arena);
            |                                                               ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:507:57: error: unknown type name ‘PyArena’
        507 | expr_ty _Ta27_Num(object n, int lineno, int col_offset, PyArena *arena);
            |                                                         ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:509:70: error: unknown type name ‘PyArena’
        509 | expr_ty _Ta27_Str(string s, string kind, int lineno, int col_offset, PyArena *arena);
            |                                                                      ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:512:37: error: unknown type name ‘PyArena’
        512 |                         col_offset, PyArena *arena);
            |                                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:515:37: error: unknown type name ‘PyArena’
        515 |                         col_offset, PyArena *arena);
            |                                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:517:84: error: unknown type name ‘PyArena’
        517 | expr_ty _Ta27_Name(identifier id, expr_context_ty ctx, int lineno, int col_offset, PyArena *arena);
            |                                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:519:86: error: unknown type name ‘PyArena’
        519 | expr_ty _Ta27_List(asdl_seq * elts, expr_context_ty ctx, int lineno, int col_offset, PyArena
            |                                                                                      ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:522:87: error: unknown type name ‘PyArena’
        522 | expr_ty _Ta27_Tuple(asdl_seq * elts, expr_context_ty ctx, int lineno, int col_offset, PyArena
            |                                                                                       ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:525:25: error: unknown type name ‘PyArena’
        525 | slice_ty _Ta27_Ellipsis(PyArena *arena);
            |                         ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:527:66: error: unknown type name ‘PyArena’
        527 | slice_ty _Ta27_Slice(expr_ty lower, expr_ty upper, expr_ty step, PyArena *arena);
            |                                                                  ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:529:42: error: unknown type name ‘PyArena’
        529 | slice_ty _Ta27_ExtSlice(asdl_seq * dims, PyArena *arena);
            |                                          ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:531:37: error: unknown type name ‘PyArena’
        531 | slice_ty _Ta27_Index(expr_ty value, PyArena *arena);
            |                                     ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:533:84: error: unknown type name ‘PyArena’
        533 | comprehension_ty _Ta27_comprehension(expr_ty target, expr_ty iter, asdl_seq * ifs, PyArena *arena);
            |                                                                                    ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:536:50: error: unknown type name ‘PyArena’
        536 |                                      col_offset, PyArena *arena);
            |                                                  ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:539:66: error: unknown type name ‘PyArena’
        539 |                              defaults, asdl_seq * type_comments, PyArena *arena);
            |                                                                  ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:541:57: error: unknown type name ‘PyArena’
        541 | keyword_ty _Ta27_keyword(identifier arg, expr_ty value, PyArena *arena);
            |                                                         ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:543:58: error: unknown type name ‘PyArena’
        543 | alias_ty _Ta27_alias(identifier name, identifier asname, PyArena *arena);
            |                                                          ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:545:57: error: unknown type name ‘PyArena’
        545 | type_ignore_ty _Ta27_TypeIgnore(int lineno, string tag, PyArena *arena);
            |                                                         ^~~~~~~
      ast27/Custom/../Include/Python-ast.h:548:39: error: unknown type name ‘PyArena’
        548 | mod_ty Ta27AST_obj2mod(PyObject* ast, PyArena* arena, int mode);
            |                                       ^~~~~~~
      In file included from ast27/Custom/typed_ast.c:1:
      ast27/Custom/../Include/compile.h:12:12: error: unknown type name ‘PyFutureFeatures’
         12 | PyAPI_FUNC(PyFutureFeatures *) PyFuture_FromAST(struct _mod *, const char *);
            |            ^~~~~~~~~~~~~~~~
      /opt/hostedtoolcache/PyPy/3.7.13/x64/include/Python.h:15:67: note: in definition of macro ‘PyAPI_FUNC’
         15 | # define PyAPI_FUNC(RTYPE) __attribute__((visibility("default"))) RTYPE
            |                                                                   ^~~~~
      In file included from ast27/Custom/typed_ast.c:7:
      ast27/Custom/../Include/ast.h:8:49: error: unknown type name ‘PyArena’
          8 |                                   const char *, PyArena *);
            |                                                 ^~~~~~~
      ast27/Custom/typed_ast.c: In function ‘source_as_string’:
      ast27/Custom/typed_ast.c:24:25: error: ‘PyCF_IGNORE_COOKIE’ undeclared (first use in this function); did you mean ‘PyPARSE_IGNORE_COOKIE’?
         24 |         cf->cf_flags |= PyCF_IGNORE_COOKIE;
            |                         ^~~~~~~~~~~~~~~~~~
            |                         PyPARSE_IGNORE_COOKIE
      ast27/Custom/typed_ast.c:24:25: note: each undeclared identifier is reported only once for each function it appears in
      ast27/Custom/typed_ast.c: In function ‘PARSER_FLAGS’:
      ast27/Custom/typed_ast.c:73:27: error: ‘PyCF_IGNORE_COOKIE’ undeclared (first use in this function); did you mean ‘PyPARSE_IGNORE_COOKIE’?
         73 |     if (flags->cf_flags & PyCF_IGNORE_COOKIE)
            |                           ^~~~~~~~~~~~~~~~~~
            |                           PyPARSE_IGNORE_COOKIE
      ast27/Custom/typed_ast.c: At top level:
      ast27/Custom/typed_ast.c:210:54: error: unknown type name ‘PyArena’
        210 |                              PyCompilerFlags *flags, PyArena *arena)
            |                                                      ^~~~~~~
      ast27/Custom/typed_ast.c: In function ‘string_object_to_py_ast’:
      ast27/Custom/typed_ast.c:244:5: error: unknown type name ‘PyArena’
        244 |     PyArena *arena = PyArena_New();
            |     ^~~~~~~
      ast27/Custom/typed_ast.c:244:22: warning: implicit declaration of function ‘PyArena_New’ [-Wimplicit-function-declaration]
        244 |     PyArena *arena = PyArena_New();
            |                      ^~~~~~~~~~~
      ast27/Custom/typed_ast.c:244:22: warning: initialization of ‘int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
      ast27/Custom/typed_ast.c:248:11: warning: implicit declaration of function ‘string_object_to_c_ast’; did you mean ‘string_object_to_py_ast’? [-Wimplicit-function-declaration]
        248 |     mod = string_object_to_c_ast(str, filename, start, flags, arena);
            |           ^~~~~~~~~~~~~~~~~~~~~~
            |           string_object_to_py_ast
      ast27/Custom/typed_ast.c:248:9: warning: assignment to ‘mod_ty’ {aka ‘struct _mod *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
        248 |     mod = string_object_to_c_ast(str, filename, start, flags, arena);
            |         ^
      ast27/Custom/typed_ast.c:250:9: warning: implicit declaration of function ‘PyArena_Free’; did you mean ‘PyMem_Free’? [-Wimplicit-function-declaration]
        250 |         PyArena_Free(arena);
            |         ^~~~~~~~~~~~
            |         PyMem_Free
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

This is coming from the requirements/requirements-mypy.txt (install mypy 1.4.1):

mypy==1.4.1; python_version=='3.7'

The mypy 1.4.1 then has this in it's setup.py:

    install_requires=[
        "typed_ast >= 1.4.0, < 2; python_version<'3.8'",
        ...
    ],

As far as I understand, the typed-ast is not supported in PyPy 3.7:

This means that the CI pipelines (with mypy) cannot be setup with PyPy 3.7. The lowest version which wakepy can officially fully support (with all the CI pipelines tests) is PyPy 3.8, but PyPy 3.7 probably works as well, or at least the test suite did run correctly on Python 3.7.10[pypy-7.3.4-final] on my local machine (w/o mypy; just pytest).

I'm going to change the pipelines to run on PyPy 3.8 and PyPy 3.10

The pypy3.7 does not support typed-ast which is a requirement in the mypy version supporting Python 3.7

fohrloop · 2024-09-20T08:27:13Z

Tested on Windows 10 + PyPy 7.3.17 (Python 3.10.14) and it works as expected.

Closes: #274 Amends: #393 The CLI spinner should show ⢎⡡ but PyPy has problems with unicode characters on Windows (printed as ÔóÄÔí░). Therefore, using the old ["|", "/", "-", "\\"] spinner when on PyPy+Windows.

fohrloop added 4 commits September 19, 2024 23:38

add pypy3.7 adn pypy3.10 to tox.ini

afc2521

replace time-machine with unittest mock

a7b5e3f

refactor the tests

0cb0f32

add pypy to gh actions test matrix

afdebb2

fohrloop added 3 commits September 20, 2024 10:30

Replace pypy3.7 with pypy3.8 in CI pipelines

6431127

The pypy3.7 does not support typed-ast which is a requirement in the mypy version supporting Python 3.7

update docs about PyPy

cc61939

update DEV.md for PyPy

d0507b1

fohrloop merged commit b8ebaf1 into main Sep 20, 2024

fohrloop deleted the issue-274-pypy-suport branch September 20, 2024 11:24

fohrloop mentioned this pull request Sep 20, 2024

Fix CLI spinner on PyPy on Windows #396

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PyPy Support + tests#393

Add PyPy Support + tests#393
fohrloop merged 7 commits intomainfrom
issue-274-pypy-suport

fohrloop commented Sep 20, 2024 •

edited

Loading

Uh oh!

fohrloop commented Sep 20, 2024 •

edited

Loading

Uh oh!

fohrloop commented Sep 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

fohrloop commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fohrloop commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fohrloop commented Sep 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fohrloop commented Sep 20, 2024 •

edited

Loading

fohrloop commented Sep 20, 2024 •

edited

Loading