Adding PyPy support

[YannickJadoul]

There's probably still a lot to discuss on choosing defaults, option names, build tag names, ... but we can already give this a go and see if we can get the technical part working :)

Cfr. #143

[YannickJadoul]

Tests are currently failing because the version of auditwheel inside of the pypywheels/manylinux2010-pypy_x86_64 Docker image is too old (==2.0) and does not include the release that uses the AUDITWHEEL_PLAT environment variable (>=2.1), yet.

[joerick]

Wow, I can't believe this works so easily!

So the idea is that CPython extensions should be able to build without modification for pypy as well? If that's the case, I'd love to just add this to CIBW_BUILD by default and users could get pypy wheels as a bonus - it'd be easy enough to turn them off with CIBW_SKIP if they weren't desired. If the compatibility is good, we could be producing pypy wheels be accident!

[YannickJadoul]

So the idea is that CPython extensions should be able to build without modification for pypy as well?

Yes, PyPy support for the C API is now complete (though it's still not as efficient), and cffi extensions both work on CPython as well as PyPy, so PyPy seems to be making a push towards building wheels and getting them uploaded to the PyPI :)

If that's the case, I'd love to just add this to CIBW_BUILD by default and users could get pypy wheels as a bonus - it'd be easy enough to turn them off with CIBW_SKIP if they weren't desired.

Yes! Sounds good to me :)

This is just manylinux2010, right now, though (manylinux1 isn't supported by the PyPy interpreter itself, as far as I know). I still need to have a look at Windows and macOS; which might be more annoying since they don't have a nice drop-in Docker image, so we'll have to manually download and install/extract PyPy.

[mayeut]

Does it make sense to try and build macOS and/or windows wheels as well ?

[YannickJadoul]

Does it make sense to try and build macOS and/or windows wheels as well ?

Yes, it's on my TODO list. But for Windows, I was waiting for a resolution of the Windows-installation choice. And for macOS, I need to dive into the versioning of macOS (cfr. #156), since for PyPy3 the download page reads "(High Sierra >= 10.13, not for Sierra and below)".

In either case, it won't be as simple as replacing a Docker image, since I can't see .pkg packages for macOS, or find PyPy on NuGet :-/ So we'll have to download and extract, I think.

[YannickJadoul]

PyPy on Windows now also works! :)

Two remarks:

• We need a workaround for an bug in PyPy related to encodings and code pages: simple_shell(['chcp', '437']) # Workaround for https://bitbucket.org/pypy/pypy/issues/3081/
• The '{package_name}-{package_version}-pp372-pp372-win32.whl' wheel filename that's produced by PyPy 3 is inconsistent, but will get fixed in a future release.

[YannickJadoul]

@Czaki: 1bba356

[YannickJadoul]

Getting close to finishing this! :-)

The pypy/manylinux PR with wheel==0.34 was just merged.
Just waiting for pypa/virtualenv#1561 / pypa/virtualenv#1641, now!

[Czaki]

The pypy/manylinux PR with wheel==0.34 was just merged.
Just waiting for pypa/virtualenv#1561 / pypa/virtualenv#1641, now!

But if we pin virtualenv bellow 20 then it works. And if they finish then minor bugfix can be released.
@mattip is waiting for release.

[mattip]

I'm not waiting, take your time and do it right. I already released the blog post with a little caveat about cibuildwheel. It didn;t get alot of traction anyway, I guess I shouldn't post on a Sunday.

[Czaki]

missing description for ²

[YannickJadoul]

Good catch, thanks!

[YannickJadoul]

Actually, since we merged #270 separately, this PR is ready to be merged, I think! We will revert #270 separately, once there's a solution, and I've vote to hold off releasing until then, but ... I think this one is ready :-)

@joerick, anyone else, any objections left? :-)

[joerick]

Congratulations on this monumental effort @YannickJadoul! I have a few comments, they are inline.

[YannickJadoul]

@joerick, I addressed the few new remarks, I think :-)

[YannickJadoul]

Woohoooooo, merged! Thanks, everyone! :-)