Wheels for Python2 no longer work

[JorjMcKie]

Since a few days, binary wheels generated for Python2.7 on Travis CI do no longer work.
When importing, the error message undefined symbol: PyUnicodeUCS2_FromStringAndSize appears:

Python 2.7.17 (default, Apr 15 2020, 17:20:14)
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import fitz
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/fitz/__init__.py", line 3, in <module>
    from fitz.fitz import *
  File "/usr/local/lib/python2.7/dist-packages/fitz/fitz.py", line 18, in <module>
    from . import _fitz
ImportError: /usr/local/lib/python2.7/dist-packages/fitz/_fitz.so: undefined symbol: PyUnicodeUCS2_FromStringAndSize
>>>

The wheel generated by auditwheel is tagged cp27-cp27mu-manylinux2010_x86_64. Same happens for cp27-cp27m-manylinux2010_x86_64.
The original wheel cp27-cp27mu-linux_x86_64 (compiled on Ubuntu) does work.

Here is a full Travis CI run log:
linux-wheel.zip

[mayeut]

Does the original wheel compiled with manylinux2010 image works ?
If so can you upload the wheel somewhere ?

Since a few days

When exactly ? that could help narrow the origin of the issue.

[JorjMcKie]

PyMuPDF-1.16.18-cp27-cp27mu-manylinux2010_x86_64.whl.zip

This wheel does work. It has been created April 22.
The first failure for me occurred on May 5.

[JorjMcKie]

Does the original wheel compiled with manylinux2010 image works ?

After experiencing the problem and upon request from user s of my repo, I built a wheel for Py27 on my own machine to confirm that the error is outside this ambiente. And, as mentioned, this does work.
So I have to assume, that the intermediate cp27-cp27mu-linux_x86_64.whl created on the Travis CI servers also would work ...

[mayeut]

So I have to assume, that the intermediate cp27-cp27mu-linux_x86_64.whl created on the Travis CI servers also would work ...

I assume it doesn't that's why I'm asking. I'm trying to reproduce this. My guess is that it has to do with a lack of isolation between builds and the .so built for one ABI of cp27 is picked up when building the wheel for the other ABI.

I will confirm this shortly.

[JorjMcKie]

Ah, ok. Thanks for the info.

[JorjMcKie]

Are you interested in the Travis log from that last successful run?

[mayeut]

The issue is indeed a build isolation issue.

pip 20.1 used in the newer build introduced an in-place build which breaks isolation:
pypa/pip#7555
This has been reverted in pip 20.1.1:
pypa/pip#8221

manylinux images have already been updated to use pip 20.1.1 (manylinux2014 will be when ci will pass on s390x, anyway this issue doesn't affect manylinux2014 which can't build for cp27).

Since you're using cibuildwheel, you have 2 options:

• Pin the manylinux image to a newer image
• Update pip in the before build command

Please refer to cibuildwheel documentation in both cases.

I will open an issue in cibuildwheel with a test case that shows the issue.

[JorjMcKie]

Based on recommendation (2), it now works.
Thank your for your support.