Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brotli extension not linked with libstdc++ on pypy3 for Linux #140

Closed
anthrotype opened this issue Aug 21, 2015 · 1 comment
Closed

brotli extension not linked with libstdc++ on pypy3 for Linux #140

anthrotype opened this issue Aug 21, 2015 · 1 comment

Comments

@anthrotype
Copy link
Member

There is an issue with the Python 3-compatible pypy (aka pypy3) whereby distutils fails to use the proper C++ compiler/linker while compiling C++ extension modules (such as Brotli).

https://bitbucket.org/pypy/pypy/issues/1763/not-using-proper-c-compilers-linker-while

In the Python2.7-compatible pypy, they have fixed this by patching distutils/unixccompiler.py so that it uses c++ as default C++ compiler instead of cc:
https://bitbucket.org/pypy/pypy/commits/c6e45dfbda905fa9e626782c8d2dd313ff3f54cf

However, they haven't ported this patch to pypy3 yet.

At behdad/fonttools, we use Brotli for WOFF2 and we test it on Travis under different python versions, including pypy3.

Because the C++ runtime library isn't being linked when compiling Brotli for pypy3, the module fails to be loaded with undefined symbol error.
For details, see: fonttools/fonttools#339

As far as I could test, this problem only occurs on pypy3 for Linux (the Travis python setup runs on Ubuntu 12.04). On OS X, where distutils.unixccompiler is also used, I verified that brotli is compiled and imported without problems when using the same pypy3 version (2.4.0) as the one used on Travis -- but I guess it's because on OSX the name cc is just a symbolic link to clang, and the latter does the right thing.

As a workaround, we currently do something like this:

python setup.py build_ext --libraries=stdc++

While pypy developers fixes the issue, Brotli's setup.py could be modified to link with libstdc++ by default whenever platform.python_implementation() == "PyPy" and sys.version_info[0] == 3. (I wonder if there would be portability issues if libstdc++ were linked all the time, on any python platform/version/implementation?)

A second alternative would be to change the value of UnixCCompiler.compiler_cxx variable from the current ['cc'] to ['c++'] if running on pypy3 (exactly like in the above mentioned pypy2 patch). This is what they do in spaCy for example (I found this by chance).

Now, both approaches seem to work, though I'm not sure which one is preferable.

A third approach, of course, is simply take note and wait.

Anyway, I have mentioned this on pypy's bug tracker, where I found at least two duplicate issues (#1099 and #1763) referencing the same problem.

Please let me know if anyone else has encountered the same problem.
Thanks,

Cosimo

@anthrotype
Copy link
Member Author

The issue with C++ and pypy3 has already been fixed here.
It should be available in the next pypy3 release.
I shall close this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant