Avoid loading libpython.so if Python symbols are already loaded globally

[benmoran]

I've been trying to resolve JuliaPy/pyjulia#38 . On Ubuntu 14.04, Python is statically linked. So when we import PyCall in a Julia interpreter running inside a Python process, as we do with PyJulia, we have to avoid dlopen'ing libpython2.7.so - otherwise we end up with two Python interpreter states and bad things ensue.

The previous version used to handle this by using the one-argument cglobal call to effectively dlopen(NULL) and get handle to the global namespace - I've restored that behaviour here. It feels more complicated, perhaps because of changes that support precompilation in the current version - maybe there is a more elegant way.

I'll make a pull request to PyJulia complementing this as well.

[stevengj]

The old behavior didn't work on Windows.

[benmoran]

@stevengj what part of it didn't work on Windows - PyCall alone or PyJulia? The windows CI tests seem to have passed (for 0.4 at least).

[stevengj]

pyjulia didn't work.

[benmoran]

@stevengj Are there changes that would make you happy to merge this anyway? PyJulia doesn't work for anyone at the moment - it can at least work on Ubuntu and possibly other platforms with this patch.

The AppVeyor Windows tests are passing for 0.4 and failing for Julia 0.3. (Do you still need to support 0.3?)

Let me know if I can help to get this working - it would be great to be able to call Julia from Python again.

[mlubin]

Bump, hoping we can resolve this. pyjulia not working is sad, means I can't share my code with python users.

[stevengj]

This is ugly, but I suppose it is okay for now.

[stevengj]

I feel like the right thing would be for pyjulia to clone PyCall into a separate directory, Pkg.build it in a special way to link Python however it wants, and have a separate precompiled image. That way:

• PyCall from Julia, which continues to be the most common use case, can have efficient precompiled ccalls that hard-code libpython, rather than the per-ccall lookups required here.
• pyjulia modules installed into different versions of Python can each link the correct version of Python, in dependent of what the user configured PyCall to use. (On Windows, this seems to involve linking the DLL rather than trying to use the global symbol table.) Note, for example, that PyCall needs to be configured differently for Python 2 and Python 3. The current setup makes it impossible to use pyjulia from both python 2 and python 3 on the same system without re-building PyCall.
• To look up things in the global symbol table, you should just need to do ccall(:foo, ...) rather than the mess with caching dlsym, no?

[stevengj]

I think I'm going to revert this patch for now, since it is causing problems in #229. Pending a better solution, pyjulia users will have to git checkout v1.3.0.