Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using pyenv breaks PythonCall #318

Closed
doorisajar opened this issue Jun 1, 2023 · 4 comments
Closed

Using pyenv breaks PythonCall #318

doorisajar opened this issue Jun 1, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@doorisajar
Copy link

Affects: PythonCall

Describe the bug
After setting up a local Python version using pyenv, creating a virtual environment, and setting up PythonCall to use that venv instead of Conda, PythonCall can't be loaded:

julia> using PythonCall
┌ Warning: Python library "/Users/rdoake/.pyenv/versions/3.9.13/lib/python3.9/config-3.9-darwin/libpython3.9.a" could not be opened.
└ @ PythonCall.C ~/.julia/packages/PythonCall/1f5yE/src/cpython/context.jl:118
┌ Warning: Python library "/Users/rdoake/.pyenv/versions/3.9.13/lib/libpython3.9.a" could not be opened.
└ @ PythonCall.C ~/.julia/packages/PythonCall/1f5yE/src/cpython/context.jl:118
ERROR: InitError: ArgumentError: NULL library handle

The files referenced in the error message do exist.

Steps to reproduce

  • Create a directory & shell into it
  • pyenv local x.x # I used 3.9 to disambiguate from my default system Python, which is 3.10
  • create a venv at .venv using the tool of your choice (I tested with Poetry and venv)
  • launch Julia
  • ] activate .
  • > ENV["JULIA_CONDAPKG_BACKEND"]="Null" # don't use Conda
  • > ENV["JULIA_PYTHONCALL_EXE"]=".venv/bin/python3" # point PythonCall to the venv python
  • ] add PythonCall
  • > using PythonCall

My system

  • MacOS 13.4
  • Julia 1.9.0
  • PythonCall 0.9.13

Additional context
If I instead use poetry to specify a Python version for the virtual environment, and then do the above steps without using pyenv, then PythonCall works as expected.

@doorisajar doorisajar added the bug Something isn't working label Jun 1, 2023
@cjdoris
Copy link
Collaborator

cjdoris commented Jun 2, 2023

How did you install the version of Python you are trying to use? From the logs you've sent, it's a statically linked build of Python which is unfortunately not supported by PythonCall (because it needs to dynamically link to libpython). Often this is the case for Python bundled in system package managers.

@doorisajar
Copy link
Author

I manage installed Python versions using pyenv.

@cjdoris
Copy link
Collaborator

cjdoris commented Jun 2, 2023

Ah, I didn't realise pyenv builds Python for you. Looks like you can get it to build the shared library here: https://github.com/pyenv/pyenv/blob/master/plugins/python-build/README.md#building-with---enable-shared

@doorisajar
Copy link
Author

Just realized I forgot to circle back to this -- that workaround does enable PythonCall.jl to work with a pyenv-installed Python version. I can keep doing that for the time being. Thanks!

@cjdoris cjdoris closed this as completed Jul 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants