Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set_path: include python version in cache dir name #17

Merged
merged 1 commit into from
Aug 2, 2019

Conversation

djsutherland
Copy link
Contributor

If you use pykeops from two different Python versions, it crashes. For example:

$ cat test.py
# from the docs, except use CPU backend
import numpy as np
import pykeops.numpy as pknp

x = np.arange(1, 10).reshape(-1, 3).astype('float32')
y = np.arange(3, 9).reshape(-1, 3).astype('float32')
my_conv = pknp.Genred('SqNorm2(x - y)', ['x = Vi(3)', 'y = Vj(3)'])
res = my_conv(x, y, backend='CPU')
assert res.shape == (2, 1)
print("okay")

$ conda create -y -n tmp36 python=3.6 numpy cmake
[...]
$ conda activate tmp36
$ pip install pykeops
$ python test.py
[...]
okay

$ conda create -y -n tmp37 python=3.7 numpy cmake
[...]
$ conda activate tmp37
$ pip install pykeops
$ python test.py
Traceback (most recent call last):
  File "run_test.py", line 8, in <module>
    my_conv = pknp.Genred('SqNorm2(x - y)', ['x = Vi(3)', 'y = Vj(3)'])
  File "/nfs/nhome/live/dougals/miniconda/envs/tmp37/lib/python3.7/site-packages/pykeops/numpy/generic/generic_red.py", line 114, in __init__
    self.myconv = LoadKEops(self.formula, self.aliases, self.dtype, 'numpy').import_module()
  File "/nfs/nhome/live/dougals/miniconda/envs/tmp37/lib/python3.7/site-packages/pykeops/common/keops_io.py", line 52, in import_module
    return importlib.import_module(self.dll_name)
  File "/nfs/nhome/live/dougals/miniconda/envs/tmp37/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 670, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 583, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1043, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dynamic module does not define module export function (PyInit_libKeOpsnumpy5ac3d464a2)

$ python -c 'import shutil, pykeops; shutil.rmtree(pykeops.bin_folder)'
$ python test.py
[...]
okay

The cache directory contains two .sos in this case, one with a full tag specific to the version (libKeOpsnumpy5ac3d464a2.cpython-37m-x86_64-linux-gnu.so) but also one that isn't (libKeOpsnumpy5ac3d464a2.so). There might be some proper way of allowing them to coexist in the same directory, which would be good e.g. also if the same home directory is used on different platforms. This PR just adds sys.implementation.cache_tag, eg cpython-36, to the default build folder name to work around this (as well as a bit of cleanup in the function overall).

Copy link
Member

@bcharlier bcharlier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, thank again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants