Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 #22

saiden89 · 2021-06-23T17:31:39Z

As discussed here, installing torch-spline-conv via the provided wheels fails with OSError: /lib64/libm.so.6: version 'GLIBC_2.27' not found .
We know that:

the issue is unaffected by the cudatoolkit version
the issue is unaffected by python version
the issue triggers only with torch 1.9.0
the issue triggers only on systems with CUDA acceleration.
recompiling solves the issue

As per @rusty1s suggestion, adding a print statement in torch_spline_conv/__init__.py between lines 10 and 11, produces the following output:

>>> import torch_spline_conv
_version
_basis
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".conda/envs/test39/lib/python3.9/site-packages/torch_spline_conv/__init__.py", line 12, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
  File ".conda/envs/test39/lib/python3.9/site-packages/torch/_ops.py", line 104, in load_library
    ctypes.CDLL(path)
  File ".conda/envs/test39/lib/python3.9/ctypes/__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by .conda/envs/test39/lib/python3.9/site-packages/torch_spline_conv/_basis_cuda.so)

Meaning that the culprit is _basis_cuda, as also mentioned in the last line of the error.
Further examination using diff between the recompiled (working) package in the pytorch conda environment and the original one the test39 environment produced the following output:

❯ diff -qr envs/test39/lib/python3.9/site-packages/torch_spline_conv envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_basis_cpu.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_basis_cpu.so differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_basis_cuda.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_basis_cuda.so differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/__pycache__/basis.cpython-39.pyc and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/__pycache__/basis.cpython-39.pyc differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/__pycache__/conv.cpython-39.pyc and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/__pycache__/conv.cpython-39.pyc differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/__pycache__/__init__.cpython-39.pyc and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/__pycache__/__init__.cpython-39.pyc differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/__pycache__/weighting.cpython-39.pyc and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/__pycache__/weighting.cpython-39.pyc differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_version_cpu.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_version_cpu.so differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_version_cuda.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_version_cuda.so differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_weighting_cpu.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_weighting_cpu.so differ
Files envs/test39/lib/python3.9/site-packages/torch_spline_conv/_weighting_cuda.so and envs/pytorch/lib/python3.9/site-packages/torch_spline_conv/_weighting_cuda.so differ

I don't know enough about the subject to make wild speculations, but could it be possible that the provided wheels were compiled in a different environment than the one they're supposed to work in?

The text was updated successfully, but these errors were encountered:

rusty1s · 2021-06-24T11:06:48Z

I'm sadly still not able to reproduce this issue on three different Ubuntu machines. Can you show me the output of running ldd --version?

saiden89 · 2021-06-24T11:09:53Z

❯ ldd --version
ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

rusty1s · 2021-06-24T12:00:36Z

I see. I'm seeing if I can go back to building via ubuntu1604 and let you know.

rusty1s · 2021-06-24T12:39:28Z

Updated the wheels. Please try again :)

saiden89 · 2021-06-24T13:27:22Z

Unfortunately, it doesn't... sorry. I've also created a new conda env from scratch and pip cache purge 😢

rusty1s · 2021-06-24T13:34:16Z

Ok, but the error message should be different, right? Other-wise this might be a cache issue with AWS.

saiden89 · 2021-06-24T13:37:53Z

It looks the same to me

❯ python
Python 3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:32:32) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch_spline_conv
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".conda/envs/spline/lib/python3.9/site-packages/torch_spline_conv/__init__.py", line 11, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
  File ".conda/envs/spline/lib/python3.9/site-packages/torch/_ops.py", line 104, in load_library
    ctypes.CDLL(path)
  File ".conda/envs/spline/lib/python3.9/ctypes/__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by .conda/envs/spline/lib/python3.9/site-packages/torch_spline_conv/_basis_cuda.so)

rusty1s · 2021-06-24T19:29:51Z

Can you try to install via:

pip uninstall torch-spline-conv
pip install https://s3.eu-central-1.amazonaws.com/pytorch-geometric.com/whl/torch-1.9.0%2Bcu102/torch_spline_conv-1.2.1-cp39-cp39-linux_x86_64.whl

saiden89 · 2021-06-24T19:35:18Z

That did it, now it works 😄. What was the issue?

rusty1s · 2021-06-24T19:50:36Z

Should work just fine with -f as well, we just need to wait a few hours till CloudFront does not longer cache the old version.

rusty1s · 2021-06-25T06:24:19Z

I'm closing this for now. Please feel free to re-open if you think this issue is not resolved yet.

hossam-zaki · 2021-11-29T06:16:23Z

Hello! I'm currently getting the same issue. I'm on torch version 1.10.0, pytorch-geometric 2.02, and cuda version 11.3. Here are the commands I ran

pip3 install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

pip3 install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.10.0+cu113.html

Here's the error message

<ipython-input-1-60ea06444bce> in <module>
     13 from tqdm import tqdm
     14 from sklearn.metrics import roc_auc_score, precision_recall_curve, auc
---> 15 from torch_spline_conv import spline_conv

/gpfs/home/hzaki1/celltypefromgrn/env/lib/python3.7/site-packages/torch_spline_conv/__init__.py in <module>
     13     spec = cuda_spec or cpu_spec
     14     if spec is not None:
---> 15         torch.ops.load_library(spec.origin)
     16     else:  # pragma: no cover
     17         raise ImportError(f"Could not find module '{library}_cpu' in "

/gpfs/home/hzaki1/celltypefromgrn/env/lib/python3.7/site-packages/torch/_ops.py in load_library(self, path)
    108             # static (global) initialization code in order to register custom
    109             # operators with the JIT.
--> 110             ctypes.CDLL(path)
    111         self.loaded_libraries.add(path)
    112 

/gpfs/runtime/opt/python/3.7.4/lib/python3.7/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    362 
    363         if handle is None:
--> 364             self._handle = _dlopen(self._name, mode)
    365         else:
    366             self._handle = handle

OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /gpfs/home/hzaki1/celltypefromgrn/env/lib/python3.7/site-packages/torch_spline_conv/_basis_cuda.so)

Then, I try importing, and it doesn't work... Any help would be great

rusty1s · 2021-11-29T09:11:49Z

GLIBC=2.27 is now required as GitHub Actions no longer provide support for ubuntu1604.

wconnell · 2021-12-15T20:19:09Z

FYI you may not require torch_spline_conv; I solved this issue by pip uninstall torch-spline-conv so the import does not search for the package

chrisyeh96 · 2022-02-10T22:44:15Z

I'm also still having the same issue: OSError: /lib64/libm.so.6: version 'GLIBC_2.27' not found. I have installed PyTorch Geometric (pyg) via conda.

Here are the relevant lines from the output of conda list

# Name                    Version         Build                         Channel
cudatoolkit               10.2.89         h8f6ccaa_10                   conda-forge
pyg                       2.0.3           py39_torch_1.10.0_cu102       pyg
python                    3.9.10          h85951f9_2_cpython            conda-forge
pytorch                   1.10.0          py3.9_cuda10.2_cudnn7.6.5_0   pytorch
pytorch-cluster           1.5.9           py39_torch_1.10.0_cu102       pyg
pytorch-mutex             1.0             cuda                          pytorch
pytorch-scatter           2.0.9           py39_torch_1.10.0_cu102       pyg
pytorch-sparse            0.6.12          py39_torch_1.10.0_cu102       pyg
pytorch-spline-conv       1.2.1           py39_torch_1.10.0_cu102       pyg

I tried conda install -c conda-forge glib but that didn't help.

Also, I'm not sure what @rusty1s means by

GLIBC=2.27 is now required as GitHub Actions no longer provide support for ubuntu1604.

Could you please elaborate on what that means for users of PyTorch Geometric?

rusty1s · 2022-02-11T08:23:57Z

We will use GitHub Actions to build our wheels. For linux, this means that we make use of ubuntu1804 which comes with GLIBC_2.27 by default. As you pointed out in pyg-team/pytorch_geometric#3477 (comment), we may can resolve this issue by installing conda and install an earlier version of glib.

hnisonoff · 2022-03-04T00:16:53Z

I am still having this issue.
I tried building with pip install torch-spline-conv but this gave me a different error:

>>> import torch_spline_conv
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/workspace/nisonoff/anaconda3/envs/pytorch_v3/lib/python3.9/site-packages/torch_spline_conv/__init__.py", line 11, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
AttributeError: 'NoneType' object has no attribute 'origin'

Do you have any advice @rusty1s ?

hnisonoff · 2022-03-04T19:08:38Z

Actually I was able to pip install this successfully once I loaded the right cuda environment variables. Thanks!

plutonium-239 · 2022-05-27T18:50:25Z

@hnisonoff can you please describe what cuda environment variables you changed to get it working?

saiden89 mentioned this issue Jun 23, 2021

Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 and CUDA 11.1 pyg-team/pytorch_geometric#2744

Closed

4 tasks

rusty1s closed this as completed Jun 25, 2021

mova mentioned this issue Jul 25, 2022

Cannot install pyg without pytorch_spline_conv in conda #29

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 #22

Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 #22

saiden89 commented Jun 23, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021 •

edited

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

rusty1s commented Jun 25, 2021

hossam-zaki commented Nov 29, 2021 •

edited

rusty1s commented Nov 29, 2021

wconnell commented Dec 15, 2021

chrisyeh96 commented Feb 10, 2022 •

edited

rusty1s commented Feb 11, 2022

hnisonoff commented Mar 4, 2022

hnisonoff commented Mar 4, 2022

plutonium-239 commented May 27, 2022

Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 #22

Cannot import torch-spline-conv when installed from pip wheel in Torch 1.9.0 #22

Comments

saiden89 commented Jun 23, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021 • edited

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

saiden89 commented Jun 24, 2021

rusty1s commented Jun 24, 2021

rusty1s commented Jun 25, 2021

hossam-zaki commented Nov 29, 2021 • edited

rusty1s commented Nov 29, 2021

wconnell commented Dec 15, 2021

chrisyeh96 commented Feb 10, 2022 • edited

rusty1s commented Feb 11, 2022

hnisonoff commented Mar 4, 2022

hnisonoff commented Mar 4, 2022

plutonium-239 commented May 27, 2022

saiden89 commented Jun 24, 2021 •

edited

hossam-zaki commented Nov 29, 2021 •

edited

chrisyeh96 commented Feb 10, 2022 •

edited