Can't get a consistent path to setup script from installation directory

[LukeLIN-web]

I am trying to install a cpp and python mixed project with editable mode. I am using : pip install --no-index -U -e .
But output is :
/opt/conda/lib/python3.8/distutils/extension.py:131: UserWarning: Unknown Extension options: 'with_cuda'
warnings.warn(msg)
error: ("Can't get a consistent path to setup script from installation directory", '/root/share')

setup.py is https://github.com/quiver-team/torch-quiver/blob/main/setup.py
May you have some ideas?

My env:

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.8.12 (default, Oct 12 2021, 13:49:34)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.15.0-167-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.3.109
CUDA_MODULE_LOADING set to:
GPU models and configuration:
GPU 0: Tesla V100-SXM2-16GB
GPU 1: Tesla V100-SXM2-16GB
GPU 2: Tesla V100-SXM2-16GB

Nvidia driver version: 515.65.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] torch==1.11.0
[pip3] torchelastic==0.2.2
[pip3] torchtext==0.12.0
[pip3] torchvision==0.12.0
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.3.1               ha36c431_9    nvidia
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py38h7f8727e_0
[conda] mkl_fft                   1.3.1            py38hd3c417c_0
[conda] mkl_random                1.2.2            py38h51133e4_0
[conda] numpy                     1.21.2           py38h20f2e39_0
[conda] numpy-base                1.21.2           py38h79a1101_0
[conda] pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchelastic              0.2.2                    pypi_0    pypi
[conda] torchtext                 0.12.0                     py38    pytorch
[conda] torchvision               0.12.0               py38_cu113    pytorch

[abravalheri]

That is a difficult one. It seems that you are using legacy editable installs (that depend on the develop command) and that the error is coming from this function:

setuptools/setuptools/command/develop.py

Line 88 in 08d3297

def _resolve_setup_path(egg_base, install_dir, egg_path):

It might be a wild guess, but have you tried to remove the ./ chars from the beginning of your package dir configuration?

[LukeLIN-web]

I change setup.py to

# package_dir = './srcs/python'
package_dir = 'srcs/python'

Amazing. It really works.

[LukeLIN-web]

That is a difficult one. It seems that you are using legacy editable installs (that depend on the develop command) and that the error is coming from this function:

setuptools/setuptools/command/develop.py

Line 88 in 08d3297

def _resolve_setup_path(egg_base, install_dir, egg_path):

It might be a wild guess, but have you tried to remove the ./ chars from the beginning of your package dir configuration?

sorry, I have a problem, may you know what is the difference between pip install -e and python setup.py develop ?

[abravalheri]

Hi @LukeLIN-web, pip install -e tries to use the interfaces/APIs defined in PEP 660.

This PEP establishes that the build-backend should create a special wheel to be installed. In the latest version of setuptools this is accomplished via the (internal/non-public) editable_wheel command.

python setup.py develop on the other hand is a legacy command that does not follow the PEP 660 standard and uses a custom mechanism to install .egg-link files.

In older versions of setuptools editable_wheel is not available so pip install -e might fallback to python setup.py develop internally. Note that the decision of using PEP 660 or the fallback mechanism is internal to pip's implementation. There might be other circumstances that this also applies and I am not aware.