Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation with pip git+ is broken #3752

Open
gdagil opened this issue Feb 28, 2024 · 0 comments
Open

Installation with pip git+ is broken #3752

gdagil opened this issue Feb 28, 2024 · 0 comments

Comments

@gdagil
Copy link

gdagil commented Feb 28, 2024

🐛 Describe the bug

My steps

I have built torch-v2.2.0 on my computer and want to install torchvision and torchaudio

  1. install torchvision - working fine

    CXX=g++-11 CUDA_ROOT=/usr \
        ${CONDA_DIR}/bin/python -m pip install --no-cache --no-cache-dir \
        git+https://github.com/pytorch/vision@v0.17.0
    with logs
    You see the torch with 2.2.0a0+gitunknown version, because i downloaded torch from releases 
     ```
 Collecting git+https://github.com/pytorch/vision@v0.17.0
   Cloning https://github.com/pytorch/vision (to revision v0.17.0) to /tmp/pip-req-build-cfnenazs
   Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision /tmp/pip-req-build-cfnenazs
   Running command git checkout -q b2383d44751bf85e58cfb9223bbf4e5961c09fa1
   Resolved https://github.com/pytorch/vision to commit b2383d44751bf85e58cfb9223bbf4e5961c09fa1
   Installing build dependencies ... done
   Getting requirements to build wheel ... done
   Preparing metadata (pyproject.toml) ... done
 Requirement already satisfied: numpy in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (1.26.0)
 Requirement already satisfied: requests in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (2.31.0)
 Requirement already satisfied: torch in /home/user/conda/lib/python3.11/site-packages (from torchvision==0.17.0+b2383d4) (2.2.0a0+gitunknown)
 Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.17.0+b2383d4)
   Downloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (9.7 kB)
 Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (2.0.4)
 Requirement already satisfied: idna<4,>=2.5 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (3.4)
 Requirement already satisfied: urllib3<3,>=1.21.1 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (1.26.18)
 Requirement already satisfied: certifi>=2017.4.17 in /home/user/conda/lib/python3.11/site-packages (from requests->torchvision==0.17.0+b2383d4) (2024.2.2)
 Requirement already satisfied: filelock in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.13.1)
 Requirement already satisfied: typing-extensions>=4.8.0 in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (4.10.0)
 Requirement already satisfied: sympy in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (1.12)
 Requirement already satisfied: networkx in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.2.1)
 Requirement already satisfied: jinja2 in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (3.1.3)
 Requirement already satisfied: fsspec in /home/user/conda/lib/python3.11/site-packages (from torch->torchvision==0.17.0+b2383d4) (2024.2.0)
 Requirement already satisfied: MarkupSafe>=2.0 in /home/user/conda/lib/python3.11/site-packages (from jinja2->torch->torchvision==0.17.0+b2383d4) (2.1.5)
 Requirement already satisfied: mpmath>=0.19 in /home/user/conda/lib/python3.11/site-packages (from sympy->torch->torchvision==0.17.0+b2383d4) (1.3.0)
 Downloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl (4.5 MB)
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 9.5 MB/s eta 0:00:00
 Building wheels for collected packages: torchvision
   Building wheel for torchvision (pyproject.toml) ... done
   Created wheel for torchvision: filename=torchvision-0.17.0+b2383d4-cp311-cp311-linux_x86_64.whl size=988453 sha256=110b27c25384b026e0d8150bd223528eb70d7f587e5e5bb21544f33b7423e194
   Stored in directory: /tmp/pip-ephem-wheel-cache-halmtbaq/wheels/dd/3b/e3/a0fb109cd18992fe8b57304416c5b8e968d2f1228db0959bfe
 Successfully built torchvision
 Installing collected packages: pillow, torchvision
 Successfully installed pillow-10.2.0 torchvision-0.17.0+b2383d4
 ```

  2. install torchaudio - not working

    LD_LIBRARY_PATH="${CONDA_DIR}/lib:/usr/local/cuda/targets/x86_64-linux/lib${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}" \
CMAKE_FIND_FRAMEWORK=LAST CMAKE_SYSTEM_PROCESSOR=x86_64 \
GLIBCXX_USE_CXX11_ABI=1 \
TORCH_CUDA_ARCH_LIST="7.0;7.2;8.0;8.6;8.7;9.0" \
CMAKE_PREFIX_PATH=${CONDA_DIR:-"$(dirname $(which conda))/../"} \
CMAKE_INCLUDE_PATH="${CONDA_DIR}/include" \
CUDA_USE_STATIC_CUDA_RUNTIME=ON \
USE_SYSTEM_NCCL=ON USE_STATIC_NCCL=OFF NCCL_ROOT="/usr" NCCL_LIB_DIR="/usr/lib/x86_64-linux-gnu" NCCL_INCLUDE_DIR="/usr/include" CUDA_HOME="/usr/local/cuda" \
TORCH_VERSION="2.2.0a0+gitunknown" \
USE_CUSPARSELT=ON CUSPARSELT_INCLUDE_PATH="/usr/include/x86_64-linux-gnu" CUSPARSELT_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu" \
USE_CUDA=ON USE_CUDNN=ON USE_STATIC_CUDNN=OFF CUDNN_ROOT="/usr" \
USE_FFMPEG=ON \
${CONDA_DIR}/bin/python -m pip install --no-cache --no-cache-dir \
        git+https://github.com/pytorch/audio@v2.2.0
    and have error 
     ```
 Collecting git+https://github.com/pytorch/audio@v2.2.0
   Cloning https://github.com/pytorch/audio (to revision v2.2.0) to /tmp/pip-req-build-aa95t9du
   Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio /tmp/pip-req-build-aa95t9du
   Running command git checkout -q 08901ade5d17d3e3cf6fc039cbd601cbd2853686
   Resolved https://github.com/pytorch/audio to commit 08901ade5d17d3e3cf6fc039cbd601cbd2853686
   Running command git submodule update --init --recursive -q
   /bin/sh: /home/user/conda/lib/libtinfo.so.6: no version information available (required by /bin/sh)
   Installing build dependencies ... done
   Getting requirements to build wheel ... error
   error: subprocess-exited-with-error
   
   × Getting requirements to build wheel did not run successfully.
   │ exit code: 1
   ╰─> [20 lines of output]
       Traceback (most recent call last):
         File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
           main()
         File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
           json_out['return_val'] = hook(**hook_input['kwargs'])
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File "/home/user/conda/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
           return hook(config_settings)
                  ^^^^^^^^^^^^^^^^^^^^^
         File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
           return self._get_build_requires(config_settings, requirements=['wheel'])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
           self.run_setup()
         File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
           super().run_setup(setup_script=setup_script)
         File "/tmp/pip-build-env-or3gcfq3/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
           exec(code, locals())
         File "<string>", line 9, in <module>
       ModuleNotFoundError: No module named 'torch'
       [end of output]
   
   note: This error originates from a subprocess, and is likely not a problem with pip.
 error: subprocess-exited-with-error
 
 × Getting requirements to build wheel did not run successfully.
 │ exit code: 1
 ╰─> See above for output.
 
 note: This error originates from a subprocess, and is likely not a problem with pip.

 ```

Question - How solve this problem?

Because as you can see, the imports are in setup.py are almost identical, why can touchvision be installed in the above way, but torchaudio is not

TorchvVision:
https://github.com/pytorch/vision/blob/2c127da8b5e2e8f44b50994c6cb931bcca267cfe/setup.py#L9

TorchAudio:

audio/setup.py

Line 9 in 87aeb55

import torch

Versions

PyTorch version: 2.2.0a0
Is debug build: False
CUDA used to build PyTorch: 12.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-6.6.12-linuxkit-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: 12.2.140
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.6
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Vendor ID: 0x00
Model: 0
Thread(s) per core: 1
Core(s) per socket: 12
Socket(s): 1
Stepping: 0x0
BogoMIPS: 48.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.0
[pip3] torch==2.2.0a0+gitunknown
[pip3] triton==2.1.0
[conda] mkl 2024.0.0 intel_49656 intel
[conda] mkl-dpcpp 2024.0.0 intel_49656 intel
[conda] mkl-include 2024.0.0 intel_49656 intel
[conda] numpy 1.26.0 py311h24aa872_0 anaconda
[conda] numpy-base 1.26.0 py311hbfb1bba_0
[conda] onemkl-sycl-blas 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-datafitting 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-dft 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-lapack 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-rng 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-sparse 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-stats 2024.0.0 intel_49656 intel
[conda] onemkl-sycl-vm 2024.0.0 intel_49656 intel
[conda] torch 2.2.0a0+gitunknown pypi_0 pypi
[conda] triton 2.1.0 pypi_0 pypi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gdagil