Installation problem: File 'submodules/langsplat-rasterization' does not exist.

[Ryan-ZL-Lin]

Hi
First of all, thanks for your amazing job here.
I have couple of question for setting LangSplat up and need your guidance.

1. When running conda env create --file environment.yml, I encountered an error below, is this submodule missing in the repo?

Pip subprocess error:
ERROR: Invalid requirement: 'submodules/langsplat-rasterization' (from line 2 of /mnt/c/Users/lzlal/LangSplat/condaenv.75uxspmj.requirements.txt)
Hint: It looks like a path. File 'submodules/langsplat-rasterization' does not exist.

failed

CondaEnvException: Pip failed

2. I noticed that the segment-anything-langsplat is one of the sub-module listed in environment.yml, so I think it woud be installed once the command conda env create --file environment.yml is completed successfully. However, you also mention segment-anything-langsplat must be installed and that makes me confused, so...do you mean that there is an additional installation process for segment-anything-langsplat or just to download the checkpoints of SAM would work?

3. Once the checkpoint is downloaded, where should I place the folder /ckpts? Is it under LangSplat/submodules/segment-anything-langsplat? /submodules/segment-anything-langsplat/segment_anything ? or LangSplat ?

[haoyuhsu]

My solution is to clone this repo: https://github.com/minghanqin/langsplat-rasterization manually and pip install submodules.

[Ryan-ZL-Lin]

@haoyuhsu
Thanks. I followed your steps and ran conda env create --file environment.yml again, this time, I encountered a different error saying Running setup.py install for diff_gaussian_rasterization did not run successfully. and it came from cuda_rasterizer/backward.h:19:10: fatal error: glm/glm.hpp: No such file or directory. Have you encountered the same issue?

I'm not sure whether it's due to my environment, I'm using Ubuntu 22.04 and here is my CUDA version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

and...here is the full error log for your reference,

(base) ryan_lin@RyanLegionPro7i:/mnt/c/Users/lzlal/LangSplat$ conda env create --file environment.yml
Retrieving notices: ...working... done
Channels:
 - pytorch
 - conda-forge
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: \ By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html

done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/ryan_lin/miniconda3/envs/langsplat/bin/python', '-m', 'pip', 'install', '-U', '-r', '/mnt/c/Users/lzlal/LangSplat/condaenv.d70i0ap0.requirements.txt', '--exists-action=b']
Pip subprocess output:
Processing ./submodules/segment-anything-langsplat
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Processing ./submodules/langsplat-rasterization
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Processing ./submodules/simple-knn
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: segment_anything, diff_gaussian_rasterization, simple_knn
  Building wheel for segment_anything (setup.py): started
  Building wheel for segment_anything (setup.py): finished with status 'done'
  Created wheel for segment_anything: filename=segment_anything-1.0-py3-none-any.whl size=32894 sha256=4e3c0bf42fb7c3232186ea1a2cb55ce49a7d669ed2ddaf1722e44e5fa7325a23
  Stored in directory: /home/ryan_lin/.cache/pip/wheels/e5/b0/d9/61fa40d187afdf3228016dc41c674cec117d99ded3599510b9
  Building wheel for diff_gaussian_rasterization (setup.py): started
  Building wheel for diff_gaussian_rasterization (setup.py): finished with status 'error'
  Running setup.py clean for diff_gaussian_rasterization
  Building wheel for simple_knn (setup.py): started
  Building wheel for simple_knn (setup.py): finished with status 'done'
  Created wheel for simple_knn: filename=simple_knn-0.0.0-cp37-cp37m-linux_x86_64.whl size=1758620 sha256=319c8d66d75c39573a7aeadc576351c935858960dd82988ac04f1117d543d783
  Stored in directory: /home/ryan_lin/.cache/pip/wheels/42/37/ba/a139c13744f017af0d6561d8bfd639dac93f97ceb92c077234
Successfully built segment_anything simple_knn
Failed to build diff_gaussian_rasterization
Installing collected packages: simple_knn, segment_anything, diff_gaussian_rasterization
  Running setup.py install for diff_gaussian_rasterization: started
  Running setup.py install for diff_gaussian_rasterization: finished with status 'error'

Pip subprocess error:
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [24 lines of output]
      running bdist_wheel
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
        warnings.warn(msg.format('we could not find ninja.'))
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-37
      creating build/lib.linux-x86_64-cpython-37/diff_gaussian_rasterization
      copying diff_gaussian_rasterization/__init__.py -> build/lib.linux-x86_64-cpython-37/diff_gaussian_rasterization
      running build_ext
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:813: UserWarning: The detected CUDA version (11.8) has a minor version mismatch with the version that was used to compile PyTorch (11.6). Most likely this shouldn't be a problem.
        warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:820: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
        warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
      building 'diff_gaussian_rasterization._C' extension
      creating build/temp.linux-x86_64-cpython-37
      creating build/temp.linux-x86_64-cpython-37/cuda_rasterizer
      /usr/local/cuda-11.8/bin/nvcc -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/TH -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.8/include -I/home/ryan_lin/miniconda3/envs/langsplat/include/python3.7m -c cuda_rasterizer/backward.cu -o build/temp.linux-x86_64-cpython-37/cuda_rasterizer/backward.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -I/mnt/c/Users/lzlal/LangSplat/submodules/langsplat-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
      In file included from cuda_rasterizer/backward.cu:12:
      cuda_rasterizer/backward.h:19:10: fatal error: glm/glm.hpp: No such file or directory
         19 | #include <glm/glm.hpp>
            |          ^~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/local/cuda-11.8/bin/nvcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for diff_gaussian_rasterization
  error: subprocess-exited-with-error

  × Running setup.py install for diff_gaussian_rasterization did not run successfully.
  │ exit code: 1
  ╰─> [37 lines of output]
      running install
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer or other
              standards-based tools.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        self.initialize_options()
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-37
      creating build/lib.linux-x86_64-cpython-37/diff_gaussian_rasterization
      copying diff_gaussian_rasterization/__init__.py -> build/lib.linux-x86_64-cpython-37/diff_gaussian_rasterization
      running build_ext
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
        warnings.warn(msg.format('we could not find ninja.'))
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:813: UserWarning: The detected CUDA version (11.8) has a minor version mismatch with the version that was used to compile PyTorch (11.6). Most likely this shouldn't be a problem.
        warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
      /home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py:820: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
        warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
      building 'diff_gaussian_rasterization._C' extension
      creating build/temp.linux-x86_64-cpython-37
      creating build/temp.linux-x86_64-cpython-37/cuda_rasterizer
      /usr/local/cuda-11.8/bin/nvcc -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/TH -I/home/ryan_lin/miniconda3/envs/langsplat/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.8/include -I/home/ryan_lin/miniconda3/envs/langsplat/include/python3.7m -c cuda_rasterizer/backward.cu -o build/temp.linux-x86_64-cpython-37/cuda_rasterizer/backward.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -I/mnt/c/Users/lzlal/LangSplat/submodules/langsplat-rasterization/third_party/glm/ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
      In file included from cuda_rasterizer/backward.cu:12:
      cuda_rasterizer/backward.h:19:10: fatal error: glm/glm.hpp: No such file or directory
         19 | #include <glm/glm.hpp>
            |          ^~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/local/cuda-11.8/bin/nvcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> diff_gaussian_rasterization

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

failed

CondaEnvException: Pip failed

[haoyuhsu]

@Ryan-ZL-Lin

I think instead of install from environment.yml, you could try install each module separately. (ex: pip install submodules/langsplat-rasterization/).

For glm error, I think you could install it in two ways:
(1) sudo apt-get install libglm-dev
(2) conda install conda-forge::glm

Both ways work for me. Hope this would help!

[Ryan-ZL-Lin]

@haoyuhsu
Thanks a lot, it works.

[KohsukeIde]

@Ryan-ZL-Lin @haoyuhsu
i've been trying to setup environment for langsplat and i've been stuck for a while now. so i wanted to make sure the step i took

1. build conda env based on the original environment.yml
2. clone https://github.com/minghanqin/langsplat-rasterization into submodule
3. run pip install for submodules/langsplat-rasterization/ ,submodules/segment-anything-langsplat, submodules/simple-knn

and it should work?

once i try to mimic the process above, it gives me the following error.

(langsplat) kohsuke@kohsuke-desktop:~/dev/aist/3DGS/LangSplat$ pip install submodules/segment-anything-langsplat
Processing ./submodules/segment-anything-langsplat
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/kohsuke/dev/aist/3DGS/LangSplat/submodules/segment-anything-langsplat/setup.py", line 16, in <module>
          "dev": ["flake8", "isort", "black", "mypy"],
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/_distutils/core.py", line 172, in setup
          ok = dist.parse_command_line()
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/_distutils/dist.py", line 475, in parse_command_line
          args = self._parse_command_opts(parser, args)
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/dist.py", line 869, in _parse_command_opts
          nargs = _Distribution._parse_command_opts(self, parser, args)
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/_distutils/dist.py", line 534, in _parse_command_opts
          cmd_class = self.get_command_class(command)
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/dist.py", line 714, in get_command_class
          self.cmdclass[command] = cmdclass = ep.load()
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/_vendor/importlib_metadata/__init__.py", line 208, in load
          module = import_module(match.group('module'))
        File "/home/kohsuke/anaconda3/envs/langsplat/lib/python3.7/importlib/__init__.py", line 127, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
        File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
        File "<frozen importlib._bootstrap>", line 983, in _find_and_load
        File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 728, in exec_module
        File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/command/egg_info.py", line 23, in <module>
          from setuptools.command.sdist import sdist
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/command/sdist.py", line 8, in <module>
          from .build import _ORIGINAL_SUBCOMMANDS
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/setuptools/command/build.py", line 1, in <module>
          from typing import Dict, List, Protocol
      ImportError: cannot import name 'Protocol' from 'typing' (/home/kohsuke/anaconda3/envs/langsplat/lib/python3.7/typing.py)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(langsplat) kohsuke@kohsuke-desktop:~/dev/aist/3DGS/LangSplat$ pip install submodules/langsplat-rasterization
Processing ./submodules/langsplat-rasterization
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [10 lines of output]
      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/kohsuke/dev/aist/3DGS/LangSplat/submodules/langsplat-rasterization/setup.py", line 13, in <module>
          from torch.utils.cpp_extension import CUDAExtension, BuildExtension
        File "/home/kohsuke/anaconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
          from pkg_resources import packaging  # type: ignore[attr-defined]
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/pkg_resources/__init__.py", line 23, in <module>
          raise RuntimeError("Python 3.8 or later is required")
      RuntimeError: Python 3.8 or later is required
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(langsplat) kohsuke@kohsuke-desktop:~/dev/aist/3DGS/LangSplat$ pip install submodules/simple-knn
Processing ./submodules/simple-knn
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [10 lines of output]
      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/kohsuke/dev/aist/3DGS/LangSplat/submodules/simple-knn/setup.py", line 13, in <module>
          from torch.utils.cpp_extension import CUDAExtension, BuildExtension
        File "/home/kohsuke/anaconda3/envs/langsplat/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
          from pkg_resources import packaging  # type: ignore[attr-defined]
        File "/home/kohsuke/.local/lib/python3.7/site-packages/setuptools-69.2.0-py3.7.egg/pkg_resources/__init__.py", line 23, in <module>
          raise RuntimeError("Python 3.8 or later is required")
      RuntimeError: Python 3.8 or later is required
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

seems like a problem with python version...
I'd like to know correct steps that needs to be taken in order to setup the environment.

[haoyuhsu]

@KohsukeIde

I think you may check if glm and CUDA are installed correctly.

[KohsukeIde]

yea. I think it was the problem with CUDA.
I was able to solve it based on graphdeco-inria/gaussian-splatting#724 and #13

Also used the following environment.yml to set up the environment
for anyone interested in setting up environment for cudatoolkit =11.7

name: langsplat
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - open-clip-torch
  - cudatoolkit=11.7
  - plyfile=0.8.1
  - python=3.8
  - pip=22.3.1  
  # - pytorch=2.0.0
  # - torchaudio=2.0.1
  # - torchvision=0.15.1
  - tqdm
  - opencv
  - tensorboard
  - jaxtyping
  - matplotlib
  - typing
  - pathlib
  - mediapy
  # - pip:
  #   - submodules/segment-anything-langsplat
  #   - submodules/langsplat-rasterization
  #   - submodules/simple-knn