Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError: Unknown compute capability. Ensure PyTorch with CUDA support is installed. #1177

Closed
grshennan opened this issue Dec 27, 2022 · 12 comments
Labels

Comments

@grshennan
Copy link

grshennan commented Dec 27, 2022

Installed nerfstudio with CUDA 11.3 as per setup page. Everything works fine. Followed the tutorial all the way to ns-train nerfacto --data data/nerfstudio/poster where I get this error:

$ ns-train nerfacto --data data/nerfstudio/poster
Traceback (most recent call last):
  File "C:\Users\glens\miniconda3\envs\ns\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\glens\miniconda3\envs\ns\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\glens\miniconda3\envs\ns\Scripts\ns-train.exe\__main__.py", line 4, in <module>
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\scripts\train.py", line 50, in <module>
    from nerfstudio.configs.method_configs import AnnotatedBaseConfigUnion
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\nerfstudio\configs\method_configs.py", line 46, in <module>
    from nerfstudio.field_components.temporal_distortions import TemporalDistortionKind
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\nerfstudio\field_components\__init__.py", line 17, in <module>
    from .encodings import Encoding, ScalingAndOffset
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\nerfstudio\field_components\encodings.py", line 34, in <module>
    import tinycudann as tcnn
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\tinycudann\__init__.py", line 9, in <module>
    from tinycudann.modules import free_temporary_memory, NetworkWithInputEncoding, Network, Encoding
  File "C:\Users\glens\miniconda3\envs\ns\lib\site-packages\tinycudann\modules.py", line 16, in <module>
    raise EnvironmentError("Unknown compute capability. Ensure PyTorch with CUDA support is installed.")
OSError: Unknown compute capability. Ensure PyTorch with CUDA support is installed.

Removed CUDA 11.3, installed 11.7 and followed the install instructions on the Pytorch page, i.e. conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia. Everything works as per the pip install instructions. tinycudann even seemed to build a lot faster. Ended up with exactly the same error: OSError: Unknown compute capability. Ensure PyTorch with CUDA support is installed.

CUDA variables are set as (not sure if this is relevant):
image

I understand devs are busy and I don't want to bother anyone but I'm trying really hard to get this working and just keep getting pipped at the post. Let me know if there's a better forum.

@tancik tancik added the windows label Dec 27, 2022
@brawlawnzo
Copy link

I ran into the same exact error today with conda installation of PyTorch with CUDA 11.7 (11.3 worked fine for me though). Seems like a problem with tinycuda rather than nerfstudio.

@THE-COB
Copy link
Contributor

THE-COB commented Jan 1, 2023

I'm also using CUDA 11.3, did the tinycuda installation go entirely smoothly? I had a bunch of issues with it before I got it to work.

@grshennan
Copy link
Author

@THE-COB I have both had tinycudann installation go smoothly and not work at all. I think @brawlawnzo is right, it's an issue with tinycuda.

I just tried to install as per nerfstudio installation instructions on my work computer. Visual Studio 2019, CUDA 11.3, Python 3.8. Completely clean install. Got this error install tinycudann:

 pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Collecting git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
  Cloning https://github.com/NVlabs/tiny-cuda-nn/ to c:\users\glens\appdata\local\temp\pip-req-build-s1rw0r85
  Running command git clone --filter=blob:none --quiet https://github.com/NVlabs/tiny-cuda-nn/ 'C:\Users\glens\AppData\Local\Temp\pip-req-build-s1rw0r85'
  Resolved https://github.com/NVlabs/tiny-cuda-nn/ to commit 14053e9a87ebf449d32bda335c0363dd4f5667a4
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tinycudann
  Building wheel for tinycudann (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [1327 lines of output]
      Building PyTorch extension for tiny-cuda-nn version 1.7
      Obtained compute capability 86 from PyTorch
      running bdist_wheel
      C:\Users\glens\miniconda3\envs\ns\lib\site-packages\torch\utils\cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
        warnings.warn(msg.format('we could not find ninja.'))
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-38
      creating build\lib.win-amd64-cpython-38\tinycudann
      copying tinycudann\modules.py -> build\lib.win-amd64-cpython-38\tinycudann
      copying tinycudann\__init__.py -> build\lib.win-amd64-cpython-38\tinycudann
      running egg_info
      creating tinycudann.egg-info
      writing tinycudann.egg-info\PKG-INFO
      writing dependency_links to tinycudann.egg-info\dependency_links.txt
      writing top-level names to tinycudann.egg-info\top_level.txt
      writing manifest file 'tinycudann.egg-info\SOURCES.txt'
      reading manifest file 'tinycudann.egg-info\SOURCES.txt'
      writing manifest file 'tinycudann.egg-info\SOURCES.txt'
      copying tinycudann\bindings.cpp -> build\lib.win-amd64-cpython-38\tinycudann
      running build_ext
      building 'tinycudann_bindings_86._C' extension
      creating build\dependencies
      creating build\dependencies\fmt
      creating build\dependencies\fmt\src
      creating build\src
      creating build\temp.win-amd64-cpython-38
      creating build\temp.win-amd64-cpython-38\Release
      creating build\temp.win-amd64-cpython-38\Release\tinycudann
      "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\gle...

followed by thousands of lines of errors, and then:

      Error limit reached.
      100 errors detected in the compilation of "../../src/common.cu".
      Compilation terminated.
      common.cu
      error: command 'C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA\\v11.3\\bin\\nvcc.exe' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> tinycudann

@THE-COB
Copy link
Contributor

THE-COB commented Jan 2, 2023

@grshennan
I think I might have run into this as well!
It seems to resemble the issue in here: NVlabs/tiny-cuda-nn#169

One of the things in that thread that I think got me passed that (and then to the next error lol) was conda installing ninja.
If that doesn't work maybe try reinstalling visual studio (2019 should be alright, but I'm currently using 2017)

@grshennan
Copy link
Author

grshennan commented Jan 3, 2023

@THE-COB I have VS 2019 installed but same idea, yes. I think it's the same as this issue: NVlabs/tiny-cuda-nn#147

Tried installing ninja and got a different error. Now it's

running build_ext
error: [WinError 2] The system cannot find the file specified

as per the tinycudann issue.

Edit: I tried conda install cython as that package wasn't installed and seems to have something to do with build_ext but it didn't work.

@THE-COB
Copy link
Contributor

THE-COB commented Jan 4, 2023

What is your current error? @grshennan

@grshennan
Copy link
Author

$ pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Collecting git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
  Cloning https://github.com/NVlabs/tiny-cuda-nn/ to c:\users\glens\appdata\local\temp\pip-req-build-6w2nnebl
  Running command git clone --filter=blob:none --quiet https://github.com/NVlabs/tiny-cuda-nn/ 'C:\Users\glens\AppData\Local\Temp\pip-req-build-6w2nnebl'
  Resolved https://github.com/NVlabs/tiny-cuda-nn/ to commit 14053e9a87ebf449d32bda335c0363dd4f5667a4
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tinycudann
  Building wheel for tinycudann (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      Building PyTorch extension for tiny-cuda-nn version 1.7
      Obtained compute capability 86 from PyTorch
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-38
      creating build\lib.win-amd64-cpython-38\tinycudann
      copying tinycudann\modules.py -> build\lib.win-amd64-cpython-38\tinycudann
      copying tinycudann\__init__.py -> build\lib.win-amd64-cpython-38\tinycudann
      running egg_info
      creating tinycudann.egg-info
      writing tinycudann.egg-info\PKG-INFO
      writing dependency_links to tinycudann.egg-info\dependency_links.txt
      writing top-level names to tinycudann.egg-info\top_level.txt
      writing manifest file 'tinycudann.egg-info\SOURCES.txt'
      reading manifest file 'tinycudann.egg-info\SOURCES.txt'
      writing manifest file 'tinycudann.egg-info\SOURCES.txt'
      copying tinycudann\bindings.cpp -> build\lib.win-amd64-cpython-38\tinycudann
      running build_ext
      error: [WinError 2] The system cannot find the file specified
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tinycudann
  Running setup.py clean for tinycudann
Failed to build tinycudann
Installing collected packages: tinycudann
  Running setup.py install for tinycudann ... error
  error: subprocess-exited-with-error

  × Running setup.py install for tinycudann did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      Building PyTorch extension for tiny-cuda-nn version 1.7
      Obtained compute capability 86 from PyTorch
      running install
      C:\Users\glens\miniconda3\envs\ns\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      running egg_info
      writing tinycudann.egg-info\PKG-INFO
      writing dependency_links to tinycudann.egg-info\dependency_links.txt
      writing top-level names to tinycudann.egg-info\top_level.txt
      reading manifest file 'tinycudann.egg-info\SOURCES.txt'
      writing manifest file 'tinycudann.egg-info\SOURCES.txt'
      running build_ext
      error: [WinError 2] The system cannot find the file specified
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> tinycudann

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

@THE-COB
Copy link
Contributor

THE-COB commented Jan 4, 2023

@grshennan that is tough, I think I might have run into something similar but I don't remember. If you want (this may or may not be worth it) try building tinycudann in the way suggested in this comment?
NVlabs/tiny-cuda-nn#169 (comment)
Sorry I wasn't able to give you concrete help

@grshennan
Copy link
Author

grshennan commented Jan 4, 2023

Oh wow, it works. Thanks, @THE-COB.

I changed CUDA to 11.7 and Visual Studio to 2022. So still using pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html. Then I followed the comment and compiled tiny-cuda-nn with

git clone --recursive https://github.com/nvlabs/tiny-cuda-nn
cmake . -B build
cmake --build build --config RelWithDebInfo -j

(cmake commands run in VS developer command prompt)

Then just pip install nerfstudio and we're off. I was getting really frustrated trying to upload photos to Colab and having the runtime crash so this is perfect. Thanks again.

@dooonnis
Copy link

dooonnis commented Feb 5, 2023

I'm fairly new to this topic.

Would you explain, where you have placed the tiny-cuda-nn files and in which directory you have started the cmake process?
Is that necessary for everything to work?

I am receiving this error:

Traceback (most recent call last):
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "F:\Programme\miniconda3\envs\nerfstudio\Scripts\ns-train.exe\__main__.py", line 7, in <module>
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\scripts\train.py", line 247, in entrypoint
    main(
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\scripts\train.py", line 233, in main
    launch(
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\scripts\train.py", line 172, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\scripts\train.py", line 86, in train_loop
    trainer.setup()
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\engine\trainer.py", line 145, in setup
    self.pipeline = self.config.pipeline.setup(
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\configs\base_config.py", line 57, in setup
    return self._target(self, **kwargs)
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\pipelines\base_pipeline.py", line 229, in __init__
    self._model = config.model.setup(
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\configs\base_config.py", line 57, in setup
    return self._target(self, **kwargs)
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\models\base_model.py", line 82, in __init__
    self.populate_modules()  # populate the modules
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\models\nerfacto.py", line 135, in populate_modules
    self.field = TCNNNerfactoField(
  File "F:\Programme\miniconda3\envs\nerfstudio\lib\site-packages\nerfstudio\fields\nerfacto_field.py", line 134, in __init__
    self.direction_encoding = tcnn.Encoding(
NameError: name 'tcnn' is not defined

@tancik
Copy link
Contributor

tancik commented Feb 5, 2023

It sounds like tiny-cuda-nn wasn't installed correctly. I would try running pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch again. Also try opening up a python prompt, python, then running import tinycudann to see what error you get.

@dooonnis
Copy link

dooonnis commented Feb 5, 2023

It sounds like tiny-cuda-nn wasn't installed correctly. I would try running pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch again. Also try opening up a python prompt, python, then running import tinycudann to see what error you get.

Thank you very much! The reinstallation actually worked, although the pip installation method of Tiny CUDA Neural Networks has never worked before for me.

To document what I have done:

  1. Installed CUDA 11.7 and VS 2019 and deinstalled all other versions

  2. Created a conda envoriment
    conda create --name nerfstudio -y python=3.8
    conda activate nerfstudio
    python -m pip install --upgrade pip

  3. Installed pytorch
    pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html

  4. Git cloned tiny-cuda-nn via the repo into miniconda3\envs\nerfstudio\Lib\site-packages
    git clone --recursive https://github.com/nvlabs/tiny-cuda-nn
    cd tiny-cuda-nn
    cmake . -B build
    cmake --build build --config RelWithDebInfo -j
    (cmake commands run in VS developer command prompt)

  5. Installed nerfstudio
    pip install nerfstudio

  6. Reinstalled tiny-cuda-nn via pip
    pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

I am not sure, why the installation via pip worked without errors during the last step, but nerfstudio is now fully functional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants