Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUDA kernel errors #55

Open
jong3 opened this issue Aug 4, 2021 · 2 comments
Open

CUDA kernel errors #55

jong3 opened this issue Aug 4, 2021 · 2 comments

Comments

@jong3
Copy link

jong3 commented Aug 4, 2021

When I try to run 'run_pyrosetta_ver.sh input.fa .', the error will be reported in 'network.stderr' file. Why is this?
Thanks.

Using backend: pytorch
Traceback (most recent call last):
  File "/home/ganjh/RoseTTAFold/network/predict_pyRosetta.py", line 199, in <module>
    pred = Predictor(model_dir=args.model_dir, use_cpu=args.use_cpu)
  File "/home/ganjh/RoseTTAFold/network/predict_pyRosetta.py", line 67, in __init__
    self.model = RoseTTAFoldModule(**MODEL_PARAM).to(self.device)
  File "/home/ganjh/.conda/envs/RoseTTAFold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 852, in to
    return self._apply(convert)
  File "/home/ganjh/.conda/envs/RoseTTAFold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 530, in _apply
    module._apply(fn)
  File "/home/ganjh/.conda/envs/RoseTTAFold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 530, in _apply
    module._apply(fn)
  File "/home/ganjh/.conda/envs/RoseTTAFold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 552, in _apply
    param_applied = fn(param)
  File "/home/ganjh/.conda/envs/RoseTTAFold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 850, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: the provided PTX was compiled with an unsupported toolchain.
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
@jong3
Copy link
Author

jong3 commented Aug 4, 2021

It seems that CUDA version and Driver version are incompatible .

CUDA and Driver's information are as follow:
NVIDIA-SMI 450.142.00 Driver Version: 450.142.00 CUDA Version: 11.0

@jong3
Copy link
Author

jong3 commented Aug 5, 2021

I have fixed it by updating my driver version.
Because Torch generally does not use the CUDA version you installed, it uses its own.
By command "print(torch.version.cuda) " in python, I know that Torch's CUDA version is 11.1, which is higher than CUDA version I have installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant