CUDA error: invalid device function in kaolin.metrics.point.chamfer_distance

[bharat-b7]

from kaolin.rep import TriangleMesh as tm
from kaolin.metrics.point import chamfer_distance

A = torch.rand(300, 3).cuda()
B = torch.rand(200, 3).cuda()
d = chamfer_distance(A, B)

Above code returns (at line 38 kaolin/kaolin/metrics/point.py):

File "/installation/kaolin/kaolin/metrics/point.py", line 38, in forward
    return idx1.long()
RuntimeError: CUDA error: invalid device function

System config:
python == 3.7
pytorch == 1.4.0
cuda == 10.0
g++ == 6.3 (compilation)
gpu == V100

[nicolasugrinovic]

@bharat-b7 this error may indicate that something does not match the compilation with the device you have (this is either your GPU or NVIDIA drive). Most likely that you installed pytorch with cudatoolkit=10.1 and given that your NVIDIA compiler (nvcc) is 10.0 there is a mismatch.

Refer to this facebookresearch/detectron2#62 (comment) for a similar problem.

I solved it by deleting kaolin and pytorch and reinstalling it with conda install pytorch torchvision cudatoolkit=10.0 -c pytorch then installing kaolin again.

[bharat-b7]

Thanks. This was useful.