GPU usage?

[martinjvickers]

Hi,

Firstly I just want to say that this tool is exactly what QGIS needs, it's fantastic. Thank you for all your hard work.

I just have a question about how this runs. It doesn't appear to use the GPU to run, so therefore takes quite a while. I'm just wondering if I'm doing something wrong that is preventing this or it's it's just it's not implemented (or actually a design decision).

Many thanks,

Martin

[przemyslaw-aszkowski]

Hi Martin,

Thanks for your positive feedback!

Actually, the GPU support is built-in in the plugin (tested only for Linux), but it may not work out of the box.
Deepness installs onnxruntime-gpu==1.12.1 and tries to use CUDAExecutionProvider. But to do it, a corresponding version of CUDA tools needs to be installed in the system. See instructions for details. I suspect you would need to install the corresponding version of CUDA toolkit globally, or reinstall a matching version of onnxruntime-gpu.
You can check whether onnxruntime is happy with your CUDA tools with command echo "import onnxruntime as ort; providers = ['CUDAExecutionProvider']; model_file_path = '<YOUR_MODEL_PATH>'; sess = ort.InferenceSession(model_file_path, providers=providers)" | python3. If it prints an error, then the model cannot be run on GPU. A similar code is inside the Deepness plugin.

I hope it helps!
Przemek

[AlainFELER]

Hello, my NVIDIA Quadro K4000 graphic card is CUDA 10.2 able with a Compute Capability 3.0 (768 cuda cores), but according to the Nvidia GPU Utilization graph (run from the Nvidia control panel), that GPU is not used by Deepness "out of the box".
What can I do to put it at work ? Install an older version of onnxruntime ? (at first I thought onnxruntime-gpu==1.12.1 was a parameter in a .ini or a set somewhere but I could not find it anywhere)

[aloboa]

I am also interested in GPU support.