DNN OPENCL is not working on AMD GPU

[wangqi-cybrook]

I want to use GPU as DNN backend to save CPU power. It works for Intel GPU. But there is problem on AMD GPU. The following are some log.

[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (891) cv::ocl::haveOpenCL Initialize OpenCL runtime...

OpenCV(ocl4dnn): consider to specify kernel configuration cache directory via OPENCV_OCL4DNN_CONFIG_PATH parameter.

[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (433) cv::ocl::OpenCLBinaryCacheConfigurator::OpenCLBinaryCacheConfigurator Successfully initialized OpenCL cache directory: C:\Users\wangq\AppData\Local\Temp\opencv\4.3\opencl_cache\

[ INFO:0] global D:\work\opencv\opencv\modules\core\src\ocl.cpp (457) cv::ocl::OpenCLBinaryCacheConfigurator::prepareCacheDirectoryForContext Preparing OpenCL cache configuration for context: Advanced_Micro_Devices__Inc_--Baffin--2906_10

OpenCL program build log: dnn/dummy

Status -66: CL_INVALID_COMPILER_OPTIONS

-cl-no-subgroup-ifp -D AMD_DEVICE

[alalek]

DNN OPENCL is not working on AMD GPU

This is not true.

CL_INVALID_COMPILER_OPTIONS

It is just a result of a runtime check, so some optimized OpenCL kernels are not used (disabled by this check).

---

There may be lack of optimizations for AMD GPUs in DNN OpenCL implementation.

---

Usage questions should go to Users OpenCV Q/A forum: http://answers.opencv.org

[wangqi-cybrook]

Thanks, alalek.

Maybe what I said is not so accurate. I could get result, but the speed is very slow.
I checked with task manager and saw all calculation is on CPU actually, and it is much slower than not setPreferableTarget(DNN_TARGET_OPENCL).

You said "some optimized OpenCL kernels are not used" and "may be lack of optimizations for AMD GPUs in DNN OpenCL implementation".
That means DNN OpenCL is not usable for AMD GPU? Or we can enable it by setting some flag?

I asked the question at OpenCV Q/A forum: http://answers.opencv.org, but nobody answer it.

[xeeetu]

DNN OPENCL is not working on AMD GPU

This is not true.

CL_INVALID_COMPILER_OPTIONS

It is just a result of a runtime check, so some optimized OpenCL kernels are not used (disabled by this check).

There may be lack of optimizations for AMD GPUs in DNN OpenCL implementation.

Usage questions should go to Users OpenCV Q/A forum: http://answers.opencv.org

Hello alalek, I ask for your advice in what direction to go. Objective is to develop universal application with yolo on windows which can use computing power of AMD/Nvidia/Intel GPU, AMD/Intel CPU (one of the devices will be used). As far as I know, the OpenCV DNN module is leading in CPU computation; for Nvidia video cards we plan to use DNN + Cuda, for Intel GPU: DNN + OpenCL. But testing AMD GPU rx580 with DNN + OpenCL, I ran into the same problem. Does this module not support AMD GPU computing at all? If so, could you please let me know what platform this is possible on and, preferably, as efficiently as possible. Any feedback would be appreciated.

[MeatyAri]

I did this on yolov8 ON WINDOWS and I believe it should work with any other yolo version out there:

1- install the DirectML version of ONNX. It's crucial to choose ONNX DirectML over any other variants or versions. The Python package you need is aptly named "onnxruntime_directml". Feel free to use:

pip install onnxruntime_directml

2- render your YOLO model into the ONNX format.

from ultralytics import YOLO

model = YOLO('yolov8n.pt')
model.export(format='onnx')

3- Add the 'DmlExecutionProvider' string to the providers list: this is lines 133 to 140 in "venv\Lib\site-packages\ultralytics\nn\autobackend.py":

133        elif onnx:  # ONNX Runtime
134        LOGGER.info(f'Loading {w} for ONNX Runtime inference...')
135        # check_requirements(('onnx', 'onnxruntime-gpu' if cuda else 'onnxruntime'))
136        import onnxruntime
137        providers = ['CUDAExecutionProvider', 'CPUExecutionProvider'] if cuda else ['DmlExecutionProvider', 'CPUExecutionProvider']
138        session = onnxruntime.InferenceSession(w, providers=providers)
139        output_names = [x.name for x in session.get_outputs()]
140        metadata = session.get_modelmeta().custom_metadata_map  # metadata

✨ Comment check_requirements line 135 ✨ Add the 'DmlExecutionProvider' string to the providers list

4- enjoy the 100% boot in terms of the model performance