OpenCL support

[pietern]

Master issue to track OpenCL support.

@danzimm -- if you end up issuing some PRs, please mention this issue. Thanks 💯 👍

[Yangqing]

cc @bwasti

[VincentSC]

Would the goal be to support AMD, or to support all GPUs (including Intel and ARM), or even also FPGAs and DSPs?

[psyhtest]

@pietern Would you like any help and contributions?

We have many years of experience with optimising applications for server, mobile and embedded OpenCL accelerators (especially for the market-dominant ARM Mali and Qualcomm Adreno GPUs), as well as tuning closed- and open-source compute libraries for Caffe1 and other DNN frameworks (e.g. see this public Jupyter Notebook).

Most importantly, we have unique expertise on how to achieve OpenCL performance portability (no mean feat!) across diverse operating environments (Android, Linux, Windows), device architectures (CPUs, GPUs, DSPs, custom accelerators), data inputs (sizes, shapes, patterns), etc.

[viper7882]

Hi @Yangqing ,

Hugh Perkins has created Coriander that could run NVIDIA® CUDA™ code on OpenCL 1.2 devices. You might want to take a look if that suits your need. Kindly attribute his name and his contribution in case if you plan to use his work.

[psyhtest]

@Yangqing @bwasti @pietern Guys, are you open to contributions, or should the OpenCL community be content with contributing to Caffe1?

[haolongzhangm]

@psyhtest any idea for caffe1 opencl on MALI OR Adreno GPU, we run caffe1 with OPENCL on MALI or Adreno GPU with android ENV , we find opencl cost so much time to run finish kernel

run fcn net, ARM*8 CPU only take 5S, but use MALI T8 GPU with opencl will take about 25S to onetime iter

[quartzsaber]

Is it planned to support OpenCL 1.1? Or just only 2.0 and above?

I heard 2.0 adds many features that CUDA had but OpenCL hadn't.

[psyhtest]

@haolongzhangm Apologies, I've only just read your message here.

Which OpenCL math library do you use with Caffe? ViennaCL and clBLAS are not optimised for Mobile. CLBlast can be tuned with very good results.

Also, are you using FCN-16 by any chance? I found this network to be a real killer for mobile GPUs, taking seconds for a single convolution layer even with adequately optimised code.

[psyhtest]

@Yangqing Any plans to support OpenCL in Caffe2?

[orionr]

There is work being done by ROCm (https://rocm.github.io/index.html) on Caffe2 at https://github.com/ROCmSoftwarePlatform/caffe2 for OpenCL. Feel free to take a look at that as well.

[naibaf7]

@orionr ROCm is not OpenCL. This will not work on any other devices than AMDGPU-PRO. It's based on HIP, an AMD drop-in replacement for CUDA.

[nuka137]

Is this support still contribution welcomed?
I'm interested in OpenCL support on caffe2.

[llhe]

For deep learning inference on mobile devices with GPU/OpenCL support, you can checkout MACE, which supports Adreno, Mali and PowerVR GPUs. Here are some benchmark results.