New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for other Device Types, OpenCL AMD GPU #621

Open
philtomson opened this Issue Nov 18, 2015 · 34 comments

Comments

Projects
None yet
@philtomson

philtomson commented Nov 18, 2015

It would be nice to eventually have OpenCL support for those of us with GPUs that don't do CUDA.

@jermainewang

This comment has been minimized.

Show comment
Hide comment
@jermainewang

jermainewang Nov 18, 2015

Contributor

Hi,

We are considering this as well! The thing is that for company like AMD, they are actually developing environments that is compatible with CUDA in the future: http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus. So we are still deciding whether we are going to put our limited human resources into supporint OpenCL. If you are interested, you are more than welcome to help us enhance the project in this direction.

Thank you,
Minjie

Contributor

jermainewang commented Nov 18, 2015

Hi,

We are considering this as well! The thing is that for company like AMD, they are actually developing environments that is compatible with CUDA in the future: http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus. So we are still deciding whether we are going to put our limited human resources into supporint OpenCL. If you are interested, you are more than welcome to help us enhance the project in this direction.

Thank you,
Minjie

@gujunli

This comment has been minimized.

Show comment
Hide comment
@gujunli

gujunli Nov 18, 2015

Hi Minjie,

Maybe we can collaborate on extending MXnet with OpenCL support. we have
Opencl caffe open sourced, I guess we can reuse the core kernels?

thanks!
Junli
On Nov 18, 2015 10:53 AM, "Minjie Wang" notifications@github.com wrote:

Hi,

We are considering this as well! The thing is that for company like AMD,
they are actually developing environments that is compatible with CUDA in
the future:
http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus.
So we are still deciding whether we are going to put our limited human
resources into supporint OpenCL. If you are interested, you are more than
welcome to help us enhance the project in this direction.

Thank you,
Minjie


Reply to this email directly or view it on GitHub
#621 (comment).

gujunli commented Nov 18, 2015

Hi Minjie,

Maybe we can collaborate on extending MXnet with OpenCL support. we have
Opencl caffe open sourced, I guess we can reuse the core kernels?

thanks!
Junli
On Nov 18, 2015 10:53 AM, "Minjie Wang" notifications@github.com wrote:

Hi,

We are considering this as well! The thing is that for company like AMD,
they are actually developing environments that is compatible with CUDA in
the future:
http://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus.
So we are still deciding whether we are going to put our limited human
resources into supporint OpenCL. If you are interested, you are more than
welcome to help us enhance the project in this direction.

Thank you,
Minjie


Reply to this email directly or view it on GitHub
#621 (comment).

@jermainewang

This comment has been minimized.

Show comment
Hide comment
@jermainewang

jermainewang Nov 18, 2015

Contributor

Ha, it's great to hear voice from AMD people here :p. I heard that the problem of integrating OpenCL is mainly due to its support for template which is widely used in mshadow. @tqchen may know more details about this.

Minjie

Contributor

jermainewang commented Nov 18, 2015

Ha, it's great to hear voice from AMD people here :p. I heard that the problem of integrating OpenCL is mainly due to its support for template which is widely used in mshadow. @tqchen may know more details about this.

Minjie

@mli

This comment has been minimized.

Show comment
Hide comment
@mli

mli Nov 18, 2015

Member

@gujunli it's very nice to see you here, (we met at icml beijing last year). we are definitely interested on opencl, and hope to support it asap. but the current issue is that we used C++ template while opencl doesn't support it, see dmlc/mshadow#71

Member

mli commented Nov 18, 2015

@gujunli it's very nice to see you here, (we met at icml beijing last year). we are definitely interested on opencl, and hope to support it asap. but the current issue is that we used C++ template while opencl doesn't support it, see dmlc/mshadow#71

@gujunli

This comment has been minimized.

Show comment
Hide comment
@gujunli

gujunli Nov 18, 2015

template is an issue. On AMD devices we have special keyword to support, no
problem. The problem is the same key word does not work on NV GPUs. we are
also figuring out a general solution. I would like to hear your thoughts on
this.@limu @minjie

junli

On Wed, Nov 18, 2015 at 11:43 AM, Mu Li notifications@github.com wrote:

@gujunli https://github.com/gujunli it's very nice to see you here, (we
met at icml beijing last year). we are definitely interested on opencl, and
hope to support it asap. but the current issue is that we used C++ template
while opencl doesn't support it, see dmlc/mshadow#71
dmlc/mshadow#71


Reply to this email directly or view it on GitHub
#621 (comment).


Junli Gu--谷俊丽
Coordinated Science Lab
University of Illinois at Urbana-Champaign


gujunli commented Nov 18, 2015

template is an issue. On AMD devices we have special keyword to support, no
problem. The problem is the same key word does not work on NV GPUs. we are
also figuring out a general solution. I would like to hear your thoughts on
this.@limu @minjie

junli

On Wed, Nov 18, 2015 at 11:43 AM, Mu Li notifications@github.com wrote:

@gujunli https://github.com/gujunli it's very nice to see you here, (we
met at icml beijing last year). we are definitely interested on opencl, and
hope to support it asap. but the current issue is that we used C++ template
while opencl doesn't support it, see dmlc/mshadow#71
dmlc/mshadow#71


Reply to this email directly or view it on GitHub
#621 (comment).


Junli Gu--谷俊丽
Coordinated Science Lab
University of Illinois at Urbana-Champaign


@mli

This comment has been minimized.

Show comment
Hide comment
@mli

mli Nov 18, 2015

Member

nvidia gpu should be fine with cuda. our main motivation to support opencl is for amd gpus and other devices, such as fpga. for example, altera also contacted us to make mxnet run on their devices.

Member

mli commented Nov 18, 2015

nvidia gpu should be fine with cuda. our main motivation to support opencl is for amd gpus and other devices, such as fpga. for example, altera also contacted us to make mxnet run on their devices.

@tqchen

This comment has been minimized.

Show comment
Hide comment
@tqchen

tqchen Nov 18, 2015

Member

This won't pose a problem as long as AMD's version of compiler support somewhat similar thing as nvcc does, i.e. template programming and allow integration of host and device code.

What can be done is to have something like tensor_gpu-inl.amd.cc to specialize for AMD's version of keyword. As long as the extra keyword is minimum and the compiler can be detected by marco, it should be fine.

Member

tqchen commented Nov 18, 2015

This won't pose a problem as long as AMD's version of compiler support somewhat similar thing as nvcc does, i.e. template programming and allow integration of host and device code.

What can be done is to have something like tensor_gpu-inl.amd.cc to specialize for AMD's version of keyword. As long as the extra keyword is minimum and the compiler can be detected by marco, it should be fine.

@tqchen tqchen changed the title from OpenCL support? to Support for other Device Types Nov 18, 2015

@philtomson

This comment has been minimized.

Show comment
Hide comment
@philtomson

philtomson Nov 18, 2015

It would be nice to be able to target FPGAs and OpenCL would allow that to be done much more easily through the Altera tool chain.

Also: I'm not sure I understand the templates issue, isn't there a C API that could be used to get around that?

philtomson commented Nov 18, 2015

It would be nice to be able to target FPGAs and OpenCL would allow that to be done much more easily through the Altera tool chain.

Also: I'm not sure I understand the templates issue, isn't there a C API that could be used to get around that?

@philtomson

This comment has been minimized.

Show comment
Hide comment
@philtomson

philtomson Nov 18, 2015

I'll also add that OpenCL would allow targetting Intel Integrated Graphics which is pretty common on a lot of laptops as well as desktops these days.

philtomson commented Nov 18, 2015

I'll also add that OpenCL would allow targetting Intel Integrated Graphics which is pretty common on a lot of laptops as well as desktops these days.

@vchuravy

This comment has been minimized.

Show comment
Hide comment
@vchuravy

vchuravy Nov 18, 2015

Contributor

@philtomson The problem is more for the kernel code. OpenCL uses C as a language for its kernels and MXNet uses C++ for CUDA and CPU kernels and is able to generate both from the same template, which is nice because you don't need to support 2 or 3 different versions of things.

Contributor

vchuravy commented Nov 18, 2015

@philtomson The problem is more for the kernel code. OpenCL uses C as a language for its kernels and MXNet uses C++ for CUDA and CPU kernels and is able to generate both from the same template, which is nice because you don't need to support 2 or 3 different versions of things.

@tqchen tqchen added the Feature label Nov 21, 2015

@tqchen tqchen changed the title from Support for other Device Types to Support for other Device Types, OpenCL AMD GPU Nov 21, 2015

@ieee8023

This comment has been minimized.

Show comment
Hide comment
@ieee8023

ieee8023 Jan 11, 2016

+1 I want to experiment on my laptop which does not have cuda support!

ieee8023 commented Jan 11, 2016

+1 I want to experiment on my laptop which does not have cuda support!

@tqchen tqchen referenced this issue Mar 30, 2016

Closed

OpenCL support? #1753

@liangfu

This comment has been minimized.

Show comment
Hide comment
@liangfu

liangfu Nov 10, 2016

Contributor

@vchuravy Speak of portability, maybe it's the problem of using template itself in mxnet, because a neural network implement doesn't really need templates for different data types. For a typical neural network implementation, single precision floating point is most commonly used, because double precision is unnecessary and leads to much more computational cost, and half precision computation is not native supported among many devices. Using fixed point data types are completely another case for performance optimization. What people really want is a single efficient, flexible, minimal and yet portable neural network implementation, that can be ported to multiple CPUs, GPUs and FPGAs. The design principle of mxnet meets almost all of these features except the last one.

Contributor

liangfu commented Nov 10, 2016

@vchuravy Speak of portability, maybe it's the problem of using template itself in mxnet, because a neural network implement doesn't really need templates for different data types. For a typical neural network implementation, single precision floating point is most commonly used, because double precision is unnecessary and leads to much more computational cost, and half precision computation is not native supported among many devices. Using fixed point data types are completely another case for performance optimization. What people really want is a single efficient, flexible, minimal and yet portable neural network implementation, that can be ported to multiple CPUs, GPUs and FPGAs. The design principle of mxnet meets almost all of these features except the last one.

@mz24cn

This comment has been minimized.

Show comment
Hide comment
@mz24cn

mz24cn Dec 16, 2016

Is there anyone who tried AMD HIP tools on MXNet?

mz24cn commented Dec 16, 2016

Is there anyone who tried AMD HIP tools on MXNet?

@kernel8liang

This comment has been minimized.

Show comment
Hide comment
@kernel8liang

kernel8liang Jan 9, 2017

Contributor

+1

Contributor

kernel8liang commented Jan 9, 2017

+1

@skirdey

This comment has been minimized.

Show comment
Hide comment
@skirdey

skirdey commented Jan 17, 2017

+1

@cathalgarvey

This comment has been minimized.

Show comment
Hide comment
@cathalgarvey

cathalgarvey Jan 18, 2017

Really want to see this happen someday for a major Python framework besides Tensorflow (and without using a limited, experimental, proprietary compiler framework). Competition!

cathalgarvey commented Jan 18, 2017

Really want to see this happen someday for a major Python framework besides Tensorflow (and without using a limited, experimental, proprietary compiler framework). Competition!

@mz24cn

This comment has been minimized.

Show comment
Hide comment
@mz24cn

mz24cn Jan 19, 2017

https://www.khronos.org/registry/OpenCL/ opencl 2.2 C++ language, including templates support, now is in provisional status. Of course, till now there is no manufacturers releases 2.2 drivers.

mz24cn commented Jan 19, 2017

https://www.khronos.org/registry/OpenCL/ opencl 2.2 C++ language, including templates support, now is in provisional status. Of course, till now there is no manufacturers releases 2.2 drivers.

@delijati

This comment has been minimized.

Show comment
Hide comment
@delijati

delijati Jan 21, 2017

This could help convert the cuda kernels to opencl https://github.com/hughperkins/cuda-on-cl

delijati commented Jan 21, 2017

This could help convert the cuda kernels to opencl https://github.com/hughperkins/cuda-on-cl

@windywinter

This comment has been minimized.

Show comment
Hide comment
@windywinter

windywinter Mar 5, 2017

Contributor

Hi all,

I've been trying to tackle this problem for some time. From my investigation, cocl does not work very well because mshadow is built on Thrust which uses a lot of CUDA host side API that are not supported by cocl. @delijati
Therefore, what we found promising is to use VexCL as the vector expression library (instead of mshadow) for GPU device. Currently I have most arithmetic operators on NDArray working but still need to fill in a hell lot of symbolic operators for the whole framework to work. Proof of concept code is here: https://github.com/windywinter/mxnet

Contributor

windywinter commented Mar 5, 2017

Hi all,

I've been trying to tackle this problem for some time. From my investigation, cocl does not work very well because mshadow is built on Thrust which uses a lot of CUDA host side API that are not supported by cocl. @delijati
Therefore, what we found promising is to use VexCL as the vector expression library (instead of mshadow) for GPU device. Currently I have most arithmetic operators on NDArray working but still need to fill in a hell lot of symbolic operators for the whole framework to work. Proof of concept code is here: https://github.com/windywinter/mxnet

@viper7882

This comment has been minimized.

Show comment
Hide comment
@viper7882

viper7882 May 19, 2017

Hi all,

I'm looking at PyOpenCL and it could be a solution for MXNet. The challenge that I've observed so far is PyOpenCL requires installation of Intel Open CL SDK on user's machine (if they are running Intel Graphics Card).

An example shared by Easy OpenCL with Python is that Gaston Hillar has demonstrated to use only 12 steps to build and deploy a kernel with PyOpenCL. I've tested his codes and it is working for me.

I wonder if MXNet would consider to support PyOpenCL?

viper7882 commented May 19, 2017

Hi all,

I'm looking at PyOpenCL and it could be a solution for MXNet. The challenge that I've observed so far is PyOpenCL requires installation of Intel Open CL SDK on user's machine (if they are running Intel Graphics Card).

An example shared by Easy OpenCL with Python is that Gaston Hillar has demonstrated to use only 12 steps to build and deploy a kernel with PyOpenCL. I've tested his codes and it is working for me.

I wonder if MXNet would consider to support PyOpenCL?

@viper7882

This comment has been minimized.

Show comment
Hide comment
@viper7882

viper7882 Jun 19, 2017

Update: I've tested DeepCL by Hugh Perkins to run using Intel Graphics Card to run Q-Learning and it runs perfectly in Python 2.7: https://github.com/viper7882/DeepCL.

Hugh Perkins has created EasyCL to access OpenCL based GPU @ https://github.com/hughperkins/EasyCL. I'm evaluating if it is possible to merge DeepCL with MXNET. Looks challenging to me to merge the two due to the difference of underlying structure. Any help is appreciated.

viper7882 commented Jun 19, 2017

Update: I've tested DeepCL by Hugh Perkins to run using Intel Graphics Card to run Q-Learning and it runs perfectly in Python 2.7: https://github.com/viper7882/DeepCL.

Hugh Perkins has created EasyCL to access OpenCL based GPU @ https://github.com/hughperkins/EasyCL. I'm evaluating if it is possible to merge DeepCL with MXNET. Looks challenging to me to merge the two due to the difference of underlying structure. Any help is appreciated.

@viper7882

This comment has been minimized.

Show comment
Hide comment
@viper7882

viper7882 Jun 20, 2017

Hi @jermainewang ,

Hugh Perkins has provided NVIDIA® CUDA™ cuDNN API for Coriander, OpenCL 1.2 which ideally should be able to interface with existing Mxnet NVIDIA® CUDA™ cuDNN API.

Could you take a look if it make sense to connect Mxnet with OpenCL through this interface?

viper7882 commented Jun 20, 2017

Hi @jermainewang ,

Hugh Perkins has provided NVIDIA® CUDA™ cuDNN API for Coriander, OpenCL 1.2 which ideally should be able to interface with existing Mxnet NVIDIA® CUDA™ cuDNN API.

Could you take a look if it make sense to connect Mxnet with OpenCL through this interface?

@cathalgarvey

This comment has been minimized.

Show comment
Hide comment
@cathalgarvey

cathalgarvey Jun 20, 2017

cathalgarvey commented Jun 20, 2017

@tqchen

This comment has been minimized.

Show comment
Hide comment
@tqchen

tqchen Aug 17, 2017

Member

Would like to update on this this can now be done via https://github.com/dmlc/tvm

Member

tqchen commented Aug 17, 2017

Would like to update on this this can now be done via https://github.com/dmlc/tvm

@mratsim mratsim referenced this issue Sep 25, 2017

Closed

OpenCL #69

@springishere

This comment has been minimized.

Show comment
Hide comment
@springishere

springishere Oct 5, 2017

@tqchen do you mean that TVM supports opencl? I would like to use mxnet with opencl to use ARM GPU (Mali).

springishere commented Oct 5, 2017

@tqchen do you mean that TVM supports opencl? I would like to use mxnet with opencl to use ARM GPU (Mali).

@tqchen

This comment has been minimized.

Show comment
Hide comment
@tqchen

tqchen Oct 5, 2017

Member

yes, TVM support OpenCL, Metal, CUDA, ARM x86 javascript

Member

tqchen commented Oct 5, 2017

yes, TVM support OpenCL, Metal, CUDA, ARM x86 javascript

@kpot

This comment has been minimized.

Show comment
Hide comment
@kpot

kpot Oct 18, 2017

Hi all,

Guys, can anyone explain why mxnet still doesn't support OpenCL out of the box, even though it is based on nnvm now and through it on tvm, and should be able to perform all necessary computations using OpenCL devices? I checked on nnvm recently and it looks fully up to the task.
But even in the upcoming mxnet 0.12, context mxnet.gpu() still means only "CUDA" and has no associations with tvm.gpu() or tvm.cl(). Why?

Perhaps more than 30% of consumer GPUs around are AMD/Intel-made devices supporting OpenCL >= 1.2. Very often it's a great, inexpensive and less restricted hardware, and making it available for training would greatly benefit ML community.

kpot commented Oct 18, 2017

Hi all,

Guys, can anyone explain why mxnet still doesn't support OpenCL out of the box, even though it is based on nnvm now and through it on tvm, and should be able to perform all necessary computations using OpenCL devices? I checked on nnvm recently and it looks fully up to the task.
But even in the upcoming mxnet 0.12, context mxnet.gpu() still means only "CUDA" and has no associations with tvm.gpu() or tvm.cl(). Why?

Perhaps more than 30% of consumer GPUs around are AMD/Intel-made devices supporting OpenCL >= 1.2. Very often it's a great, inexpensive and less restricted hardware, and making it available for training would greatly benefit ML community.

@welnaseth

This comment has been minimized.

Show comment
Hide comment
@welnaseth

welnaseth Feb 19, 2018

Any updates on this? @kpot makes a good point above, tvm (and nnvm due to being built off of it) supports opencl, so to me it seems like it shouldn't be too hard to implement opencl as an option. It would be nice to have a timeline for when this can be implemented and if not, what things are blocking it?

welnaseth commented Feb 19, 2018

Any updates on this? @kpot makes a good point above, tvm (and nnvm due to being built off of it) supports opencl, so to me it seems like it shouldn't be too hard to implement opencl as an option. It would be nice to have a timeline for when this can be implemented and if not, what things are blocking it?

@conradwt

This comment has been minimized.

Show comment
Hide comment
@conradwt

conradwt Mar 24, 2018

Hi All, are there any updates regarding this topic because I would like to see OpenCL be the default for MXNet as well as other ML libraries and frameworks instead of restricting GPU compute only to Nvidia hardware and CUDA?

conradwt commented Mar 24, 2018

Hi All, are there any updates regarding this topic because I would like to see OpenCL be the default for MXNet as well as other ML libraries and frameworks instead of restricting GPU compute only to Nvidia hardware and CUDA?

@itsergiu

This comment has been minimized.

Show comment
Hide comment
@itsergiu

itsergiu Jun 19, 2018

Do you already provide an installation kit for AMD GPU RX550?
Does it work with Windows 10?
Does it work with Jupyter, Anaconda and Keras on top of Tensorflow?

itsergiu commented Jun 19, 2018

Do you already provide an installation kit for AMD GPU RX550?
Does it work with Windows 10?
Does it work with Jupyter, Anaconda and Keras on top of Tensorflow?

@edmondja

This comment has been minimized.

Show comment
Hide comment
@edmondja

edmondja Jul 3, 2018

+1 waiting for it

edmondja commented Jul 3, 2018

+1 waiting for it

@dmidge8

This comment has been minimized.

Show comment
Hide comment
@dmidge8

dmidge8 Jul 16, 2018

Also hoping to have it!

dmidge8 commented Jul 16, 2018

Also hoping to have it!

@imkow

This comment has been minimized.

Show comment
Hide comment
@imkow

imkow Jul 21, 2018

waiting for this...

imkow commented Jul 21, 2018

waiting for this...

@aenikata

This comment has been minimized.

Show comment
Hide comment
@aenikata

aenikata Sep 17, 2018

At the moment there's cloud providers like gpueater pushing the AMD option, which naturally leads towards Keras+PlaidML not MXNet. My ideal would be to be able to take one of the (almost universally AMD-based) cryptocurrency rigs you can pick up for a reasonable price and see what deep learning you can do with it.

aenikata commented Sep 17, 2018

At the moment there's cloud providers like gpueater pushing the AMD option, which naturally leads towards Keras+PlaidML not MXNet. My ideal would be to be able to take one of the (almost universally AMD-based) cryptocurrency rigs you can pick up for a reasonable price and see what deep learning you can do with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment