add cl functions for CLArrays

[SimonDanisch]

No description provided.

[SimonDanisch]

needs JuliaGPU/CLArrays.jl#3

[MikeInnes]

This is great to have! However, it's not ideal for Flux to know about all backends like this, so I just added a mechanism that enables the same thing in a generic way. Would CLArrays be able to hook into that, so that we get all this functionality for free?

[SimonDanisch]

I think it's a bit odd to rely on NNlib for that... Why not have this in GPUArrays and have Flux rely on that?
It's quite natural to have Flux rely on GPUArrays!

[MikeInnes]

See JuliaGPU/CUDAnative.jl#121. I can move that function into a separate package if others are on board with the approach.

[SimonDanisch]

Isn't that the perfect use case for moving it into GPUArrays?

[MikeInnes]

Not really. It's completely orthogonal from GPU support (that just happens to be our use case) and CUDAnative isn't going to want to depend on an array abstraction any more than an ML lib.

[SimonDanisch]

Okay didn't realize this isn't just functionality for GPU Arrays anymore...

[MikeInnes]

Also, if you want CLArrays to work with Flux then you'll need to implement the NNlib API for softmax, convolution etc, so it dovetails quite nicely with that.

[SimonDanisch]

Well, as long as I don't have clDNN wrapped, that would live in GPUArrays ;)
And if I do, I'd expect the interface to live in GPUArrays (maybe just reexporting NNlib), and then overload it in CLArrays with clDNN...

[MikeInnes]

Closing in favour of JuliaGPU/GPUArrays.jl#89