copied the old rnn.jl->rnncompat.jl for Flux compatibility #738
Conversation
|
Rebased on top of the |
This is up to Flux people, Knet does not use the interface in rnncompat.jl. I'd rather move Flux specific code out of CUDA/lib/cudnn into their respective packages. |
Codecov Report
@@ Coverage Diff @@
## compat #738 +/- ##
==========================================
- Coverage 79.10% 77.94% -1.16%
==========================================
Files 123 124 +1
Lines 6705 6807 +102
==========================================
+ Hits 5304 5306 +2
- Misses 1401 1501 +100
Continue to review full report at Codecov.
|
|
yes |
|
IIUC the |
No problem @CarloLucibello. There are a few compat files in CUDA/lib/cudnn that will probably be deprecated in the next major CUDA.jl release - if you want to keep using them in NNlib/Flux please copy them there:
|
FWIW, the next release is bound to be a breaking one, CUDA.jl 3.0. So now is the time maybe to remove those? The alternative is to release another breaking release some time after that, but that may make people unhappy too. |
|
it makes sense to remove the NNlib dependence already in 3.0. We'll create a NNlibCUDA.jl package as a subpackage of NNlib. |
|
We have a bunch of benefits from testing with the cuda system and checking with cudnn has been helpful as well. |
|
Is it not possible to do a similar test set up with a NNlibCUDA.jl? We've been talking a bit on Zulip about this with @jpsamaroo. Seems like separating out backend code and glue code into contained packages might be nice when we add support for AMD GPUs. Would also be nicer for dependency management and bug fix releases. |
|
@denizyuret I run Flux's test on the compat branch here and got this error R = RNN: Error During Test at /home/carlo/.julia/packages/Flux/goUGu/test/cuda/curnn.jl:4
Got exception outside of a @test
UndefVarError: DropoutDesc not defined
Stacktrace:
[1] CUDA.CUDNN.RNNDesc{Float32}(::CUDA.CUDNN.cudnnRNNMode_t, ::Int64, ::Int64; layers::Int64) at /home/carlo/.julia/packages/CUDA/omtO5/lib/cudnn/rnncompat.jl:46
[2] RNNDesc at /home/carlo/.julia/packages/CUDA/omtO5/lib/cudnn/rnncompat.jl:43 [inlined]
[3] RNNDesc at /home/carlo/.julia/packages/Flux/goUGu/src/cuda/curnn.jl:13 [inlined]
[4] desc(::Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}) at /home/carlo/.julia/packages/Flux/goUGu/src/cuda/curnn.jl:20
[5] adjoint at /home/carlo/.julia/packages/Flux/goUGu/src/cuda/curnn.jl:70 [inlined]
[6] _pullback at /home/carlo/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:57 [inlined]
[7] adjoint at /home/carlo/.julia/packages/Zygote/KpME9/src/lib/lib.jl:188 [inlined]
[8] adjoint(::Zygote.Context, ::typeof(Core._apply_iterate), ::typeof(iterate), ::Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}, ::Tuple{CuArray{Float32,1}}, ::Tuple{CuArray{Float32,1}}) at ./none:0
[9] _pullback at /home/carlo/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:57 [inlined]
[10] Recur at /home/carlo/.julia/packages/Flux/goUGu/src/layers/recurrent.jl:36 [inlined]
[11] _pullback(::Zygote.Context, ::Flux.Recur{Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}}, ::CuArray{Float32,1}) at /home/carlo/.julia/packages/Zygote/KpME9/src/compiler/interface2.jl:0
[12] #166 at /home/carlo/.julia/packages/Flux/goUGu/test/cuda/curnn.jl:7 [inlined]
[13] _pullback(::Zygote.Context, ::var"#166#168"{CuArray{Float32,1}}, ::Flux.Recur{Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}}) at /home/carlo/.julia/packages/Zygote/KpME9/src/compiler/interface2.jl:0
[14] _pullback(::Function, ::Flux.Recur{Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}}) at /home/carlo/.julia/packages/Zygote/KpME9/src/compiler/interface.jl:33
[15] pullback(::Function, ::Flux.Recur{Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}}) at /home/carlo/.julia/packages/Zygote/KpME9/src/compiler/interface.jl:39
[16] gradient(::Function, ::Flux.Recur{Flux.RNNCell{typeof(tanh),CuArray{Float32,2},CuArray{Float32,1}}}) at /home/carlo/.julia/packages/Zygote/KpME9/src/compiler/interface.jl:48
[17] top-level scope at /home/carlo/.julia/packages/Flux/goUGu/test/cuda/curnn.jl:7
[18] top-level scope at /build/julia/src/julia-1.5.3/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1190
[19] include(::String) at ./client.jl:457
[20] top-level scope at /home/carlo/.julia/packages/Flux/goUGu/test/cuda/runtests.jl:13
[21] include(::String) at ./client.jl:457
[22] top-level scope at /home/carlo/.julia/packages/Flux/goUGu/test/runtests.jl:39
[23] top-level scope at /build/julia/src/julia-1.5.3/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
[24] top-level scope at /home/carlo/.julia/packages/Flux/goUGu/test/runtests.jl:38
[25] include(::String) at ./client.jl:457
[26] top-level scope at none:6 |
|
This may be a simple fix: try copying the old https://github.com/JuliaGPU/CUDA.jl/blob/v2.6.1/lib/cudnn/dropout.jl into rnncompat.jl. |
copied the old rnn.jl->rnncompat.jl for Flux compatibility
No description provided.