Reshaping CuArray throws error during backpropagation

[dpk1729]

System information

• OS Platform and Distribution : Ubuntu 16.04.3 LTS
• CuArray version : 0.6.2
• Julia version: 0.6.4
• CUDA/cuDNN version: 8.0/6
• GPU model and memory: Tesla K40

The forward propagation contains reshape operations like below

P = reshape(exp.(logsoftmax(Aout[1:end])), (1,:))
u_k_combined = reshape(cat(u_k_lin, u_k_nonLin, dims=1), (1,:))
input_i = reshape(input[i,:], (1, edim))

These operations work fine on CPU during forward and back propagation.
On GPU it causes the following error

ERROR: LoadError: MethodError: Base._reshape(::CuArray{Float32,1}, ::Tuple{Int64}) is ambiguous. Candidates:
  _reshape(A::GPUArrays.GPUArray{T,N} where N, dims::Tuple{Vararg{Int64,N}} where N) where T in GPUArrays at /home/dpk1729/.julia/packages/GPUArrays/NHNmH/src/abstractarray.jl:236
  _reshape(A::GPUArrays.GPUArray{T,1}, dims::Tuple{Integer}) where T in GPUArrays at /home/dpk1729/.julia/packages/GPUArrays/NHNmH/src/abstractarray.jl:242
  _reshape(parent::CuArray, dims::Tuple{Vararg{Int64,N}} where N) in CuArrays at /home/dpk1729/.julia/packages/CuArrays/clDeS/src/array.jl:68
  _reshape(v::AbstractArray{T,1} where T, dims::Tuple{Int64}) in Base at reshapedarray.jl:148
Possible fix, define
  _reshape(::CuArray{T,1}, ::Tuple{Int64})
Stacktrace:
 [1] reshape(::CuArray{Float32,1}, ::Tuple{Int64}) at ./reshapedarray.jl:96
 [2] (::getfield(Flux.Tracker, Symbol("##285#290")){CuArray{Float32,1},Int64})(::TrackedArray{…,CuArray{Float32,1}}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/array.jl:177
 [3] iterate at ./generator.jl:47 [inlined]
 [4] collect(::Base.Generator{Tuple{TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}}},getfield(Flux.Tracker, Symbol("##285#290")){CuArray{Float32,1},Int64}}) at ./array.jl:619
 [5] JuliaGPU/CuArrays.jl#283 at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/array.jl:173 [inlined]
 [6] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##283#288")){Int64,Tuple{TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,1}},Flux.Tracker.Tracked{CuArray{Float32,1}}}}, ::CuArray{Float32,1}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:23
 [7] back(::Flux.Tracker.Tracked{CuArray{Float32,1}}, ::CuArray{Float32,1}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:45
 [8] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##293#294")){TrackedArray{…,CuArray{Float32,1}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,1}},Nothing}}, ::CuArray{Float32,2}) at ./abstractarray.jl:1844
 [9] back(::Flux.Tracker.Tracked{CuArray{Float32,2}}, ::CuArray{Float32,2}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:45
 [10] foreach at ./abstractarray.jl:1844 [inlined]
 [11] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##326#327")){TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,2}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,2}},Flux.Tracker.Tracked{CuArray{Float32,2}}}}, ::CuArray{Float32,2}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:26
 ... (the last 5 lines are repeated 1 more time)
 [17] back(::Flux.Tracker.Tracked{CuArray{Float32,1}}, ::CuArray{Float32,1}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:45
 [18] foreach(::Function, ::Tuple{Nothing,Flux.Tracker.Tracked{CuArray{Float32,1}},Nothing}, ::Tuple{CuArray{Float32,1},CuArray{Float32,1},Float32}) at ./abstractarray.jl:1844
 [19] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("#back#353")){3,getfield(Base.Broadcast, Symbol("##26#28")){getfield(Base.Broadcast, Symbol("##27#29")){typeof(*),getfield(Base.Broadcast, Symbol("##9#10")){getfield(Base.Broadcast, Symbol("##9#10")){getfield(Base.Broadcast, Symbol("##11#12"))}},getfield(Base.Broadcast, Symbol("##13#14")){getfield(Base.Broadcast, Symbol("##13#14")){getfield(Base.Broadcast, Symbol("##15#16"))}},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##3#4"))}}}},typeof(*)},Tuple{CuArray{Float32,1},TrackedArray{…,CuArray{Float32,1}},Int64}},Tuple{Nothing,Flux.Tracker.Tracked{CuArray{Float32,1}},Nothing}}, ::CuArray{Float32,1}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:26
 [20] back(::Flux.Tracker.Tracked{CuArray{Float32,1}}, ::CuArray{Float32,1}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:45
 [21] foreach at ./abstractarray.jl:1844 [inlined]
 [22] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##299#300")){TrackedArray{…,CuArray{Float32,1}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,1}}}}, ::Float32) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:26
 [23] back(::Flux.Tracker.Tracked{Float32}, ::Float32) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:43
 ... (the last 3 lines are repeated 1 more time)
 [27] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##194#195")){Flux.Tracker.TrackedReal{Float32},Int64},Tuple{Flux.Tracker.Tracked{Float32},Nothing}}, ::Float32) at ./abstractarray.jl:1844
 [28] back(::Flux.Tracker.Tracked{Float32}, ::Float32) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:43
 ... (the last 2 lines are repeated 1 more time)
 [31] foreach at ./abstractarray.jl:1844 [inlined]
 [32] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##202#203")),Tuple{Flux.Tracker.Tracked{Float32},Flux.Tracker.Tracked{Float32}}}, ::Float32) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:26
 [33] back(::Flux.Tracker.Tracked{Float32}, ::Float32) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:43
 ... (the last 3 lines are repeated 126 more times)
 [412] back!(::Flux.Tracker.TrackedReal{Float32}) at /home/dpk1729/.julia/packages/Flux/UHjNa/src/tracker/back.jl:62
 [413] train(::Data, ::Memory, ::Int64, ::Int64, ::Int64, ::Float64) at /home/dpk1729/code/MemN2N_model_gpu.jl:212
 [414] main() at /home/dpk1729/code/main.jl:68
 [415] top-level scope at none:0
 [416] include at ./boot.jl:317 [inlined]
 [417] include_relative(::Module, ::String) at ./loading.jl:1038
 [418] include(::Module, ::String) at ./sysimg.jl:29
 [419] exec_options(::Base.JLOptions) at ./client.jl:239
in expression starting at /home/dpk1729/code/main.jl:72

[MikeInnes]

The core issue in an ambiguity with a GPUArrays method – Base._reshape(::CuArray{Float32,1}, ::Tuple{Int64}) is ambiguous.. @SimonDanisch do you know what the best fix is for this?

[ChrisRackauckas]

Where is the CuArray reshape defined? I can add a method but don't know where I am looking for this.

[KristofferC]

https://github.com/JuliaGPU/CuArrays.jl/blob/bd5f82ca1ca7a2bd8f20d6087308ba4cdb44c6e7/src/array.jl#L126-L131 is where the ambiguous definition used to be.

[ChrisRackauckas]

This in principle should be fixed. At least, tests now show that reshaping a CuVector to itself now works without ambiguity. We can either close or directly test this on Flux.