Making a new CUDA 3 compatible release?

[jonathan-laurent]

I see that the latest Knet release (1.4.6) is not compatible with CUDA 3.
The master version seems to work fine with CUDA 3.3.3 and AlphaZero.jl though.

Are there any known issues remaining to be fixed before Knet officially supports CUDA 3?
If not, what about making a new release?

[denizyuret]

Started working on this, just fixing a few minor incompatibilities and a possibly major one effecting inplace operations in test/karray.jl. Latest work at the dy/cudnn branch.

[denizyuret]

... hmm, and there is an issue of across the board slow-down, which may be related to aforementioned inplace operations or not. This is going to take a bit longer than I thought.

[jonathan-laurent]

I think I observed the slow-down you mentioned. Indeed, Knet used to be 20-30% faster than Flux on my connect four benchmark. However, when I tried Knet#master with CUDA 3.3.0, it was about 20% slower than the latest version of Flux. (You should take these numbers with a grain of salt though as my measurements weren't very rigorous.)

[denizyuret]

@maleadt Knet tests with CUDA 3.0 give me some errors that I do not understand, maybe you can point me in the right direction:

First pkg"test Knet" fails with errors of the following type when testing in-place addition:

  Test threw exception                                                                    
  Expression: (a4 .+= a3) == (k4 .+= k3)                                                  
  Scalar indexing is disallowed.                                                          
  Invocation of getindex resulted in scalar indexing of a GPU array.                      
  This is typically caused by calling an iterating implementation of a method.            
  Such implementations *do not* execute on the GPU, but very slowly on the CPU,           
  and therefore are only permitted from the REPL for prototyping purposes.                
  If you did intend to index this array, annotate the caller with @allowscalar.           

However when I include the individual test file with include(Knet.dir("test/karray.jl")), I do not get an error but just a warning and the tests pass:

julia> include(Knet.dir("test/karray.jl"))                                                
┌ Warning: Performing scalar indexing on task Task (runnable) @0x00007f338d034010.        
│ Invocation of getindex resulted in scalar indexing of a GPU array.                      
│ This is typically caused by calling an iterating implementation of a method.            
│ Such implementations *do not* execute on the GPU, but very slowly on the CPU,           
│ and therefore are only permitted from the REPL for prototyping purposes.                
│ If you did intend to index this array, annotate the caller with @allowscalar.           
└ @ GPUArrays ~/.julia/packages/GPUArrays/8dzSJ/src/host/indexing.jl:56                   
Test Summary: | Pass  Total                                                               
karray        |  318    318                                                               

None of this happened pre-CUDA-3.x.

• Did something change with the default behavior of allowscalar?
• Do you have any idea why Pkg.test would fail but including the failing test file would pass?
• Finally, none of these calls should result in scalar indexing in the first place, did anything change with array indexing?

(a4 .+= a3) == (k4 .+= k3)
(a4 .= a3) == (k4 .= k3)
(a4[:] .= a3[:]) == (k4[:] .= k3[:])
(a4[:, :] .= a3) == (k4[:, :] .= k3)

[maleadt]

Scalar iteration is now disallowed by default, but allowed with a warning in interactive sessions. This to facilitate debugging. But you can always force it to be off in your interactive session too by calling CUDA.allowscalar(false).

I don't recall changing iteration specifically, but lots has change since pre-3.0. Doesn't the backtrace tell you anything?

[denizyuret]

Thanks, that helps. The problem was with the == comparison of Array vs CuArray.

[denizyuret]

@jonathan-laurent The CUDA 3 compatible Knet-1.4.7 is passing tests, I will release today. The tests still seem slower but not for any simple reason I could detect. My profiling script Knet/prof/ops20.jl gives similar timings for 1.4.6 and 1.4.7. If you have something that can help me debug performance issues I can work on it for the next release.