Array2D.copy is slow, you can use array.Clone

[dsyme]

The perf of the recurrent neural networks sample is dominated by Array2D.copy as part of DM addiiton as part of pushRec in DiffSharp.

1. It might be possible to use in-place addition. I'm not entirely sure which addition operation in pushRec is dominating, but in cases like this it looks like in-place addition might be appropriate:

                   d.A <- d.A + (v :?> DM)
   

2. Regardless, you're using Array2D.copy in DiffSharp and that seems to be slower than it should be since it does an initBased. That should be fixed in FSharp.Core. But in the meantime you can do this which seems 4x faster.

module Array2D =  
    let copyFast (array : 'T[,]) =  array.Clone() :?> 'T[,]

e.g.

#time "on"

let test1() = 
    let mutable res  = Array2D.zeroCreate<float32> 100 100 
    for i in 0 .. 1000 do   
        for j in 0 .. 100 do
            res <- Array2D.copy res

let test2() = 
    let mutable res  = Array2D.zeroCreate<float32> 100 100 
    for i in 0 .. 1000 do   
        for j in 0 .. 100 do
            res <- Array2D.copyFast res

test1()  // 4.4s
test2() // 0.98s

[gbaydin]

In-place operations are definitely needed. It's already on our list for DiffSharp, here:

https://github.com/DiffSharp/DiffSharp/blob/master/Roadmap.txt

The reverse adjoint update rule d.A <- d.A + (v :?> DM) is a very good example of how much it will be useful.

For the Array2D.copy issue, I will definitely follow your suggestion and make the change in DiffSharp until FSharp.Core is fixed.

[gbaydin]

This is fixed in DiffSharp 0.7.5.