Make use of SparseArrays.jl

[MartinuzziFrancesco]

In a lot of initializers for ESNs the majority of the entries are zero (>90%). An earlier version of ReservoirComputing.jl used SparseArrays to build them, but this has since changed. It would help computation times to make full use of sparse matrices when possible. This could be also be left as a choice to the user, adding something like

#init_reservoir is any reservoir initializer here
julia> reservoir_matrix = init_reservoir(10, 10; return_sparse = true)
10×10 SparseMatrixCSC{Float32, Int64} with 10 stored entries:
  ⋅        -1.16794    ⋅          ⋅       …   ⋅          ⋅         ⋅       1.03808
  ⋅          ⋅         ⋅        -1.25129      ⋅          ⋅         ⋅        ⋅ 
  ⋅          ⋅         ⋅          ⋅          0.952151    ⋅         ⋅        ⋅ 
  ⋅          ⋅         ⋅          ⋅           ⋅          ⋅         ⋅        ⋅ 
  ⋅          ⋅         ⋅          ⋅           ⋅          ⋅         ⋅        ⋅ 
  ⋅          ⋅         ⋅          ⋅       …   ⋅          ⋅         ⋅        ⋅ 
  ⋅          ⋅         ⋅          ⋅           ⋅        -2.04221    ⋅        ⋅ 
 0.599564    ⋅         ⋅          ⋅           ⋅          ⋅       -1.06445   ⋅ 
  ⋅          ⋅         ⋅          ⋅           ⋅          ⋅         ⋅        ⋅ 
  ⋅          ⋅       -0.826278    ⋅           ⋅          ⋅         ⋅        ⋅ 

I think the reason it was removed before had to do with the regression training, so this would need to be tested on it

[MartinuzziFrancesco]

Initializers like pseudo_svd can even be represented as Diagonal

julia> reservoir_matrix = psuedo_svd(5, 5;
    return_sparse = false, #users may still need a sparse matrix
    return_diag = true) #would need a check that both aren't true at the same time
5×5 Diagonal{Float32, Vector{Float32}}:
 0.306998   ⋅         ⋅         ⋅         ⋅ 
  ⋅        0.325977   ⋅         ⋅         ⋅ 
  ⋅         ⋅        0.549051   ⋅         ⋅ 
  ⋅         ⋅         ⋅        0.726199   ⋅ 
  ⋅         ⋅         ⋅         ⋅        1.0

[MartinuzziFrancesco]

Should SparseArrays.jl be an external dependency? At the moment it's return_sparse=true as the default, but there's an argument to be made for reducing dependencies and keeping this an user choice

[ChrisRackauckas]

Yes as much as possible making it an extension is a pretty major latency improvement so we are trying to do that across the whole ecosystem

[MartinuzziFrancesco]

Yeah makes sense, I'll work on it this week