I can’t multiply a CSR sparse matrix anymore

[rveltz]

Sanity checks (read this first, then remove this section)

• Always ensure you're using the latest version of CUDA.jl, and if possible, please
  check the master branch to see if your issue hasn't been resolved yet.

If your bug is still valid, please go ahead and fill out the template below.

Describe the bug

Hi,

I was asked on slack to open an issue. I can’t multiply a CSR sparse matrix anymore. The following used to work with CuArrays.jl, is there a workaround?

julia> Jpo_gpu * CuArray(orbitguess_f)
ERROR: MethodError: no method matching mv!(::Char, ::Float64, ::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}, ::Float64, ::CuArray{Float64,1}, ::Char)
Closest candidates are:
  mv!(::Char, ::Float64, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Float64}, ::CuArray{Float64,1}, ::Float64, ::CuArray{Float64,1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  mv!(::Char, ::Complex{Float64}, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Complex{Float64}}, ::CuArray{Complex{Float64},1}, ::Complex{Float64}, ::CuArray{Complex{Float64},1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  mv!(::Char, ::Complex{Float32}, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Complex{Float32}}, ::CuArray{Complex{Float32},1}, ::Complex{Float32}, ::CuArray{Complex{Float32},1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  ...
Stacktrace:
 [1] mul!(::CuArray{Float64,1}, ::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/interfaces.jl:12
 [2] *(::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}) at /home/rveltz/julia/usr/share/julia/stdlib/v1.4/LinearAlgebra/src/matmul.jl:51
 [3] top-level scope at REPL[1946]:1
 [4] eval(::Module, ::Any) at ./boot.jl:331
 [5] eval_user_input(::Any, ::REPL.REPLBackend) at /home/rveltz/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:86
 [6] run_backend(::REPL.REPLBackend) at /home/rveltz/.julia/packages/Revise/ucYAZ/src/Revise.jl:1184
 [7] top-level scope at REPL[1785]:0

To reproduce

The Minimal Working Example (MWE) for this bug:

using SparseArrays, LinearAlgebra
A = I+sprand(100,100,0.01)
x = rand(100)
Ag = CUDA.CUSPARSE.CuSparseMatrixCSR(A)
julia> Ag*CuArray(x)
ERROR: MethodError: no method matching mv!(::Char, ::Float64, ::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}, ::Float64, ::CuArray{Float64,1}, ::Char)
Closest candidates are:
  mv!(::Char, ::Float64, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Float64}, ::CuArray{Float64,1}, ::Float64, ::CuArray{Float64,1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  mv!(::Char, ::Complex{Float64}, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Complex{Float64}}, ::CuArray{Complex{Float64},1}, ::Complex{Float64}, ::CuArray{Complex{Float64},1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  mv!(::Char, ::Complex{Float32}, ::CUDA.CUSPARSE.CuSparseMatrixBSR{Complex{Float32}}, ::CuArray{Complex{Float32},1}, ::Complex{Float32}, ::CuArray{Complex{Float32},1}, ::Char) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/wrappers.jl:193
  ...
Stacktrace:
 [1] mul!(::CuArray{Float64,1}, ::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}) at /home/rveltz/.julia/packages/CUDA/dZvbp/lib/cusparse/interfaces.jl:12
 [2] *(::CUDA.CUSPARSE.CuSparseMatrixCSR{Float64}, ::CuArray{Float64,1}) at /home/rveltz/julia/usr/share/julia/stdlib/v1.4/LinearAlgebra/src/matmul.jl:51
 [3] top-level scope at REPL[1794]:1

Manifest.toml

Paste your Manifest.toml here, or accurately describe which version of CUDA.jl and its dependencies (GPUArrays.jl, GPUCompiler.jl, LLVM.jl) you are using.

Expected behavior

A clear and concise description of what you expected to happen.

Version info

Details on Julia:

# please post the output of:
versioninfo()
julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Silver 4215 CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

Details on CUDA:

# please post the output of:
julia> CUDA.versioninfo()

CUDA toolkit 11.0.3, artifact installation
CUDA driver 11.0.0
NVIDIA driver 450.51.6

Libraries:
- CUBLAS: 11.2.0
- CURAND: 10.2.1
- CUFFT: 10.2.1
- CUSOLVER: 10.6.0
- CUSPARSE: 11.1.1
- CUPTI: 13.0.0
- NVML: 11.0.0+450.51.6
- CUDNN: 8.0.2 (for CUDA 11.0.0)
- CUTENSOR: 1.2.0 (for CUDA 11.0.0)

Toolchain:
- Julia: 1.4.2
- LLVM: 8.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3
- Device support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75

1 device:
  0: Tesla V100-PCIE-32GB (sm_70, 31.730 GiB / 31.749 GiB available)

[maleadt]

Did you test on CUDA#master with Julia 1.6 as the bug filing template suggests?

[rveltz]

No, it is on a cluster and I dont control everything (quickly).

[maleadt]

Quickly downloading Julia 1.6 and installing CUDA.jl doesn't seem like that much effort.

This works on #master:

julia> Ag*CuArray(x)
100-element CuArray{Float64, 1}

Do note however that coverage of CUSPARSE is still not perfect, so feel free to open more issues or create PRs if anything else fails.