CUDA 11.2 Support

[kmaehashi]

Tasks:

• Read CUDA Release Notes.
• Update CUDA driver in Linux (Jenkins) / Windows (FlexCI) test infrastructure (ask @kmaehashi).
• Update chainer-test to include the CUDA version. Support CUDA 11.2 and NCCL 2.8 chainer/chainer-test#635
• Update CuPy code to support the CUDA version.
  • Compute capability support: Check NVRTC Reference (--gpu-architecture option) and change the minimum value of supported compute capability defined in cupy/cuda/compiler.py if needed. Support CUDA 11.2 #4702
  • Update bundled FP16 header files. Copy FP16 header files to cupy/core/include/cupy/_cuda/, then update cupy/core/core.pyx to use it. Support CUDA 11.2 #4702
  • Backport. [backport] Support CUDA 11.2 #4708
• Update the library installer
  • Fix cupyx/tools/install_library.py Add cudnn and cutensor for CUDA 11.2 in install_library #4680
  • Backport [backport] Add cudnn and cutensor for CUDA 11.2 in install_library #4703
• Wheel distribution
  • Update cupy-release-tools Support CUDA 11.2 and NCCL 2.8 cupy-release-tools#66
  • Ask PyPI team to increase the limit (ask @kmaehashi) Limit Request: cupy-cuda112 - 250 MB pypi/support#939
• Documentation
  • Update Requirmenets section in Installtion Guide Add CUDA 11.2 to docs #4705
  • Add new wheel package to README.md. Add CUDA 11.2 to docs #4705
  • Backport of the two [backport] Add CUDA 11.2 to docs #4716
  • Add new wheel package to the website. Add cupy-cuda112 cupy.dev#32

Additional tasks:

• Use MSVC 14.0 with CUDA 11.2 #4698 and backport [backport] Use MSVC 14.0 with CUDA 11.2 #4701

[kmaehashi]

cudaMallocAsync and cudaFreeAsync in CUDA 11.2:
https://developer.nvidia.com/blog/enhancing-memory-allocation-with-new-cuda-11-2-features/

[kmaehashi]

Need to verify if cupy-cuda111 will work with CUDA 11.2

File name of some libraries have been changed between 11.1 and 11.2 (e.g. libnvrtc.so.11.1 in 11.1 and libnvrtc.so.11.2 in 11.2) so cupy-cuda111 won't run on CUDA 11.2.
c.f. #4043 (comment) and following discussions

[leofang]

cudaMallocAsync and cudaFreeAsync in CUDA 11.2:

From https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#cuda-general-new-features:

Stream ordered memory allocator: Added new APIs cudaMallocAsync() and cudaFreeAsync() to enable applications to order memory allocation and deallocation with other work launched into a CUDA stream. Provides significant performance improvements compared to cudaMalloc()/cudaFree(). Introduces the concept of memory pools to provide the application with more control over memory management. Each device has a default memory pool and custom memory pools can be created as needed.

Looks promising!

[leofang]

Other potentially useful additions or notes:

• Two new API functions have been added to get the list of architectures supported by the NVRTC library:
  • nvrtcGetNumSupportedArchs
  • nvrtcGetSupportedArchs
• Cooperative groups enhancement: cudaLaunchCooperativeKernel now enables simultaneous launch in multiple streams allowing multiple co-operative grids to run concurrently.
• nvcc now provides support for the following builtin functions, for providing optimization hints to the compiler:
  • __builtin_assume()
  • __assume()
  • __builtin_assume_aligned()

In addition, pre-existing support for __builtin_expect() is now documented. See Compiler Optimization Hint Functions for details.

• A preview version of a new tool, cu++filt, is included in this release. NVCC produces mangled names, appearing in PTX files, which do not strictly follow the mangling conventions of the Itanium ABI--and are thus not properly demangled by standard tools such as binutils' c++filt. Specifically, this is true for PTX function parameters. The new cu++filt utility will demangle all of these correctly. As this is a preview version of the utility, feedback is invited. For more information, see cu++filt.
• Support for Visual Studio versions <= 2015 has been dropped in CUDA 11.2, following its deprecation in CUDA 11.1.

[leofang]

Stream Ordered Memory Allocator

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY__POOLS.html#group__CUDART__MEMORY__POOLS

[leofang]

In addition to texture reference, I saw several additional deprecation warnings from cuSPARSE and Profiler with CUDA 11.2 (the log is interleaving due to the parallel build). Also, the built-in Thrust/CUB complains that -std=c++14 is not used.

output

    building 'cupy_backends.cuda.api.driver' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    building 'cupy_backends.cuda.api.runtime' extension
    creating build/temp.linux-x86_64-3.7/cupy_backends
    building 'cupy_backends.cuda.libs.cublas' extension
    building 'cupy_backends.cuda.libs.curand' extension
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda/api
    creating build/temp.linux-x86_64-3.7/cupy_backends/cuda/api
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/libs/cublas.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/cublas.o
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/libs/curand.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/curand.o
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/api/runtime.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/api/runtime.o
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/api/driver.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/api/driver.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cupy_backends/cuda/api/driver.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetAddress(intptr_t, intptr_t, size_t, int)’:
    cupy_backends/cuda/api/driver.cpp:6276:143: warning: ‘CUresult cuTexRefSetAddress_v2(size_t*, CUtexref, CUdeviceptr, size_t)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetAddress((&__pyx_v_ByteOffset), ((CUtexref_st*)__pyx_v_texref), ((CUdeviceptr)__pyx_v_dptr), __pyx_v_nbytes);
                                                                                                                                                   ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:124:45: note: declared here
     #define cuTexRefSetAddress                  cuTexRefSetAddress_v2
                                                 ^
    /usr/local/cuda-11.2/include/cuda.h:15543:36: note: in expansion of macro ‘cuTexRefSetAddress’
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddress(size_t *ByteOffset, CUtexref hTexRef, CUdeviceptr dptr, size_t bytes);
                                        ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetAddress2D(intptr_t, intptr_t, intptr_t, size_t, int)’:
    cupy_backends/cuda/api/driver.cpp:6480:168: warning: ‘CUresult cuTexRefSetAddress2D_v3(CUtexref, const CUDA_ARRAY_DESCRIPTOR*, CUdeviceptr, size_t)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetAddress2D(((CUtexref_st*)__pyx_v_texref), ((CUDA_ARRAY_DESCRIPTOR const *)__pyx_v_desc), ((CUdeviceptr)__pyx_v_dptr), __pyx_v_Pitch);
                                                                                                                                                                            ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:132:45: note: declared here
     #define cuTexRefSetAddress2D                cuTexRefSetAddress2D_v3
                                                 ^
    /usr/local/cuda-11.2/include/cuda.h:15598:36: note: in expansion of macro ‘cuTexRefSetAddress2D’
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddress2D(CUtexref hTexRef, const CUDA_ARRAY_DESCRIPTOR *desc, CUdeviceptr dptr, size_t Pitch);
                                        ^~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetAddressMode(intptr_t, int, int, int)’:
    cupy_backends/cuda/api/driver.cpp:6688:122: warning: ‘CUresult cuTexRefSetAddressMode(CUtexref, int, CUaddress_mode)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetAddressMode(((CUtexref_st*)__pyx_v_texref), __pyx_v_dim, ((CUaddress_mode)__pyx_v_am));
                                                                                                                              ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15679:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddressMode(CUtexref hTexRef, int dim, CUaddress_mode am);
                                        ^~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetArray(intptr_t, intptr_t, int)’:
    cupy_backends/cuda/api/driver.cpp:6885:171: warning: ‘CUresult cuTexRefSetArray(CUtexref, CUarray, unsigned int)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetArray(((CUtexref_st*)__pyx_v_texref), ((CUarray_st*)__pyx_v_array), __pyx_e_13cupy_backends_4cuda_3api_6driver_CU_TRSA_OVERRIDE_FORMAT);
                                                                                                                                                                               ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15467:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetArray(CUtexref hTexRef, CUarray hArray, unsigned int Flags);
                                        ^~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetBorderColor(intptr_t, PyObject*, int)’:
    cupy_backends/cuda/api/driver.cpp:7104:102: warning: ‘CUresult cuTexRefSetBorderColor(CUtexref, float*)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetBorderColor(((CUtexref_st*)__pyx_v_texref), __pyx_v_colors.data());
                                                                                                          ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15877:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetBorderColor(CUtexref hTexRef, float *pBorderColor);
                                        ^~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetFilterMode(intptr_t, int, int)’:
    cupy_backends/cuda/api/driver.cpp:7290:107: warning: ‘CUresult cuTexRefSetFilterMode(CUtexref, CUfilter_mode)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetFilterMode(((CUtexref_st*)__pyx_v_texref), ((CUfilter_mode)__pyx_v_fm));
                                                                                                               ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15715:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFilterMode(CUtexref hTexRef, CUfilter_mode fm);
                                        ^~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetFlags(intptr_t, unsigned int, int)’:
    cupy_backends/cuda/api/driver.cpp:7476:88: warning: ‘CUresult cuTexRefSetFlags(CUtexref, unsigned int)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetFlags(((CUtexref_st*)__pyx_v_texref), __pyx_v_Flags);
                                                                                            ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15922:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFlags(CUtexref hTexRef, unsigned int Flags);
                                        ^~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetFormat(intptr_t, int, int, int)’:
    cupy_backends/cuda/api/driver.cpp:7662:134: warning: ‘CUresult cuTexRefSetFormat(CUtexref, CUarray_format, int)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetFormat(((CUtexref_st*)__pyx_v_texref), ((CUarray_format)__pyx_v_fmt), __pyx_v_NumPackedComponents);
                                                                                                                                          ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15633:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFormat(CUtexref hTexRef, CUarray_format fmt, int NumPackedComponents);
                                        ^~~~~~~~~~~~~~~~~
    cupy_backends/cuda/api/driver.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_3api_6driver_texRefSetMaxAnisotropy(intptr_t, unsigned int, int)’:
    cupy_backends/cuda/api/driver.cpp:7859:99: warning: ‘CUresult cuTexRefSetMaxAnisotropy(CUtexref, unsigned int)’ is deprecated [-Wdeprecated-declarations]
             __pyx_v_status = cuTexRefSetMaxAnisotropy(((CUtexref_st*)__pyx_v_texref), __pyx_v_maxAniso);
                                                                                                       ^
    In file included from cupy_backends/cuda/api/../../cuda/cupy_cuda.h:4:0,
                     from cupy_backends/cuda/api/../../cupy_backend.h:10,
                     from cupy_backends/cuda/api/driver.cpp:679:
    /usr/local/cuda-11.2/include/cuda.h:15841:36: note: declared here
     __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetMaxAnisotropy(CUtexref hTexRef, unsigned int maxAniso);
                                        ^~~~~~~~~~~~~~~~~~~~~~~~
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/cupy_backends
    creating build/lib.linux-x86_64-3.7/cupy_backends/cuda
    creating build/lib.linux-x86_64-3.7/cupy_backends/cuda/libs
    g++ -pthread -shared -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -L/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,-rpath=/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/curand.o -L/usr/local/cuda-11.2/lib64 -lcublas -lcuda -lcudart -lcufft -lcurand -lcusparse -lnvrtc -o build/lib.linux-x86_64-3.7/cupy_backends/cuda/libs/curand.cpython-37m-x86_64-linux-gnu.so -Wl,--disable-new-dtags,-rpath,/usr/local/cuda-11.2/lib64
    building 'cupy_backends.cuda.libs.cusparse' extension
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/libs/cusparse.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/cusparse.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    creating build/lib.linux-x86_64-3.7/cupy_backends/cuda/api
    g++ -pthread -shared -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -L/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,-rpath=/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/cupy_backends/cuda/api/driver.o -L/usr/local/cuda-11.2/lib64 -lcublas -lcuda -lcudart -lcufft -lcurand -lcusparse -lnvrtc -o build/lib.linux-x86_64-3.7/cupy_backends/cuda/api/driver.cpython-37m-x86_64-linux-gnu.so -Wl,--disable-new-dtags,-rpath,/usr/local/cuda-11.2/lib64
    building 'cupy_backends.cuda.libs.nvrtc' extension
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/libs/nvrtc.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/nvrtc.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_sgthr(intptr_t, int, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:12935:203: warning: ‘cusparseStatus_t cusparseSgthr(cusparseHandle_t, int, const float*, float*, const int*, cusparseIndexBase_t)’ is deprecated: please use cusparseGather instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseSgthr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_nnz, ((float const *)__pyx_v_y), ((float *)__pyx_v_xVal), ((int const *)__pyx_v_xInd), ((cusparseIndexBase_t)__pyx_v_idxBase));
                                                                                                                                                                                                               ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:480:1: note: declared here
     cusparseSgthr(cusparseHandle_t    handle,
     ^~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_dgthr(intptr_t, int, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:13141:205: warning: ‘cusparseStatus_t cusparseDgthr(cusparseHandle_t, int, const double*, double*, const int*, cusparseIndexBase_t)’ is deprecated: please use cusparseGather instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDgthr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_nnz, ((double const *)__pyx_v_y), ((double *)__pyx_v_xVal), ((int const *)__pyx_v_xInd), ((cusparseIndexBase_t)__pyx_v_idxBase));
                                                                                                                                                                                                                 ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:489:1: note: declared here
     cusparseDgthr(cusparseHandle_t    handle,
     ^~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_cgthr(intptr_t, int, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:13347:211: warning: ‘cusparseStatus_t cusparseCgthr(cusparseHandle_t, int, const cuComplex*, cuComplex*, const int*, cusparseIndexBase_t)’ is deprecated: please use cusparseGather instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCgthr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_nnz, ((cuComplex const *)__pyx_v_y), ((cuComplex *)__pyx_v_xVal), ((int const *)__pyx_v_xInd), ((cusparseIndexBase_t)__pyx_v_idxBase));
                                                                                                                                                                                                                       ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:498:1: note: declared here
     cusparseCgthr(cusparseHandle_t    handle,
     ^~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zgthr(intptr_t, int, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:13553:223: warning: ‘cusparseStatus_t cusparseZgthr(cusparseHandle_t, int, const cuDoubleComplex*, cuDoubleComplex*, const int*, cusparseIndexBase_t)’ is deprecated: please use cusparseGather instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZgthr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_nnz, ((cuDoubleComplex const *)__pyx_v_y), ((cuDoubleComplex *)__pyx_v_xVal), ((int const *)__pyx_v_xInd), ((cusparseIndexBase_t)__pyx_v_idxBase));
                                                                                                                                                                                                                                   ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:507:1: note: declared here
     cusparseZgthr(cusparseHandle_t       handle,
     ^~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_createCsrgemm2Info(int)’:
    cupy_backends/cuda/libs/cusparse.cpp:32781:68: warning: ‘cusparseStatus_t cusparseCreateCsrgemm2Info(csrgemm2Info**)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
             __pyx_v_status = cusparseCreateCsrgemm2Info((&__pyx_v_info));
                                                                        ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3395:1: note: declared here
     cusparseCreateCsrgemm2Info(csrgemm2Info_t* info);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_destroyCsrgemm2Info(size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:32923:84: warning: ‘cusparseStatus_t cusparseDestroyCsrgemm2Info(csrgemm2Info_t)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
             __pyx_v_status = cusparseDestroyCsrgemm2Info(((csrgemm2Info_t)__pyx_v_info));
                                                                                        ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3399:1: note: declared here
     cusparseDestroyCsrgemm2Info(csrgemm2Info_t info);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_scsrgemm2_bufferSizeExt(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:33057:555: warning: ‘cusparseStatus_t cusparseScsrgemm2_bufferSizeExt(cusparseHandle_t, int, int, int, const float*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, const float*, cusparseMatDescr_t, int, const int*, const int*, csrgemm2Info_t, size_t*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseScsrgemm2_bufferSizeExt(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((float *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((float *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((csrgemm2Info_t)__pyx_v_info), (&__pyx_v_bufferSize));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3403:1: note: declared here
     cusparseScsrgemm2_bufferSizeExt(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_dcsrgemm2_bufferSizeExt(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:33403:557: warning: ‘cusparseStatus_t cusparseDcsrgemm2_bufferSizeExt(cusparseHandle_t, int, int, int, const double*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, const double*, cusparseMatDescr_t, int, const int*, const int*, csrgemm2Info_t, size_t*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDcsrgemm2_bufferSizeExt(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((double *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((double *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((csrgemm2Info_t)__pyx_v_info), (&__pyx_v_bufferSize));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3426:1: note: declared here
     cusparseDcsrgemm2_bufferSizeExt(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ccsrgemm2_bufferSizeExt(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:33749:563: warning: ‘cusparseStatus_t cusparseCcsrgemm2_bufferSizeExt(cusparseHandle_t, int, int, int, const cuComplex*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, const cuComplex*, cusparseMatDescr_t, int, const int*, const int*, csrgemm2Info_t, size_t*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCcsrgemm2_bufferSizeExt(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((cuComplex *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((cuComplex *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((csrgemm2Info_t)__pyx_v_info), (&__pyx_v_bufferSize));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3449:1: note: declared here
     cusparseCcsrgemm2_bufferSizeExt(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘size_t __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zcsrgemm2_bufferSizeExt(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:34095:575: warning: ‘cusparseStatus_t cusparseZcsrgemm2_bufferSizeExt(cusparseHandle_t, int, int, int, const cuDoubleComplex*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, const cuDoubleComplex*, cusparseMatDescr_t, int, const int*, const int*, csrgemm2Info_t, size_t*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZcsrgemm2_bufferSizeExt(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((cuDoubleComplex *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((cuDoubleComplex *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((csrgemm2Info_t)__pyx_v_info), (&__pyx_v_bufferSize));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3472:1: note: declared here
     cusparseZcsrgemm2_bufferSizeExt(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_xcsrgemm2Nnz(intptr_t, int, int, int, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, intptr_t, size_t, intptr_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:34451:601: warning: ‘cusparseStatus_t cusparseXcsrgemm2Nnz(cusparseHandle_t, int, int, int, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int, const int*, const int*, cusparseMatDescr_t, int*, int*, csrgemm2Info_t, void*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseXcsrgemm2Nnz(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((cusparseMatDescr_t)__pyx_v_descrC), ((int *)__pyx_v_csrRowPtrC), ((int *)__pyx_v_nnzTotalDevHostPtr), ((csrgemm2Info_t)__pyx_v_info), ((void *)__pyx_v_pBuffer));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3495:1: note: declared here
     cusparseXcsrgemm2Nnz(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_scsrgemm2(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, size_t, size_t, size_t, intptr_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:34822:753: warning: ‘cusparseStatus_t cusparseScsrgemm2(cusparseHandle_t, int, int, int, const float*, cusparseMatDescr_t, int, const float*, const int*, const int*, cusparseMatDescr_t, int, const float*, const int*, const int*, const float*, cusparseMatDescr_t, int, const float*, const int*, const int*, cusparseMatDescr_t, float*, const int*, int*, csrgemm2Info_t, void*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseScsrgemm2(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((float *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((float *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((float *)__pyx_v_csrValB), ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((float *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((float *)__pyx_v_csrValD), ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((cusparseMatDescr_t)__pyx_v_descrC), ((float *)__pyx_v_csrValC), ((int *)__pyx_v_csrRowPtrC), ((int *)__pyx_v_csrColIndC), ((csrgemm2Info_t)__pyx_v_info), ((void *)__pyx_v_pBuffer));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3519:1: note: declared here
     cusparseScsrgemm2(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_dcsrgemm2(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, size_t, size_t, size_t, intptr_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:35259:759: warning: ‘cusparseStatus_t cusparseDcsrgemm2(cusparseHandle_t, int, int, int, const double*, cusparseMatDescr_t, int, const double*, const int*, const int*, cusparseMatDescr_t, int, const double*, const int*, const int*, const double*, cusparseMatDescr_t, int, const double*, const int*, const int*, cusparseMatDescr_t, double*, const int*, int*, csrgemm2Info_t, void*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDcsrgemm2(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((double *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((double *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((double *)__pyx_v_csrValB), ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((double *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((double *)__pyx_v_csrValD), ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((cusparseMatDescr_t)__pyx_v_descrC), ((double *)__pyx_v_csrValC), ((int *)__pyx_v_csrRowPtrC), ((int *)__pyx_v_csrColIndC), ((csrgemm2Info_t)__pyx_v_info), ((void *)__pyx_v_pBuffer));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3549:1: note: declared here
     cusparseDcsrgemm2(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ccsrgemm2(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, size_t, size_t, size_t, intptr_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:35696:777: warning: ‘cusparseStatus_t cusparseCcsrgemm2(cusparseHandle_t, int, int, int, const cuComplex*, cusparseMatDescr_t, int, const cuComplex*, const int*, const int*, cusparseMatDescr_t, int, const cuComplex*, const int*, const int*, const cuComplex*, cusparseMatDescr_t, int, const cuComplex*, const int*, const int*, cusparseMatDescr_t, cuComplex*, const int*, int*, csrgemm2Info_t, void*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCcsrgemm2(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((cuComplex *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((cuComplex *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((cuComplex *)__pyx_v_csrValB), ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((cuComplex *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((cuComplex *)__pyx_v_csrValD), ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((cusparseMatDescr_t)__pyx_v_descrC), ((cuComplex *)__pyx_v_csrValC), ((int *)__pyx_v_csrRowPtrC), ((int *)__pyx_v_csrColIndC), ((csrgemm2Info_t)__pyx_v_info), ((void *)__pyx_v_pBuffer));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3579:1: note: declared here
     cusparseCcsrgemm2(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zcsrgemm2(intptr_t, int, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, int, size_t, size_t, size_t, size_t, size_t, size_t, size_t, size_t, intptr_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:36133:813: warning: ‘cusparseStatus_t cusparseZcsrgemm2(cusparseHandle_t, int, int, int, const cuDoubleComplex*, cusparseMatDescr_t, int, const cuDoubleComplex*, const int*, const int*, cusparseMatDescr_t, int, const cuDoubleComplex*, const int*, const int*, const cuDoubleComplex*, cusparseMatDescr_t, int, const cuDoubleComplex*, const int*, const int*, cusparseMatDescr_t, cuDoubleComplex*, const int*, int*, csrgemm2Info_t, void*)’ is deprecated: please use cusparseSpGEMM instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZcsrgemm2(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, __pyx_v_k, ((cuDoubleComplex *)__pyx_v_alpha), ((cusparseMatDescr_t)__pyx_v_descrA), __pyx_v_nnzA, ((cuDoubleComplex *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA), ((cusparseMatDescr_t)__pyx_v_descrB), __pyx_v_nnzB, ((cuDoubleComplex *)__pyx_v_csrValB), ((int *)__pyx_v_csrRowPtrB), ((int *)__pyx_v_csrColIndB), ((cuDoubleComplex *)__pyx_v_beta), ((cusparseMatDescr_t)__pyx_v_descrD), __pyx_v_nnzD, ((cuDoubleComplex *)__pyx_v_csrValD), ((int *)__pyx_v_csrRowPtrD), ((int *)__pyx_v_csrColIndD), ((cusparseMatDescr_t)__pyx_v_descrC), ((cuDoubleComplex *)__pyx_v_csrValC), ((int *)__pyx_v_csrRowPtrC), ((int *)__pyx_v_csrColIndC), ((csrgemm2Info_t)__pyx_v_info), ((void *)__pyx_v_pBuffer));
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:3609:1: note: declared here
     cusparseZcsrgemm2(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_scsc2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:36776:290: warning: ‘cusparseStatus_t cusparseScsc2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const float*, const int*, const int*, float*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseScsc2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((float const *)__pyx_v_cscSortedValA), ((int const *)__pyx_v_cscSortedRowIndA), ((int const *)__pyx_v_cscSortedColPtrA), ((float *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                      ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4196:1: note: declared here
     cusparseScsc2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_dcsc2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:37015:292: warning: ‘cusparseStatus_t cusparseDcsc2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const double*, const int*, const int*, double*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDcsc2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((double const *)__pyx_v_cscSortedValA), ((int const *)__pyx_v_cscSortedRowIndA), ((int const *)__pyx_v_cscSortedColPtrA), ((double *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                        ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4208:1: note: declared here
     cusparseDcsc2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ccsc2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:37254:298: warning: ‘cusparseStatus_t cusparseCcsc2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuComplex*, const int*, const int*, cuComplex*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCcsc2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuComplex const *)__pyx_v_cscSortedValA), ((int const *)__pyx_v_cscSortedRowIndA), ((int const *)__pyx_v_cscSortedColPtrA), ((cuComplex *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                              ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4220:1: note: declared here
     cusparseCcsc2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zcsc2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:37493:310: warning: ‘cusparseStatus_t cusparseZcsc2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuDoubleComplex*, const int*, const int*, cuDoubleComplex*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZcsc2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuDoubleComplex const *)__pyx_v_cscSortedValA), ((int const *)__pyx_v_cscSortedRowIndA), ((int const *)__pyx_v_cscSortedColPtrA), ((cuDoubleComplex *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                                          ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4232:1: note: declared here
     cusparseZcsc2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_scsr2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:39026:290: warning: ‘cusparseStatus_t cusparseScsr2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const float*, const int*, const int*, float*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseScsr2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((float const *)__pyx_v_csrSortedValA), ((int const *)__pyx_v_csrSortedRowPtrA), ((int const *)__pyx_v_csrSortedColIndA), ((float *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                      ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4096:1: note: declared here
     cusparseScsr2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_dcsr2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:39265:292: warning: ‘cusparseStatus_t cusparseDcsr2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const double*, const int*, const int*, double*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDcsr2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((double const *)__pyx_v_csrSortedValA), ((int const *)__pyx_v_csrSortedRowPtrA), ((int const *)__pyx_v_csrSortedColIndA), ((double *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                        ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4108:1: note: declared here
     cusparseDcsr2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ccsr2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:39504:298: warning: ‘cusparseStatus_t cusparseCcsr2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuComplex*, const int*, const int*, cuComplex*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCcsr2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuComplex const *)__pyx_v_csrSortedValA), ((int const *)__pyx_v_csrSortedRowPtrA), ((int const *)__pyx_v_csrSortedColIndA), ((cuComplex *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                              ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4120:1: note: declared here
     cusparseCcsr2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zcsr2dense(intptr_t, int, int, size_t, size_t, size_t, size_t, size_t, int, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:39743:310: warning: ‘cusparseStatus_t cusparseZcsr2dense(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuDoubleComplex*, const int*, const int*, cuDoubleComplex*, int)’ is deprecated: please use cusparseSparseToDense instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZcsr2dense(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuDoubleComplex const *)__pyx_v_csrSortedValA), ((int const *)__pyx_v_csrSortedRowPtrA), ((int const *)__pyx_v_csrSortedColIndA), ((cuDoubleComplex *)__pyx_v_A), __pyx_v_lda);
                                                                                                                                                                                                                                                                                                                          ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4132:1: note: declared here
     cusparseZcsr2dense(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_sdense2csc(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:42034:301: warning: ‘cusparseStatus_t cusparseSdense2csc(cusparseHandle_t, int, int, cusparseMatDescr_t, const float*, int, const int*, float*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseSdense2csc(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t const )__pyx_v_descrA), ((float const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerCol), ((float *)__pyx_v_cscValA), ((int *)__pyx_v_cscRowIndA), ((int *)__pyx_v_cscColPtrA));
                                                                                                                                                                                                                                                                                                                 ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4144:1: note: declared here
     cusparseSdense2csc(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ddense2csc(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:42284:303: warning: ‘cusparseStatus_t cusparseDdense2csc(cusparseHandle_t, int, int, cusparseMatDescr_t, const double*, int, const int*, double*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDdense2csc(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t const )__pyx_v_descrA), ((double const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerCol), ((double *)__pyx_v_cscValA), ((int *)__pyx_v_cscRowIndA), ((int *)__pyx_v_cscColPtrA));
                                                                                                                                                                                                                                                                                                                   ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4157:1: note: declared here
     cusparseDdense2csc(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_cdense2csc(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:42534:309: warning: ‘cusparseStatus_t cusparseCdense2csc(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuComplex*, int, const int*, cuComplex*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCdense2csc(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t const )__pyx_v_descrA), ((cuComplex const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerCol), ((cuComplex *)__pyx_v_cscValA), ((int *)__pyx_v_cscRowIndA), ((int *)__pyx_v_cscColPtrA));
                                                                                                                                                                                                                                                                                                                         ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4170:1: note: declared here
     cusparseCdense2csc(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zdense2csc(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:42784:321: warning: ‘cusparseStatus_t cusparseZdense2csc(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuDoubleComplex*, int, const int*, cuDoubleComplex*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZdense2csc(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t const )__pyx_v_descrA), ((cuDoubleComplex const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerCol), ((cuDoubleComplex *)__pyx_v_cscValA), ((int *)__pyx_v_cscRowIndA), ((int *)__pyx_v_cscColPtrA));
                                                                                                                                                                                                                                                                                                                                     ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4183:1: note: declared here
     cusparseZdense2csc(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_sdense2csr(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:43034:294: warning: ‘cusparseStatus_t cusparseSdense2csr(cusparseHandle_t, int, int, cusparseMatDescr_t, const float*, int, const int*, float*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseSdense2csr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((float const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerRow), ((float *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA));
                                                                                                                                                                                                                                                                                                          ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4044:1: note: declared here
     cusparseSdense2csr(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_ddense2csr(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:43284:296: warning: ‘cusparseStatus_t cusparseDdense2csr(cusparseHandle_t, int, int, cusparseMatDescr_t, const double*, int, const int*, double*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseDdense2csr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((double const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerRow), ((double *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA));
                                                                                                                                                                                                                                                                                                            ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4057:1: note: declared here
     cusparseDdense2csr(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_cdense2csr(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:43534:302: warning: ‘cusparseStatus_t cusparseCdense2csr(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuComplex*, int, const int*, cuComplex*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseCdense2csr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuComplex const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerRow), ((cuComplex *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA));
                                                                                                                                                                                                                                                                                                                  ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4070:1: note: declared here
     cusparseCdense2csr(cusparseHandle_t           handle,
     ^~~~~~~~~~~~~~~~~~
    cupy_backends/cuda/libs/cusparse.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8cusparse_zdense2csr(intptr_t, int, int, size_t, size_t, int, size_t, size_t, size_t, size_t, int)’:
    cupy_backends/cuda/libs/cusparse.cpp:43784:314: warning: ‘cusparseStatus_t cusparseZdense2csr(cusparseHandle_t, int, int, cusparseMatDescr_t, const cuDoubleComplex*, int, const int*, cuDoubleComplex*, int*, int*)’ is deprecated: please use cusparseDenseToSparse instead [-Wdeprecated-declarations]
       __pyx_v_status = cusparseZdense2csr(((cusparseHandle_t)__pyx_v_handle), __pyx_v_m, __pyx_v_n, ((cusparseMatDescr_t)__pyx_v_descrA), ((cuDoubleComplex const *)__pyx_v_A), __pyx_v_lda, ((int const *)__pyx_v_nnzPerRow), ((cuDoubleComplex *)__pyx_v_csrValA), ((int *)__pyx_v_csrRowPtrA), ((int *)__pyx_v_csrColIndA));
                                                                                                                                                                                                                                                                                                                              ^
    In file included from cupy_backends/cuda/libs/../../cuda/cupy_cusparse.h:5:0,
                     from cupy_backends/cuda/libs/../../cupy_sparse.h:6,
                     from cupy_backends/cuda/libs/cusparse.cpp:676:
    /usr/local/cuda-11.2/include/cusparse.h:4083:1: note: declared here
     cusparseZdense2csr(cusparseHandle_t         handle,
     ^~~~~~~~~~~~~~~~~~
    g++ -pthread -shared -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -L/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,-rpath=/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/nvrtc.o -L/usr/local/cuda-11.2/lib64 -lcublas -lcuda -lcudart -lcufft -lcurand -lcusparse -lnvrtc -o build/lib.linux-x86_64-3.7/cupy_backends/cuda/libs/nvrtc.cpython-37m-x86_64-linux-gnu.so -Wl,--disable-new-dtags,-rpath,/usr/local/cuda-11.2/lib64
    building 'cupy_backends.cuda.libs.profiler' extension
    gcc -pthread -B /home/leofang/miniconda3/envs/cupy_cuda112_dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FORCE_INLINES=1 -DCUPY_CUB_VERSION_CODE=101000 -DCUPY_JITIFY_VERSION_CODE=-1 -I/usr/local/cuda-11.2/include/cub -I/home/leofang/dev/cupy_cuda112/install/../cupy/core/include -I/usr/local/cuda-11.2/include -I/home/leofang/miniconda3/envs/cupy_cuda112_dev/include/python3.7m -c cupy_backends/cuda/libs/profiler.cpp -o build/temp.linux-x86_64-3.7/cupy_backends/cuda/libs/profiler.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    cupy_backends/cuda/libs/profiler.cpp: In function ‘PyObject* __pyx_f_13cupy_backends_4cuda_4libs_8profiler_initialize(PyObject*, PyObject*, int, int)’:
    cupy_backends/cuda/libs/profiler.cpp:1518:136: warning: ‘cudaError_t cudaProfilerInitialize(const char*, const char*, cudaOutputMode_t)’ is deprecated [-Wdeprecated-declarations]
       __pyx_v_status = cudaProfilerInitialize(((char const *)__pyx_t_2), ((char const *)__pyx_t_3), ((cudaOutputMode_t)__pyx_v_output_mode));
                                                                                                                                            ^
    In file included from cupy_backends/cuda/libs/../../cupy_profiler.h:10:0,
                     from cupy_backends/cuda/libs/profiler.cpp:674:
    /usr/local/cuda-11.2/include/cuda_profiler_api.h:134:57: note: declared here
     extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaProfilerInitialize(const char *configFile,
                                                             ^~~~~~~~~~~~~~~~~~~~~~

[nouiz]

Hi,

Current if we want a working cupy with CUDA 11.2, we need compile from scratch. The compilation of cupy is very slow. Would it be possible to make a minimal pipy release of cupy_cuda112 sooner? This will help the community try/adopt CUDA 11.2 faster.

From this threads, I do not see any big problems. All issues are to support new CUDA 11.2 features or warnings that can be ignore (but aren't ideal to have).

Did I miss something?
Thanks.

[leofang]

@nouiz CUDA provides a strong guarantee for forward compatibility since CUDA 11.0, meaning that applications built on CUDA 11.x can run on 11.y with y>=x, so in theory you could do pip install cupy-cuda111 in a CUDA 11.2 environment and it should just work. I haven't tested it myself, though.

[nouiz]

Good point. Thanks.

[eghbalhosseini]

Hi,
I have tested pip install cupy-cuda111 with cuda11.2 and got the following error :
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/cupy/init.py", line 20, in
from cupy import core # NOQA
File "/usr/local/lib/python3.7/site-packages/cupy/core/init.py", line 1, in
from cupy.core import core # NOQA
ImportError: libcublas.so.11: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.7/site-packages/cupy/init.py", line 41, in
raise ImportError(_msg) from e
ImportError: CuPy is not correctly installed.

If you are using wheel distribution (cupy-cudaXX), make sure that the version of CuPy you installed matches with the version of CUDA on your host.
Also, confirm that only one CuPy package is installed:
$ pip freeze

If you are building CuPy from source, please check your environment, uninstall CuPy and reinstall it with:
$ pip install cupy --no-cache-dir -vvvv

Check the Installation Guide for details:
https://docs.cupy.dev/en/latest/install.html

original error: libcublas.so.11: cannot open shared object file: No such file or directory

please let me know if there is a fix that could be applied here.

Thank you!

[outofculture]

@eghbalhosseini you currently must compile cupy from source to have 11.2 support; the e.g. cupy-cuda111 wheel has the library version compiled in.

[kmaehashi]

@takagi Could you please work on this?

[leofang]

btw CUDA 11.2.1 is out...