Changes in Matrix 1.6-2 breaking irlba: `function 'as_cholmod_sparse' not provided by package 'Matrix'` #70

PeteHaitch · 2023-11-09T00:31:17Z

Changes in Matrix 1.6-2 seem to have broken irlba::irlba() for at least some types of sparse matrices.
I'm afraid I don't understand enough of Matrix and irlba to provide much help at this stage.

Cheers,
Pete

With Matrix 1.6-2

# Example taken from help("sparseMatrix", package = "Matrix").
i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7)
A <- Matrix::sparseMatrix(i, j, x = x)

irlba::irlba(A)
#> Warning in irlba::irlba(A): You're computing too large a percentage of total
#> singular values, use a standard svd instead.
#> Error in irlba::irlba(A): function 'as_cholmod_sparse' not provided by package 'Matrix'

Session info

sessionInfo()
#> R version 4.3.1 (2023-06-16)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: CentOS Linux 7 (Core)
#> 
#> Matrix products: default
#> BLAS:   /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so 
#> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so;  LAPACK version 3.11.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Australia/Melbourne
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.33     fastmap_1.1.1     Matrix_1.6-2      xfun_0.41        
#>  [5] lattice_0.21-8    glue_1.6.2        knitr_1.45        htmltools_0.5.7  
#>  [9] rmarkdown_2.25    lifecycle_1.0.4   cli_3.6.1         grid_4.3.1       
#> [13] reprex_2.0.2      withr_2.5.2       compiler_4.3.1    rstudioapi_0.15.0
#> [17] tools_4.3.1       irlba_2.3.5.1     evaluate_0.23     yaml_2.3.7       
#> [21] rlang_1.1.2       fs_1.6.3

With Matrix 1.6-1

# Example taken from help("sparseMatrix", package = "Matrix").
i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7)
A <- Matrix::sparseMatrix(i, j, x = x)

irlba::irlba(A)
#> Warning in irlba::irlba(A): You're computing too large a percentage of total
#> singular values, use a standard svd instead.
#> $d
#> [1] 49.00000 44.27189 35.00000 28.00000 21.00000
#> 
#> $u
#>               [,1]          [,2]          [,3]          [,4]          [,5]
#> [1,]  1.179612e-16 -6.418477e-17 -1.942890e-16  6.938894e-17  1.110223e-16
#> [2,]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00
#> [3,]  3.105155e-16  3.162278e-01 -6.245005e-17  1.110223e-16 -3.816392e-17
#> [4,] -1.734723e-16 -2.428613e-17 -3.885781e-16  5.273559e-16  1.000000e+00
#> [5,] -1.942890e-16 -2.602085e-16  9.436896e-16  1.000000e+00 -3.608225e-16
#> [6,] -2.498002e-16 -2.012279e-16 -1.000000e+00  1.054712e-15  2.775558e-17
#> [7,]  9.419548e-16  9.486833e-01 -1.318390e-16  2.914335e-16 -1.734723e-17
#> [8,]  1.000000e+00 -1.334002e-15 -3.885781e-16 -3.053113e-16  8.326673e-17
#> 
#> $v
#>                [,1]          [,2]          [,3]          [,4]          [,5]
#>  [1,] -6.938894e-17 -1.474515e-16 -1.665335e-16 -1.387779e-16  0.000000e+00
#>  [2,] -9.714451e-17 -9.020562e-17  2.220446e-16  5.551115e-17  1.665335e-16
#>  [3,]  6.071532e-18  1.192622e-17  2.081668e-17  1.040834e-17  1.734723e-18
#>  [4,]  1.127570e-17  1.474515e-17  3.122502e-17  2.775558e-17 -1.040834e-17
#>  [5,] -1.994932e-17 -4.770490e-17 -4.857226e-17 -1.734723e-17 -1.040834e-17
#>  [6,] -7.632783e-17  5.898060e-17 -3.330669e-16  6.938894e-16  1.000000e+00
#>  [7,] -8.326673e-17 -5.898060e-17  1.249001e-15  1.000000e+00 -3.053113e-16
#>  [8,] -3.816392e-16 -1.734723e-16 -1.000000e+00  9.714451e-16  2.775558e-17
#>  [9,]  1.065120e-15  1.000000e+00 -1.474515e-16  2.029626e-16 -1.908196e-16
#> [10,]  1.000000e+00 -1.234689e-15 -5.134781e-16  2.012279e-16  2.463307e-16
#> 
#> $iter
#> [1] 1
#> 
#> $mprod
#> [1] 16

Session info

sessionInfo()
#> R version 4.3.1 (2023-06-16)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: CentOS Linux 7 (Core)
#> 
#> Matrix products: default
#> BLAS:   /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so 
#> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so;  LAPACK version 3.11.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Australia/Melbourne
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.33     fastmap_1.1.1     Matrix_1.6-1      xfun_0.41        
#>  [5] lattice_0.21-8    glue_1.6.2        knitr_1.45        htmltools_0.5.7  
#>  [9] rmarkdown_2.25    lifecycle_1.0.4   cli_3.6.1         grid_4.3.1       
#> [13] reprex_2.0.2      withr_2.5.2       compiler_4.3.1    rstudioapi_0.15.0
#> [17] tools_4.3.1       irlba_2.3.5.1     evaluate_0.23     yaml_2.3.7       
#> [21] rlang_1.1.2       fs_1.6.3

The text was updated successfully, but these errors were encountered:

bwlewis · 2023-11-09T19:20:38Z

Thanks for reporting this! This will force me to revisit a project we started a while back at re-implementing the sparse parts of the code in a more generic way. I'll try and get that ready as soon as I can.

…

On 11/8/23, Peter Hickey ***@***.***> wrote: Hi @bwlewis, Changes in **Matrix** 1.6-2 seem to have broken `irlba::irlba()` for at least some types of sparse matrices. I'm afraid I don't understand enough of **Matrix** and **irlba** to provide much help at this stage. Cheers, Pete --------- # With **Matrix** 1.6-2 ``` r # Example taken from help("sparseMatrix", package = "Matrix"). i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7) A <- Matrix::sparseMatrix(i, j, x = x) irlba::irlba(A) #> Warning in irlba::irlba(A): You're computing too large a percentage of total #> singular values, use a standard svd instead. #> Error in irlba::irlba(A): function 'as_cholmod_sparse' not provided by package 'Matrix' ``` <details style="margin-bottom:10px;"> <summary> Session info </summary> ``` r sessionInfo() #> R version 4.3.1 (2023-06-16) #> Platform: x86_64-pc-linux-gnu (64-bit) #> Running under: CentOS Linux 7 (Core) #> #> Matrix products: default #> BLAS: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so #> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so; LAPACK version 3.11.0 #> #> locale: #> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C #> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 #> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 #> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C #> [9] LC_ADDRESS=C LC_TELEPHONE=C #> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C #> #> time zone: Australia/Melbourne #> tzcode source: system (glibc) #> #> attached base packages: #> [1] stats graphics grDevices utils datasets methods base #> #> loaded via a namespace (and not attached): #> [1] digest_0.6.33 fastmap_1.1.1 Matrix_1.6-2 xfun_0.41 #> [5] lattice_0.21-8 glue_1.6.2 knitr_1.45 htmltools_0.5.7 #> [9] rmarkdown_2.25 lifecycle_1.0.4 cli_3.6.1 grid_4.3.1 #> [13] reprex_2.0.2 withr_2.5.2 compiler_4.3.1 rstudioapi_0.15.0 #> [17] tools_4.3.1 irlba_2.3.5.1 evaluate_0.23 yaml_2.3.7 #> [21] rlang_1.1.2 fs_1.6.3 ``` </details> ------- # With **Matrix** 1.6-1 ``` r # Example taken from help("sparseMatrix", package = "Matrix"). i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7) A <- Matrix::sparseMatrix(i, j, x = x) irlba::irlba(A) #> Warning in irlba::irlba(A): You're computing too large a percentage of total #> singular values, use a standard svd instead. #> $d #> [1] 49.00000 44.27189 35.00000 28.00000 21.00000 #> #> $u #> [,1] [,2] [,3] [,4] [,5] #> [1,] 1.179612e-16 -6.418477e-17 -1.942890e-16 6.938894e-17 1.110223e-16 #> [2,] 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 #> [3,] 3.105155e-16 3.162278e-01 -6.245005e-17 1.110223e-16 -3.816392e-17 #> [4,] -1.734723e-16 -2.428613e-17 -3.885781e-16 5.273559e-16 1.000000e+00 #> [5,] -1.942890e-16 -2.602085e-16 9.436896e-16 1.000000e+00 -3.608225e-16 #> [6,] -2.498002e-16 -2.012279e-16 -1.000000e+00 1.054712e-15 2.775558e-17 #> [7,] 9.419548e-16 9.486833e-01 -1.318390e-16 2.914335e-16 -1.734723e-17 #> [8,] 1.000000e+00 -1.334002e-15 -3.885781e-16 -3.053113e-16 8.326673e-17 #> #> $v #> [,1] [,2] [,3] [,4] [,5] #> [1,] -6.938894e-17 -1.474515e-16 -1.665335e-16 -1.387779e-16 0.000000e+00 #> [2,] -9.714451e-17 -9.020562e-17 2.220446e-16 5.551115e-17 1.665335e-16 #> [3,] 6.071532e-18 1.192622e-17 2.081668e-17 1.040834e-17 1.734723e-18 #> [4,] 1.127570e-17 1.474515e-17 3.122502e-17 2.775558e-17 -1.040834e-17 #> [5,] -1.994932e-17 -4.770490e-17 -4.857226e-17 -1.734723e-17 -1.040834e-17 #> [6,] -7.632783e-17 5.898060e-17 -3.330669e-16 6.938894e-16 1.000000e+00 #> [7,] -8.326673e-17 -5.898060e-17 1.249001e-15 1.000000e+00 -3.053113e-16 #> [8,] -3.816392e-16 -1.734723e-16 -1.000000e+00 9.714451e-16 2.775558e-17 #> [9,] 1.065120e-15 1.000000e+00 -1.474515e-16 2.029626e-16 -1.908196e-16 #> [10,] 1.000000e+00 -1.234689e-15 -5.134781e-16 2.012279e-16 2.463307e-16 #> #> $iter #> [1] 1 #> #> $mprod #> [1] 16 ``` <details style="margin-bottom:10px;"> <summary> Session info </summary> ``` r sessionInfo() #> R version 4.3.1 (2023-06-16) #> Platform: x86_64-pc-linux-gnu (64-bit) #> Running under: CentOS Linux 7 (Core) #> #> Matrix products: default #> BLAS: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so #> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so; LAPACK version 3.11.0 #> #> locale: #> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C #> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 #> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 #> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C #> [9] LC_ADDRESS=C LC_TELEPHONE=C #> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C #> #> time zone: Australia/Melbourne #> tzcode source: system (glibc) #> #> attached base packages: #> [1] stats graphics grDevices utils datasets methods base #> #> loaded via a namespace (and not attached): #> [1] digest_0.6.33 fastmap_1.1.1 Matrix_1.6-1 xfun_0.41 #> [5] lattice_0.21-8 glue_1.6.2 knitr_1.45 htmltools_0.5.7 #> [9] rmarkdown_2.25 lifecycle_1.0.4 cli_3.6.1 grid_4.3.1 #> [13] reprex_2.0.2 withr_2.5.2 compiler_4.3.1 rstudioapi_0.15.0 #> [17] tools_4.3.1 irlba_2.3.5.1 evaluate_0.23 yaml_2.3.7 #> [21] rlang_1.1.2 fs_1.6.3 ``` </details> -- Reply to this email directly or view it on GitHub: #70 You are receiving this because you were mentioned. Message ID: ***@***.***>

PeteHaitch · 2023-11-09T20:31:26Z

Re-installing irlba from source after upgrading to Matrix v1.6-2 seems to have resolved the error.

# Example taken from help("sparseMatrix", package = "Matrix").
i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7)
A <- Matrix::sparseMatrix(i, j, x = x)

irlba::irlba(A)
#> Warning in irlba::irlba(A): You're computing too large a percentage of total
#> singular values, use a standard svd instead.
#> $d
#> [1] 49.00000 44.27189 35.00000 28.00000 21.00000
#> 
#> $u
#>               [,1]          [,2]          [,3]          [,4]          [,5]
#> [1,] -4.076600e-17 -1.873501e-16 -1.387779e-17 -2.775558e-17  1.387779e-16
#> [2,]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00
#> [3,]  8.023096e-18 -3.162278e-01 -1.942890e-16 -1.873501e-16  1.040834e-16
#> [4,]  3.469447e-18 -1.665335e-16  7.216450e-16  1.426637e-14 -1.000000e+00
#> [5,] -3.122502e-17 -1.249001e-16 -4.440892e-16  1.000000e+00  1.459943e-14
#> [6,]  1.301043e-16  5.689893e-16 -1.000000e+00 -4.440892e-16 -8.881784e-16
#> [7,] -6.938894e-18 -9.486833e-01 -4.857226e-16 -5.273559e-16  8.326673e-17
#> [8,]  1.000000e+00 -1.656661e-16  1.578598e-16 -2.081668e-17 -5.551115e-17
#> 
#> $v
#>                [,1]          [,2]          [,3]          [,4]          [,5]
#>  [1,] -7.047314e-18  1.214306e-17  3.469447e-18  6.938894e-18  6.938894e-18
#>  [2,]  2.081668e-17 -9.020562e-17  3.053113e-16  1.110223e-16  1.110223e-16
#>  [3,]  8.890458e-18 -1.214306e-17  1.387779e-17 -3.469447e-18 -5.551115e-17
#>  [4,] -4.250073e-17  5.204170e-17 -3.469447e-17  6.938894e-18  0.000000e+00
#>  [5,] -2.222614e-17  3.122502e-17 -1.734723e-17 -3.469447e-18  1.387779e-17
#>  [6,] -3.729655e-17 -5.551115e-17  1.054712e-15  1.915135e-14 -1.000000e+00
#>  [7,] -1.175275e-16 -9.714451e-17 -6.522560e-16  1.000000e+00  1.948441e-14
#>  [8,] -2.211772e-17  3.920475e-16 -1.000000e+00 -7.563394e-16 -1.415534e-15
#>  [9,]  3.642919e-17 -1.000000e+00 -8.187895e-16 -3.677614e-16  2.012279e-16
#> [10,]  1.000000e+00 -5.204170e-18  1.600282e-16  5.854692e-17 -8.847090e-17
#> 
#> $iter
#> [1] 1
#> 
#> $mprod
#> [1] 16

Session info

sessionInfo()
#> R version 4.3.1 (2023-06-16)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: CentOS Linux 7 (Core)
#> 
#> Matrix products: default
#> BLAS:   /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so 
#> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so;  LAPACK version 3.11.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Australia/Melbourne
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.33     fastmap_1.1.1     Matrix_1.6-2      xfun_0.41        
#>  [5] lattice_0.22-5    glue_1.6.2        knitr_1.45        htmltools_0.5.7  
#>  [9] rmarkdown_2.25    lifecycle_1.0.4   cli_3.6.1         grid_4.3.1       
#> [13] reprex_2.0.2      withr_2.5.2       compiler_4.3.1    rstudioapi_0.15.0
#> [17] tools_4.3.1       irlba_2.3.5.1     evaluate_0.23     yaml_2.3.7       
#> [21] rlang_1.1.2       fs_1.6.3

I never fully understand when packages have to be re-built/re-installed when dependencies like Matrix update.
But if I had to guess, perhaps these remaps (https://github.com/cran/Matrix/blob/323f4d9b0722ba6f61682874a8d92ebdbf93d8fa/inst/include/Matrix/remap.h) are picked up when irlba is re-installed from source.

bwlewis · 2023-11-09T23:59:47Z

Also good to know. There is a re-write of the sparse/Matrix-dependency code that has been planned for a long time that will simplify installation and handle more special structured matrix cases. I will work on this this fall.

…

On 11/9/23, Peter Hickey ***@***.***> wrote: Re-installing *irlba* from source after upgrading to *Matrix* v1.6-2 seems to have resolved the error. ``` r # Example taken from help("sparseMatrix", package = "Matrix"). i <- c(1,3:8); j <- c(2,9,6:10); x <- 7 * (1:7) A <- Matrix::sparseMatrix(i, j, x = x) irlba::irlba(A) #> Warning in irlba::irlba(A): You're computing too large a percentage of total #> singular values, use a standard svd instead. #> $d #> [1] 49.00000 44.27189 35.00000 28.00000 21.00000 #> #> $u #> [,1] [,2] [,3] [,4] [,5] #> [1,] -4.076600e-17 -1.873501e-16 -1.387779e-17 -2.775558e-17 1.387779e-16 #> [2,] 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 #> [3,] 8.023096e-18 -3.162278e-01 -1.942890e-16 -1.873501e-16 1.040834e-16 #> [4,] 3.469447e-18 -1.665335e-16 7.216450e-16 1.426637e-14 -1.000000e+00 #> [5,] -3.122502e-17 -1.249001e-16 -4.440892e-16 1.000000e+00 1.459943e-14 #> [6,] 1.301043e-16 5.689893e-16 -1.000000e+00 -4.440892e-16 -8.881784e-16 #> [7,] -6.938894e-18 -9.486833e-01 -4.857226e-16 -5.273559e-16 8.326673e-17 #> [8,] 1.000000e+00 -1.656661e-16 1.578598e-16 -2.081668e-17 -5.551115e-17 #> #> $v #> [,1] [,2] [,3] [,4] [,5] #> [1,] -7.047314e-18 1.214306e-17 3.469447e-18 6.938894e-18 6.938894e-18 #> [2,] 2.081668e-17 -9.020562e-17 3.053113e-16 1.110223e-16 1.110223e-16 #> [3,] 8.890458e-18 -1.214306e-17 1.387779e-17 -3.469447e-18 -5.551115e-17 #> [4,] -4.250073e-17 5.204170e-17 -3.469447e-17 6.938894e-18 0.000000e+00 #> [5,] -2.222614e-17 3.122502e-17 -1.734723e-17 -3.469447e-18 1.387779e-17 #> [6,] -3.729655e-17 -5.551115e-17 1.054712e-15 1.915135e-14 -1.000000e+00 #> [7,] -1.175275e-16 -9.714451e-17 -6.522560e-16 1.000000e+00 1.948441e-14 #> [8,] -2.211772e-17 3.920475e-16 -1.000000e+00 -7.563394e-16 -1.415534e-15 #> [9,] 3.642919e-17 -1.000000e+00 -8.187895e-16 -3.677614e-16 2.012279e-16 #> [10,] 1.000000e+00 -5.204170e-18 1.600282e-16 5.854692e-17 -8.847090e-17 #> #> $iter #> [1] 1 #> #> $mprod #> [1] 16 ``` <details style="margin-bottom:10px;"> <summary> Session info </summary> ``` r sessionInfo() #> R version 4.3.1 (2023-06-16) #> Platform: x86_64-pc-linux-gnu (64-bit) #> Running under: CentOS Linux 7 (Core) #> #> Matrix products: default #> BLAS: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRblas.so #> LAPACK: /stornext/System/data/apps/R/R-4.3.1/lib64/R/lib/libRlapack.so; LAPACK version 3.11.0 #> #> locale: #> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C #> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 #> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 #> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C #> [9] LC_ADDRESS=C LC_TELEPHONE=C #> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C #> #> time zone: Australia/Melbourne #> tzcode source: system (glibc) #> #> attached base packages: #> [1] stats graphics grDevices utils datasets methods base #> #> loaded via a namespace (and not attached): #> [1] digest_0.6.33 fastmap_1.1.1 Matrix_1.6-2 xfun_0.41 #> [5] lattice_0.22-5 glue_1.6.2 knitr_1.45 htmltools_0.5.7 #> [9] rmarkdown_2.25 lifecycle_1.0.4 cli_3.6.1 grid_4.3.1 #> [13] reprex_2.0.2 withr_2.5.2 compiler_4.3.1 rstudioapi_0.15.0 #> [17] tools_4.3.1 irlba_2.3.5.1 evaluate_0.23 yaml_2.3.7 #> [21] rlang_1.1.2 fs_1.6.3 ``` </details> I never fully understand when packages have to be re-built/re-installed when dependencies like *Matrix* update. But if I had to guess, perhaps these remaps (https://github.com/cran/Matrix/blob/323f4d9b0722ba6f61682874a8d92ebdbf93d8fa/inst/include/Matrix/remap.h) are picked up when *irlba* is re-installed from source. -- Reply to this email directly or view it on GitHub: #70 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

PeteHaitch · 2023-11-11T11:17:48Z

And a bit more info now posted by the Matrix developers: https://stat.ethz.ch/pipermail/r-package-devel/2023q4/010051.html

hpages · 2023-11-16T15:35:17Z

Thanks @PeteHaitch.

@bwlewis Note that it's enough for you to bump irlba version and resubmit to CRAN. The new version will then propagate to the user machine which will trigger a reinstallation.

The problem affects any user who installed irlba before Matrix 1.6-2 got into the wild and landed on their machine. Unfortunately the info posted by the Matrix developers won't help them, and it's not realistic to expect that the thousands of users in that situation reinstall irlba manually. Most users will see the error in a context involving many packages and won't have a clue what package is actually the culprit. This is the case for Bioconductor users where 40+ Bioconductor packages are affected indirectly via irlba.

However, if a new version of irlba shows up on CRAN, then the problem will be solved for all the users and system admins that run update.packages() on a regular basis.

Thanks,
H.

kvittingseerup · 2023-11-21T05:49:22Z

Is there any update on how to solve this problem?

sfplab · 2023-11-26T16:10:03Z

I tried reinstalling irlba, but the issue persists with Matrix 1.6-3. Major issue for anyone running UMAP.

PeteHaitch · 2023-11-26T21:23:00Z

@sfplab Did you re-install both from source?
This is what worked for me:

install.packages("Matrix", type = "source")
install.packages("irlba", type = "source")

sfplab · 2023-11-27T02:57:56Z

Hi @PeteHaitch - I hadn't installed from source, because it threw an error on my M1:

ld: warning: search path '/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0' not found
ld: warning: search path '/opt/R/arm64/gfortran/lib' not found
ld: library 'gfortran' not found

After going down some rabbit holes, I finally found this page on CRAN - downloading the gnu fortran package at the top of the page solved my issues installing Matrix and irlba from source:
https://cran.r-project.org/bin/macosx/tools/

Leaving this info here in case anyone else has issues with fortran-dependent source install in R. And yes, RunUMAP calls to Matrix & irlba now work as expected. Thank you, @PeteHaitch!

PeteHaitch · 2023-11-27T03:08:42Z

I use an M1 and I know that there's a lot of conflicting advice online re macOS, arm64 and R.
I try to follow the advice of Simon Urbanek (R/CRAN's macOS maintainer) because that's what's officially supported and works well for me.

kvittingseerup · 2023-11-28T11:33:24Z

I have an R package that depends on irlba working - is there any solution on the horizon that does not require all my users to install from source?

bwlewis · 2023-11-28T11:36:07Z

Yes, I am preparing an update for CRAN presently.

…

On Tue, Nov 28, 2023 at 6:33 AM Kristoffer Vitting-Seerup < ***@***.***> wrote: I have an R package that depends on irlba working - is there any solution on the horizon that does not require all my users to install from source? — Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABCZHR4FZBBTYAO2ILXV3TYGXDY7AVCNFSM6AAAAAA7DXI7PGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRZGY2DINRUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

zijunmeng · 2023-12-24T09:42:05Z

reinstall Matrix version = 1.6_1.1 can solve this problem.

qingmemng · 2024-01-16T01:15:59Z

Is there any solutions now ? I meet the same problem and it does not work even I changed Matrix version changes to 1.6-1.1.
seurat v 4.4.0 seuratobject v4.1.4
The below is the code.

scRNA1 <- RunUMAP(scRNA1, reduction = "pca", dims = 1:20)

Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
This message will be shown once per session
09:40:18 UMAP embedding parameters a = 0.9922 b = 1.112
09:40:18 Read 12279 rows and found 20 numeric columns
09:40:18 Using Annoy for neighbor search, n_neighbors = 30
09:40:18 Building Annoy index with metric = cosine, n_trees = 50
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
09:40:19 Writing NN index file to temp file C:\Users\Admini\AppData\Local\Temp\RtmpiCBaL8\filed0c4d6a7319
09:40:19 Searching Annoy index using 12 threads, search_k = 3000
09:40:19 Annoy recall = 100%
09:40:20 Commencing smooth kNN distance calibration using 12 threads with target n_neighbors = 30
09:40:20 Initializing from normalized Laplacian + noise (using irlba)
Error in irlba::irlba(L, nv = n, nu = 0, maxit = iters) :
function 'sexp_as_cholmod_sparse' not provided by package 'Matrix'

hpages · 2024-01-16T23:29:14Z

@bwlewis Any ETA for a new version of irlba on CRAN? Just to be clear, you don't need to make any change to the package, only bump its version and resubmit to CRAN. Thanks!

liuyifang · 2024-01-19T12:45:01Z

I am using mac M3, after install gfortran-12.2-universal.pkg
then I can install

install.packages("Matrix", type = "source")
install.packages("irlba", type = "source")

HaidongDeng · 2024-02-27T16:22:36Z

I am using mac M3, after install gfortran-12.2-universal.pkg then I can install
install.packages("Matrix", type = "source")
install.packages("irlba", type = "source")

I also met the same problem and your method solved it, thanks!

shnsk1995 · 2024-03-20T23:36:49Z

I have tried installing Matrix and irlba packages from source after installing gfortran-12.2-universal.pkg as suggested. However, I get the following error while trying to install Matrix package. Here is the screenshot. Can anyone help please?

shnsk1995 · 2024-03-21T00:16:52Z

Installing gfortran-ARM-13.2-Sonoma.dmg for Sonoma 14 macOS made it possible for me to install the Matrix and irlba packages from source and this in turn solved the error "function 'as_cholmod_sparse' not provided by package 'Matrix'".

I have tried installing Matrix and irlba packages from source after installing gfortran-12.2-universal.pkg as suggested. However, I get the following error while trying to install Matrix package. Here is the screenshot. Can anyone help please?

Thank you all!

hpages · 2024-04-19T22:16:03Z

@VitorAguiar Your comment is very misleading! Nobody said that you had to install Matrix v1.6_1.1 to fix the "as_cholmod_sparse" issue. If your version of Matrix is >= 1.6.2, all you need to do to fix this issue is reinstall irlba from source. And that does solve the problem.

Also why would you install an archived version of Matrix? Current version is 1.6-5 and this is what everybody uses so it should work with Seurat. The fact that Seurat v4.1.1 required at least v1.6-3 doesn't mean that you must install this exact version.

Anyways it's unfortunate that 5 months have passed and irlba version still didn't get bumped (with resubmission to CRAN) as this would have solved the problem once for all, and by doing so avoided a lot of confusion and frustration.

VitorAguiar · 2024-04-19T22:24:59Z

@hpages I was just sharing what worked for me. Your suggestion to reinstall irlba from source and current version of Matrix did not solve the problem for me. That's why I had to install an archived version of Matrix, one that was new enough to work with Seurat. Since you feel that my comment was misleading, I am deleting it.

hpages · 2024-04-19T23:12:16Z

@VitorAguiar The breaking change in Matrix happened when they went from version 1.6.1 to 1.6.2. So why installing Matrix 1.6.5 (current version) vs 1.6-3 would make any difference as far as the "as_cholmod_sparse" issue is concerned? Therefore when you say that

install.packages("Matrix", type = "source")
install.packages("irlba", type = "source")

didn't fix the problem for you, but that specifically installing Matrix 1.6.3 did solve it, I suspect that something else was going on in your case. And since we don't have the full story (i.e. output of commands + sessionInfo()) I guess we'll never know.

In any case it is very very unlikely that installing Matrix 1.6.3 instead of the current Matrix was necessary. Just wanted to clarify this in case other people seeing your comment might think this is a good idea .

RoganGrant · 2024-07-03T23:15:43Z

In case anyone is having issues with this solution and also has renv installed, beware that the source install may be ignored in favor of a cached version. This solved the issue for me: renv::install("irlba", type = "source", rebuild = TRUE)

alekseybelikov · 2024-07-16T17:21:15Z

Solved matrix errors by downgrading to version 1.6-5 and then installing irlba from source

install.packages("https://cran.r-project.org/src/contrib/Archive/Matrix/Matrix_1.6-5.tar.gz", repos = NULL, type = "source")

install.packages("irlba", type = "source")

PeteHaitch changed the title ~~Changes in **Matrix** 1.6-2 breaking **irlba**: function 'as_cholmod_sparse' not provided by package 'Matrix'~~ Changes in Matrix 1.6-2 breaking irlba: function 'as_cholmod_sparse' not provided by package 'Matrix' Nov 9, 2023

Michael-Geuenich mentioned this issue Nov 9, 2023

Matrix 1.6.2 update as_cholmod_sparse issue with runPCA alanocallaghan/scater#201

Closed

$@frac2738$ frac2738 mentioned this issue Nov 9, 2023

Matrix 1.6-2 breaking irlba GreenleafLab/ArchR#2050

Open

hpages mentioned this issue Nov 17, 2023

Recent changes to **Matrix** break installed SeuratObject satijalab/seurat-object#172

Closed

cyang-2014 mentioned this issue Nov 18, 2023

preprocess_cds Error in function 'as_cholmod_sparse' not provided by package 'Matrix' cole-trapnell-lab/monocle3#690

Open

alanocallaghan mentioned this issue Nov 21, 2023

runUMAP Error in irlba through 'Matrix' package on a non-conventional sce dataset OSCA-source/OSCA.basic#12

Closed

lambdamoses mentioned this issue Nov 22, 2023

Matrix >= 1.6.2 breaks Voyager pachterlab/voyager#15

Closed

plger mentioned this issue Nov 24, 2023

Unable to run scDblFinder - "as_cholmod_sparse" does not exist plger/scDblFinder#91

Open

jlmelville mentioned this issue Nov 27, 2023

irlba as_cholmod_sparse problems jlmelville/uwot#115

Open

SunShiFu86 mentioned this issue Dec 5, 2023

Error：SCP::RunCellQC(srt = pancreas_sub) zhanghao-njmu/SCP#206

Open

mluzvercesi mentioned this issue Dec 12, 2023

function 'as_cholmod_sparse' not provided by package 'Matrix' due to Matrix version chernolabs/scX#7

Closed

EmilHvitfeldt mentioned this issue Jan 10, 2024

skip tests for Matrix issue in irlba tidymodels/embed#211

Merged

lassedochreden mentioned this issue Jan 12, 2024

packages dosen't contain needed functions BodenmillerGroup/IMCDataAnalysis#88

Closed

sreichl mentioned this issue Jan 21, 2024

address scaling limitation epigen/mixscape_seurat#2

Closed

jiajic mentioned this issue Feb 7, 2024

getting error while using osmFISH Mouse SS Cortex tutorial drieslab/Giotto#875

Closed

rfriedman22 mentioned this issue Feb 9, 2024

UMAP fails with later version of irlba package cole-trapnell-lab/monocle3#701

Open

nigiord mentioned this issue Feb 27, 2024

AddDoubletScore in ArchR throws error irlba : <simpleError in irlba::irlba(mat, nDimensions, nDimensions): function 'as_cholmod_sparse' not provided by package 'Matrix'> #71

Open

evanbiederstedt mentioned this issue Mar 9, 2024

'as_cholmod_sparse' not provided by package 'Matrix' kharchenkolab/pagoda2#145

Closed

shwetha141 mentioned this issue Apr 11, 2024

Bdiv ordinal plot Umap cmmr/rbiom#18

Closed

lgeistlinger mentioned this issue Apr 18, 2024

add github action + pkgdown page ccb-hms/scDiagnostics#22

Closed

eds35016 mentioned this issue Jun 5, 2024

New Error with subsetting in Seurat V5 satijalab/seurat#8000

Closed

nigiord mentioned this issue Jul 8, 2024

SketchData() function error " function 'as_cholmod_sparse' not provided by package 'Matrix' " satijalab/seurat#9079

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changes in Matrix 1.6-2 breaking irlba: `function 'as_cholmod_sparse' not provided by package 'Matrix'` #70

Changes in Matrix 1.6-2 breaking irlba: `function 'as_cholmod_sparse' not provided by package 'Matrix'` #70

PeteHaitch commented Nov 9, 2023

bwlewis commented Nov 9, 2023 via email

PeteHaitch commented Nov 9, 2023 •

edited

Loading

bwlewis commented Nov 9, 2023 via email

PeteHaitch commented Nov 11, 2023

hpages commented Nov 16, 2023 •

edited

Loading

kvittingseerup commented Nov 21, 2023

sfplab commented Nov 26, 2023

PeteHaitch commented Nov 26, 2023

sfplab commented Nov 27, 2023 •

edited

Loading

PeteHaitch commented Nov 27, 2023

kvittingseerup commented Nov 28, 2023

bwlewis commented Nov 28, 2023 via email

zijunmeng commented Dec 24, 2023

qingmemng commented Jan 16, 2024 •

edited

Loading

hpages commented Jan 16, 2024

liuyifang commented Jan 19, 2024

HaidongDeng commented Feb 27, 2024

shnsk1995 commented Mar 20, 2024

shnsk1995 commented Mar 21, 2024

hpages commented Apr 19, 2024

VitorAguiar commented Apr 19, 2024 •

edited

Loading

hpages commented Apr 19, 2024 •

edited

Loading

RoganGrant commented Jul 3, 2024

alekseybelikov commented Jul 16, 2024

Changes in Matrix 1.6-2 breaking irlba: function 'as_cholmod_sparse' not provided by package 'Matrix' #70

Changes in Matrix 1.6-2 breaking irlba: function 'as_cholmod_sparse' not provided by package 'Matrix' #70

Comments

PeteHaitch commented Nov 9, 2023

With Matrix 1.6-2

With Matrix 1.6-1

bwlewis commented Nov 9, 2023 via email

PeteHaitch commented Nov 9, 2023 • edited Loading

bwlewis commented Nov 9, 2023 via email

PeteHaitch commented Nov 11, 2023

hpages commented Nov 16, 2023 • edited Loading

kvittingseerup commented Nov 21, 2023

sfplab commented Nov 26, 2023

PeteHaitch commented Nov 26, 2023

sfplab commented Nov 27, 2023 • edited Loading

PeteHaitch commented Nov 27, 2023

kvittingseerup commented Nov 28, 2023

bwlewis commented Nov 28, 2023 via email

zijunmeng commented Dec 24, 2023

qingmemng commented Jan 16, 2024 • edited Loading

hpages commented Jan 16, 2024

liuyifang commented Jan 19, 2024

HaidongDeng commented Feb 27, 2024

shnsk1995 commented Mar 20, 2024

shnsk1995 commented Mar 21, 2024

hpages commented Apr 19, 2024

VitorAguiar commented Apr 19, 2024 • edited Loading

hpages commented Apr 19, 2024 • edited Loading

RoganGrant commented Jul 3, 2024

alekseybelikov commented Jul 16, 2024

Changes in Matrix 1.6-2 breaking irlba: `function 'as_cholmod_sparse' not provided by package 'Matrix'` #70

Changes in Matrix 1.6-2 breaking irlba: `function 'as_cholmod_sparse' not provided by package 'Matrix'` #70

PeteHaitch commented Nov 9, 2023 •

edited

Loading

hpages commented Nov 16, 2023 •

edited

Loading

sfplab commented Nov 27, 2023 •

edited

Loading

qingmemng commented Jan 16, 2024 •

edited

Loading

VitorAguiar commented Apr 19, 2024 •

edited

Loading

hpages commented Apr 19, 2024 •

edited

Loading