cholmod error 'problem too large' at file

[ncborcherding]

Had an email with this error.

Issue here is that the count matrices within the Seurat or SingleCellExperiment object is too large to convert to traditional matrix using the call as.matrix(). This will be an issue if you have a ton of cells or conversely, a ton of features.

Methods to circumvent

1. Summarize and Filter
Filter the sparse matrix for features that are expressed (or change 0 to percent of sample expressed)

expr_matrix <- integrated@assays$RNA@counts #sparsematrix
expr_matrix <- expr_matrix[tabulate(summary(expr_matrix)$i) != 0, , drop = FALSE] #remove any feature without a single count
expr_matrix <- as.matrix(expr_matrix) #convert to matrix

2. Split Seurat Object and Loop

Although not ideal - there is also a possibility of looping through the seurat objects or initial count matrices.

Seurat_loop <- SplitObject(integrated, split.by = "ident")
ES_list <- list[[]]
for (i in seq_along(Seurat_loop) {
           tmp <- enrichIt(ES_list[[i]], GS)
           ES_list[[i]] <- tmp
}

[atimms]

I get the cholmod error, so I attempted your solution, but then got the error message:

expr_matrix <- cbl_dev_clean@assays$RNA@counts #sparsematrix
expr_matrix <- expr_matrix[tabulate(summary(expr_matrix)$i) != 0, , drop = FALSE] #remove any feature without a single count
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'i' in selecting a method for function '[': $ operator is invalid for atomic vectors

Any ideas?

Thanks for providing this tool.

Andrew

[ncborcherding]

Hey Andrew,

Sorry for the delay - I actually forgot to put in the nessecary library

library(Matrix) #Here is the problem
expr_matrix <- integrated@assays$RNA@counts #sparsematrix
expr_matrix <- expr_matrix[tabulate(summary(expr_matrix)$i) != 0, , drop = FALSE] #remove any feature without a single count
expr_matrix <- as.matrix(expr_matrix) #convert to matrix

Hopefully that helps, I did check the code with and without the library(Matrix) and I was able to reproduce the error you saw. So I am hoping that is all the fix, but please let me know if you have any other issues.

Thanks
Nick

[atimms]

Hello Nick,

Thanks for getting back to me.

That resolved the error message as you expected unfortunately my dataset still seems to big, so I get the same Cholmod when running the as.matrix step.

I guess 70k cells must be too many and I don't want to split my seurat object into different samples.

Andrew

[ncborcherding]

If any user is still having this problem - there is no longer a conversion from a sparse matrix to a full matrix - this eliminates the error (will be in the dev branch of escape committing this afternoon).