Error in quadprog::solve.QP(D, d, A, bzero) : matrix D in quadratic function is not positive definite!

[bmill3r]

Hello,

I am trying to perform deconvolution on some spatial data. I have had success running Giotto::runDWLSDeconv() previosuly but now I am using a larger signature matrix and it appears that quadprog::solve.QP() is complaining about a matrix not being positive definite, which is something I don't have much experience nor understand what it means exactly.

The full error message is:

Error in quadprog::solve.QP(D, d, A, bzero) : matrix D in quadratic function is not positive definite!
6.
stop("matrix D in quadratic function is not positive definite!")
5.
quadprog::solve.QP(D, d, A, bzero)
4.
solve_OLS_internal(S, B)
3.
optimize_solveDampenedWLS(S_k, B[uniq_ct_k_gene, ], constant_J)
2.
spot_deconvolution(expr = filter_expr, cluster_info = cluster, ct_exp = filter_Sig, binary_matrix = binarize_proportion)
1.
Giotto::runDWLSDeconv(gobject = giottoobj_mpoa100um, expression_values = c("normalized"), logbase = 2, cluster_column = "cluster_9", sign_matrix = t(neuronalCellTypes_100um$gtCtGenes), n_cell = 50, cutoff = 2, name = NULL, return_gobject = TRUE)

The command I am running is:

giottoobj <- Giotto::runDWLSDeconv(gobject = giottoobj,
                                    expression_values = c('normalized'), # selects @norm_expr 
                                    logbase = 2,
                                    cluster_column = 'cluster_9', # 'leiden_clus'
                                    sign_matrix = enrich_matrix,
                                    n_cell = 50,
                                    cutoff = 2,
                                    name = NULL,
                                    return_gobject = TRUE)

Here, I'm selecting the log2 transformed and normalized expression matrix (giottoobj@norm_expr), which is 135 genes x 3072 pixels. I previously clustered these pixels into 9 clusters, which are being selected via cluster_column = 'cluster_9' (giottoobj@cell_metadata$cluster_9).

The enrich_matrix is currently a 135 gene (same genes in giottoobj@norm_expr) by 76 cell-type binary (0/1) gene enrichment matrix. I have also used a gene expression matrix for these same 135 genes and 76 cell-types but receive the same error. All cell-types are enriched in at least two of the genes. Most of the genes are specifically enriched in one cell-type.

Do you know what might be going on? I've gone through the Giotto source code but it's pretty dense to keep track of the variables as they are passed to internal functions, and I cannot execute the internal functions themselves to investigate as many are not exported and thus not callable independently.

Thanks in advance

[bmill3r]

Received another error related to quadprog::solve.QP when trying to deconvolve a larger spatial dataset and a larger reference.
Error in quadprog::solve.QP(D/sc, d/sc, A, bzero) : NA/NaN/Inf in foreign function call (arg 1)

Call to Giotto::runDWLSDeconv was same as before, but this time using a spatial dataset of 135 genes x 45978 pixels and a signature matrix of 19234 genes x 23 cell-types. Note that these genes were selected from a scRNA-seq reference using Seurat::FindAllMarkers(). If this being too large could be contributing to the issue, then perhaps some guidance on selecting marker genes in the signature matrix would be helpful.

Thank you and sorry for the trouble!

[gcyuan]

Hi, If this is a general question re positive definitive, you could try to use the following tip to see if this solves the problem. Alternatively, we could help you troubleshoot if you could send us the data in some way (for example, removing the gene and cell-type names so that the data remain private). https://stackoverflow.com/questions/39129253/error-in-solve-qp

[bmill3r]

gobject.tar.gz

Thanks for getting back to me. My question is less about positive definite matrices in general and more about why it is occurring in Giotto::runDWLSDeconv when larger matrices are being used.

To this post I have added a gzipped file of the giottoobj and enrich_matrix I have been using, both of which have been de-identified in terms of the cell and gene names.

Both files have worked in conjunction with other enrich files or gioobjects, respectively, but here when used together quadprog::solve.QP complains about matrix D in quadratic function is not positive definite!.

The command I have been using is:

giottoobj <- Giotto::runDWLSDeconv(gobject = giottoobj,
                                    expression_values = c('normalized'), # selects @norm_expr 
                                    logbase = 2,
                                    cluster_column = 'cluster_9', # 'leiden_clus'
                                    sign_matrix = enrich_matrix,
                                    n_cell = 50,
                                    cutoff = 2,
                                    name = NULL,
                                    return_gobject = TRUE)

If you have any insight into why a non-positive definite matrix is being generated using this combination of normalized ST counts and enrich matrix I would greatly appreciate it!

Thank you

[ndelrossi7]

Hi @bmill3r - I wasn't able to download the data you posted. (Edited) I noticed above you were using a binary matrix (for more information on expected input for runDWLSDeconv() please see this paper.
Please install the Giotto suite branch and create your signature matrix using the function makeSignMatrixDWLS() if using data from a processed Giotto Object, or makeSignMatrixDWLSfromMatrix() if your processed data is elsewhere. You can find more information about those functions in this script.

Can you give this a try and let us know if you are still having any problems?
Thanks!

[bmill3r]

Thanks for getting back to me,

I actually tried running Giotto::runDWLSDeconv() using both the binary signature matrix and the expression matrix of my scRNA-seq reference (it was not immediately clear what the sign_matrix parameter in runDWLSDeconv() needed to be).

In either case, I still receive the same error. Note that this error occurs when I use a signature matrix of 135 genes and 76 cell-types on a larger spatial dataset but not when I use a smaller spatial dataset. For instance, my smaller spatial dataset is 135 genes x 3072 pixels whereas my larger one is 135 genes x 13477 pixels. Conversely, if I use the larger spatial dataset (or the smaller one) but a smaller signature matrix of 135 genes and 9 cell-types, it works. So I don't think it is the spatial dataset or the signature matrix specifically but it is the combination of both the larger signature matrix and the larger spatial dataset.

If I use an even bigger spatial dataset, I get a different error: Error in quadprog::solve.QP(D/sc, d/sc, A, bzero) : NA/NaN/Inf in foreign function call (arg 1)

I am using Giotto version 1.0.4 if that helps.

I can try to send you the data another way if you are having trouble downloading from GitHub directly.

Thanks again

[ndelrossi7]

Hi @bmill3r, thanks for clarifying! Our Giotto suite branch includes the latest updates for the DWLS functions.
As previously mentioned, please install the Giotto suite branch. You can do so by running the following:
remotes::install_github("RubD/Giotto@suite")

From there, you can create your signature matrix either from data pre-processed with Giotto (makeSignMatrixDWLS()) or from your own data processed elsewhere (makeSignMatrixDWLSfromMatrix()).

Below, I have written an explanation for each parameter used in both of the above functions mentioned.

makeSignMatrixDWLS(gobject = Giotto Object, 
                   expression_values = expression values to use, 
                   reverse_log = default is set to TRUE, 
                   log_base = default is set to 2, 
                   sign_gene = marker genes to use, 
                   cell_type_vector = vector of cell type for each cell in your dataset (length = ncol(matrix)))

makeSignMatrixDWLSfromMatrix(matrix = scRNA-seq matrix, 
                             sign_gene = marker genes to use, 
                             cell_type_vector = vector of cell type for each cell in your dataset (length = ncol(matrix)))

Let me know how this goes or if you have any further questions.

Thank you for the feedback, we are working on improving our documentation to be the clearest possible and we are making this a priority.

[bmill3r]

Hi again,

Unfortunately installation of the Giotto suite branch, generation of a signature matrix via makeSignMatrixDWLSfromMatrix from my own single cell data, and then rerunning runDWLSDeconv still resulted in the same error Error in quadprog::solve.QP(Dmat = D, dvec = d, Amat = A, bvec = bzero) : matrix D in quadratic function is not positive definite!. This was using my larger signature matrix of 76 cell-types, but my smaller signature matrix of 9 cell-types does work as it did before. I confirmed that my previous results stayed the same using the smaller signature matrix, using the suite branch, and making my own signature matrix from the single-cell expression data.

Some minor comments:
Installation of the suite branch changed the slots of my giotto objects such that gobject@norm_expr, gobject@custom_expr, gobject@scaled_expr no longer exist and it seems that the gene expression matrices I had there in my objects are gone. I think I was able to get these back in new slots gobject@expression$rna. Had a similar issue getting the right clusters assigned to column in gobject@cell_metadata which is now gobject@cell_metadata$rna

[ndelrossi7]

Hi, thanks for trying that out! If you would like, I would be happy to take a closer look with your specific data. Perhaps you can either share a Google drive link or email me the files at natalie.delrossi@mssm.edu.

Regarding the slots, these changes reflect the development of the suite branch to accommodate multiple modalities (such as protein expression). You can access the data using gobject@slot$rna.

[bmill3r]

I have sent the data via google drive and also a separate email with some information about the files just in case. Any help would be appreciated. Thank you for all of your assistance!