Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MemoryError in calculate_embedding_shift #188

Open
jacobkimmel opened this issue Apr 8, 2019 · 0 comments
Open

MemoryError in calculate_embedding_shift #188

jacobkimmel opened this issue Apr 8, 2019 · 0 comments

Comments

@jacobkimmel
Copy link

If the embedding attribute of a Velocyto object is set to a matrix with many dimensions, and many cells are present, this line in calculate_embedding_shift will try to allocate a huge array and fail with a cryptic MemoryError.

unitary_vectors = self.embedding.T[:, None, :] - self.embedding.T[:, :, None] # shape (2,ncells,ncells)

I'd suggest adding a check that the resulting EmbeddingDim, Cells, Cells array fits within a reasonable amount of memory, and printing a warning if it seems too large (i.e. any matrix that would require >32GB of memory could print a "WARNING - Embedding contains N dims and there are M cells, this is a huge matrix that may lead to a memory error").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant