Skip to content

[SYSTEMDS-3355] MatrixBlock size using CSR when allowed#1593

Closed
Baunsgaard wants to merge 2 commits into
apache:mainfrom
Baunsgaard:SparseCSRSize
Closed

[SYSTEMDS-3355] MatrixBlock size using CSR when allowed#1593
Baunsgaard wants to merge 2 commits into
apache:mainfrom
Baunsgaard:SparseCSRSize

Conversation

@Baunsgaard
Copy link
Copy Markdown
Contributor

This PR change/fix the size estimation to correctly use CSR size if that is allowed.
I have artificially limited it to only cases with more than one column,
since we have code paths that exploit the fact that we do not have any sparse single columns.

I personally would like this merged in, but i can see that it potentially reduce performance in some cases.

@mboehm7
Copy link
Copy Markdown
Contributor

mboehm7 commented Apr 21, 2022

I'll have a look in the next days - although I would prefer bringing this in after the release, I think with four eyes and the perftest experiments we should find any potential regression.

@mboehm7
Copy link
Copy Markdown
Contributor

mboehm7 commented May 15, 2022

LGTM - thanks for the patch @Baunsgaard. This is a good improvement of examSparsity while preserving the worst-case characteristic of other memory estimates.

@asfgit asfgit closed this in 7153749 May 15, 2022
@Baunsgaard Baunsgaard deleted the SparseCSRSize branch August 18, 2022 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants