Skip to content

Micro-optimize computation of the rechunk graph#1827

Merged
mrocklin merged 18 commits intodask:masterfrom
pitrou:rechunk_micro_optimize
Dec 10, 2016
Merged

Micro-optimize computation of the rechunk graph#1827
mrocklin merged 18 commits intodask:masterfrom
pitrou:rechunk_micro_optimize

Conversation

@pitrou
Copy link
Copy Markdown
Member

@pitrou pitrou commented Dec 1, 2016

Nothing miraculous, but speeds up the graph computation by 60%. This helps performance when interacting with the client, since graph computation is synchronous.

Based on PR #1737, so need to merge that one first to review these changes.

@pitrou pitrou added the array label Dec 1, 2016
@pitrou pitrou force-pushed the rechunk_micro_optimize branch 4 times, most recently from 7623d65 to c634253 Compare December 1, 2016 14:52
@pitrou pitrou force-pushed the rechunk_micro_optimize branch from c634253 to 0cca8fc Compare December 1, 2016 15:54
@pitrou
Copy link
Copy Markdown
Member Author

pitrou commented Dec 2, 2016

I realize using something else than getitem disables the optimize_slices optimization (not that it seems to do anything on a rechunk graph), so I'll revert that part.

@mrocklin
Copy link
Copy Markdown
Member

Merging soon if no comments

@pitrou
Copy link
Copy Markdown
Member Author

pitrou commented Dec 10, 2016

I realized I had forgotten to push a small change, sorry.

@mrocklin mrocklin merged commit 21e9c5c into dask:master Dec 10, 2016
@sinhrks sinhrks added this to the 0.13.0 milestone Jan 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants