Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delay long list arguments #4735

Merged
merged 2 commits into from Apr 26, 2019
Merged

Delay long list arguments #4735

merged 2 commits into from Apr 26, 2019

Conversation

@mrocklin
Copy link
Member

@mrocklin mrocklin commented Apr 25, 2019

This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes #4734

  • Tests added / passed
  • Passes flake8 dask
This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes dask#4734
@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Apr 25, 2019

cc @jcrist

Copy link
Member

@jcrist jcrist left a comment

Overall this seems fine to me.

dask/array/core.py Show resolved Hide resolved
@jcrist
Copy link
Member

@jcrist jcrist commented Apr 26, 2019

Looks good to me, merging. Thanks @mrocklin.

@jcrist jcrist merged commit 446103e into dask:master Apr 26, 2019
2 checks passed
@jcrist jcrist deleted the list-arguments branch Apr 26, 2019
jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this issue May 14, 2019
* Delay long list arguments

This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes dask#4734

* use normalize_arg in map_partitions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants