Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delay long list arguments #4735

Merged
merged 2 commits into from Apr 26, 2019

Conversation

Projects
None yet
2 participants
@mrocklin
Copy link
Member

commented Apr 25, 2019

This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes #4734

  • Tests added / passed
  • Passes flake8 dask
Delay long list arguments
This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes #4734
@mrocklin

This comment has been minimized.

Copy link
Member Author

commented Apr 25, 2019

cc @jcrist

@jcrist
Copy link
Member

left a comment

Overall this seems fine to me.

Show resolved Hide resolved dask/array/core.py

@mrocklin mrocklin force-pushed the mrocklin:list-arguments branch from e4e5203 to ce80f35 Apr 25, 2019

@jcrist

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

Looks good to me, merging. Thanks @mrocklin.

@jcrist jcrist merged commit 446103e into dask:master Apr 26, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@jcrist jcrist deleted the mrocklin:list-arguments branch Apr 26, 2019

jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this pull request May 14, 2019

Delay long list arguments (dask#4735)
* Delay long list arguments

This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes dask#4734

* use normalize_arg in map_partitions

Thomas-Z added a commit to Thomas-Z/dask that referenced this pull request May 17, 2019

Delay long list arguments (dask#4735)
* Delay long list arguments

This turns long lists into separate tasks to avoid repeated
serialization and traversal times.

Fixes dask#4734

* use normalize_arg in map_partitions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.