Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: use dask.bag in Tessellation #296

Merged
merged 1 commit into from Aug 23, 2021
Merged

PERF: use dask.bag in Tessellation #296

merged 1 commit into from Aug 23, 2021

Conversation

martinfleis
Copy link
Member

Fixes #295

@matthew-law You were right, in my test, the single-core implementation was actually a bit faster than parallel. This seems to fix the problem.

I am a bit worried though. I used bag initially and then when moving the code to momepy, I have switched to dask.Series. I just don't remember why. Can you try this version on your data? My assumption is that it should go from 30 minutes to about 5.

@codecov
Copy link

codecov bot commented Aug 23, 2021

Codecov Report

Merging #296 (0af60d3) into main (3dd803b) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #296   +/-   ##
=======================================
  Coverage   95.66%   95.66%           
=======================================
  Files          13       13           
  Lines        2792     2792           
=======================================
  Hits         2671     2671           
  Misses        121      121           
Impacted Files Coverage Δ
momepy/elements.py 96.28% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3dd803b...0af60d3. Read the comment docs.

@matthew-law
Copy link
Contributor

@martinfleis even better than predicted, thanks for having a look and fixing the issue!

image

@martinfleis martinfleis merged commit e31d514 into main Aug 23, 2021
@martinfleis martinfleis deleted the bag_tess branch August 23, 2021 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

momepy.Tessellation doesn't appear to run faster on a machine with many more cores
2 participants