r.neighbors: Add parallel support #1654
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aaronsms
force-pushed
the
parallel-r.neighbor
branch
from
b2c2db7
to
171aba3
Compare
|
Nice! |
test_pygrass_modules_interface_doctests.py was failing, I am guessing this has to do with the newly added test cases. 20.04 build fail is incidental, due to connection issues to repository. |
aaronsms
force-pushed
the
parallel-r.neighbor
branch
from
171aba3
to
c4f1559
Compare
10 tasks
|
I will close this PR in favor of #1724. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
All input and output variables are transformed into pointers to support parallel execution. There is no change in the underlying algorithm except of the introduction of segmented temporary files to allow for random access write. So the extra memory usage will be scaled according to the number of threads used plus a constant overhead from the usage of segmented files. I think we should introduce a memory parameter to allow users to control the how much memory to be used for the segment. Right now, it is just a fixed constant, which is to be changed.
This PR contains an addition of a test file which contains the parallel version of the original sequential test cases. Upon addition of benchmark script, this will be combined into the original test file to avoid code duplication. On my machine, there is some speedup of ~2x on 4 threads, though this is not accurate without the benchmark.
I am thinking to refactor my code to encapsulate the usage of Segments.