Halos - making better use of halos to improve parallel performance #220
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As noted in #218, there are a lot of exchange calls that could be removed if the halos were being used more efficiently. This PR does just that, and partially closes #218.
There are a few bugs fixed in this PR:
eta_star
values from the halo that had never been computed. Oops. It's fixed by wrappingeta_star
in that subroutine. The performance hit is irrelevant, since the internal solver is orders of magnitude slower, and can only be used on a single core.The real meat of this PR is flushing many exchange calls, which leads to substantial improvements in scaling and performance improvement when running in parallel. The 1.5 layer version is now faster when running on multiple cores than it is on a single core, which is good. The previous scaling was embarrassing.
Still to do: