Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add edge_interior_ratio argument to TilePartitioner #44

Merged
merged 27 commits into from
Feb 3, 2022

Conversation

yniederm
Copy link
Contributor

@yniederm yniederm commented Dec 3, 2021

  • tests/test_partitioner.py updated to reflect some more advanced assignments.
  • subtile_extent method of Partitioner classes now takes in a required rank argument
  • TilePartitioner has a new edge_interior_ratio argument which defaults to 1.0, and lets the user specify the relative 1-dimensional extent of edge and corner compute domains compared to interior compute domains. In all cases, the closest valid value will be used, which enables some previously invalid configurations (e.g. C128 on a 3 by 3 layout will use the closest valid edge_interior_ratio to 1.0)
  • This can later be used in TilePartitioner.subtile_slice() to do custom subdomain partitioning.

This PR is replacing #33 with its feature branch moved from my fork to the pace repo.

@ofuhrer
Copy link
Contributor

ofuhrer commented Dec 3, 2021

Can one of the admins verify this patch?

@mcgibbon
Copy link
Collaborator

mcgibbon commented Dec 3, 2021

ok to test

@twicki
Copy link
Contributor

twicki commented Dec 3, 2021

launch jenkins

2 similar comments
@twicki
Copy link
Contributor

twicki commented Dec 3, 2021

launch jenkins

@twicki
Copy link
Contributor

twicki commented Dec 3, 2021

launch jenkins

@mcgibbon mcgibbon self-requested a review December 3, 2021 19:14
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/tests/test_partitioner.py Outdated Show resolved Hide resolved
@mcgibbon
Copy link
Collaborator

mcgibbon commented Dec 6, 2021

launch jenkins

fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/tests/test_partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
fv3gfs-util/fv3gfs/util/partitioner.py Outdated Show resolved Hide resolved
@mcgibbon mcgibbon changed the title Feature/shrinked edge partitioning V2 Add edge_interior_ratio argument to TilePartitioner Feb 2, 2022
@@ -7,13 +7,16 @@ latest
Major changes:
- Renamed DummyComm to LocalComm, and added support for message tags. The DummyComm symbol is still in place for backwards compatibility, but points to LocalComm
- added error in CubedSphereCommunicator init if given a communicator with a size not equal to the total ranks of the given partitioner
- `subtile_extent` method of Partitioner classes now takes in a required `rank` argument
- TilePartitioner has a new `edge_interior_ratio` argument which defaults to 1.0, and lets the user specify the relative 1-dimensional extent of edge and corner compute domains compared to interior compute domains. In all cases, the closest valid value will be used, which enables some previously invalid configurations (e.g. C128 on a 3 by 3 layout will use the closest valid edge_interior_ratio to 1.0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- TilePartitioner has a new `edge_interior_ratio` argument which defaults to 1.0, and lets the user specify the relative 1-dimensional extent of edge and corner compute domains compared to interior compute domains. In all cases, the closest valid value will be used, which enables some previously invalid configurations (e.g. C128 on a 3 by 3 layout will use the closest valid edge_interior_ratio to 1.0)
- TilePartitioner has a new `edge_interior_ratio` argument which defaults to 1.0, and lets the user specify the relative 1-dimensional extent of the boundaries (edge and corner) compute domains compared to interior compute domains. In all cases, the closest valid value will be used, which enables some previously invalid configurations (e.g. C128 on a 3 by 3 layout will use the closest valid edge_interior_ratio to 1.0)

Trying to align the wording with the code (e,g, ratio between interior and boundary tile sizes. in function comment). Boundary is a bit of a loaded term here, so you might want to flip for a constructed "edge-corner" concept or something better.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@FlorianDeconinck
Copy link
Contributor

Do we want to have at least one test using a non-default ratio that actually exchange data? Not quite a unit test per se, but it sounds safer to have at least one of those

@mcgibbon
Copy link
Collaborator

mcgibbon commented Feb 3, 2022

Good catch, I hadn't noticed there wasn't one. I'll add at least one each of a true MPI and mock MPI case.

Copy link
Contributor

@FlorianDeconinck FlorianDeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mcgibbon mcgibbon enabled auto-merge (squash) February 3, 2022 20:59
@mcgibbon
Copy link
Collaborator

mcgibbon commented Feb 3, 2022

launch jenkins

@mcgibbon mcgibbon merged commit 1bd99f1 into main Feb 3, 2022
@mcgibbon mcgibbon deleted the feature/shrinked_edge_partitioning branch February 3, 2022 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants