Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add RRS6to18 and RRSwISC6to18 meshes #576

Merged
merged 19 commits into from
Aug 15, 2023
Merged

Add RRS6to18 and RRSwISC6to18 meshes #576

merged 19 commits into from
Aug 15, 2023

Conversation

xylar
Copy link
Collaborator

@xylar xylar commented Mar 24, 2023 •
edited

In a later PR, these will be fully vetted and become the E3SMv3 versions of the oRRS18to6v3 mesh.

Checklist

  • User's Guide has been updated
  • Developer's Guide has been updated
  • API documentation in the Developer's Guide (api.rst) has any new or modified class, method and/or functions listed
  • Documentation has been built locally and changes look as expected
  • Document (in a comment titled Testing in this PR) any testing that was used to verify the changes
  • New tests have been added to a test suite

@xylar xylar added enhancement New feature or request ocean in progress This PR is not ready for review or merging labels Mar 24, 2023
@xylar
Copy link
Collaborator Author

xylar commented Mar 24, 2023

Based on #575, so needs to be rebased after than PR gets merged.

@xylar xylar force-pushed the add_rrs6to18 branch 2 times, most recently from b82df6e to c7de2aa Compare March 29, 2023 12:16
@xylar xylar removed the in progress This PR is not ready for review or merging label Mar 29, 2023
@xylar xylar marked this pull request as ready for review March 29, 2023 12:16
@xylar xylar self-assigned this Mar 29, 2023
@xylar xylar mentioned this pull request Jul 4, 2023
Merged
@xylar
Copy link
Collaborator Author

xylar commented Jul 25, 2023

Progress:

  • base mesh - takes 28 hours (on a login node)
  • remap topography - takes 4096 cores
  • culled mesh - requires using the step directory as the temp directory for cull() calls
  • initial condition - hangs on loading tracer lat/lon from woa23.nc
  • performance test
  • dynamic adjustment
  • files for E3SM

@xylar xylar force-pushed the add_rrs6to18 branch 2 times, most recently from e11aa1b to ee813d3 Compare August 1, 2023 10:06
@xylar xylar mentioned this pull request Aug 3, 2023
Merged
1 task
@xylar
Copy link
Collaborator Author

xylar commented Aug 9, 2023 •
edited

Testing

The full set of test cases is being run on Chrysalis at:

/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230809/rrswisc6to18

Progress so far:

  • base mesh
  • remap topography
  • culled mesh
  • initial condition
  • performance test
  • data ice-shelf fluxes
  • dynamic adjustment
  • files for E3SM -- seaice_graph_partition isn't done but proceeding as expected; ocean_graph_partition can't make partitions bigger than 746496 tasks because of a gpmetis error but this is a ludicrously large count.

@xylar xylar force-pushed the add_rrs6to18 branch 7 times, most recently from 7f4cfb3 to f43e6a1 Compare August 13, 2023 13:32
@xylar xylar mentioned this pull request Aug 14, 2023
Merged
3 tasks
@xylar xylar force-pushed the add_rrs6to18 branch 2 times, most recently from 0048e6c to d09e120 Compare August 14, 2023 10:52
@xylar
Add RRS6to18 and RRSwISC6to18 meshes
0e4c964 
These will become the E3SMv3 versions of the oRRS18to6v3 mesh.
@xylar
Add RRS6to18 test suites
319544a
@xylar
Add RRS6to18 to the docs
44666ae
@xylar
More cores for remapping topo
9828cf8
@xylar
Fix vertical grid name
6347aff
@xylar
Fix rayleight damping namelist option
642dd1c
@xylar
Add init-mode namelist and streams to RRS6to18
6568429 
These provide more stable parameters for computing the Haney-number
vertical coordinate. They also write out files in the "pnetcdf,cdf5"
format.
@xylar
Don't set PIO tasks and stride in init mode for RRS
3ccb3be 
This merge also sets the number of CPUs per task to 4 to
prevent out-of-memory errors.
@xylar
Make forward RRS outputs in pnetcdf,cdf5 format
9e2c306
@xylar
Set io_type for the RRS mesh in ssh adjustment
fa02c69
@xylar
Make RRS restart files pnetcdf,cdf5 format
44c513a
@xylar
Switch RRS meshes to using NETCDF4 format and netcdf4 engine
5d1f4f3
@xylar
Don't set PIO tasks and stride for RRS meshes
7bacc2a 
This doesn't seem to work well with files in NETCDF4 format.
@xylar
Turn on conversion to CDF5 during ssh adj. for RRS
8743160
@xylar
Update dynamic adjustment
8be28fe
@xylar
Skip ocean and sea-ice partitions if they exist
95ebf49 
This saves a lot of time when rerunning these steps.
@xylar
Update documentation
1f13dd7
@xylar
Allow meshes to set min and max cells in graph partitions
cccccb5 
Use these new config options to set the minimum to a higher number
for RRS meshes, since gpmetis is failing for very large task
counts.
@xylar xylar merged commit a4e8a38 into MPAS-Dev:main Aug 15, 2023
4 checks passed
@xylar xylar deleted the add_rrs6to18 branch August 15, 2023 07:58
@xylar xylar mentioned this pull request Aug 16, 2023
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@xylar xylar

Labels
enhancement New feature or request ocean
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@xylar