Enable use of create_test workflow #45

gdicker1 · 2024-05-10T18:25:41Z

Add definitions of compsets and tests so that typical EarthWorks cases can be run as CIME tests instead.

Testing instructions (in this thread): #45 (comment)

This begins steps to use the workflow in CIME. A section now exists for EarthWorks specific tests and such compsets should include the '-EW' postfix.

This first test is a 5 day (default length) "smoke-test", using 1 Derecho node, nuopc coupled, on the 480km grid.

gdicker1 · 2024-05-10T18:28:00Z

gdicker1 · 2024-05-10T18:30:12Z

Some info about tests can be found in the CIME Documentation: https://esmci.github.io/cime/versions/master/html/users_guide/testing.html#testname-syntax

These tests ensure that the other EarthWorks compsets aren't affected by a PR including FHS94, FKESSLER, QPC6, F2000climo at 120km. A few compilers are checked for compatability. Checking smoke tests of these runs should help catch many reversions or issues.

This makes use of some syntatic sugar to make sure the FHS94 and FKESSLER don't try to use 'vcoords' files for ncdata. Asking for 'mpasa120z32_mpasa120' sets 'nlev' to 32 (instead of 30 which is the default).

All 120km runs came in under these new limits. This change should help ew-pr tests get through the Derecho queue more quickly.

This change should reduce the overall cost of running the ew-pr test category by doing wide testing for the least and most complicated compsets. The NVHPC compilers are used for every compset since EarthWorks depends on it and NVHPC testing of these compsets isn't commonly done.

Add EarthWorks equivalents of the B2000 and F2000climo compsets that use MPAS-O and MPAS-SI instead.

gdicker1 · 2024-05-16T22:57:21Z

cime_config/testmods_dirs/earthworks/fullycoupled/user_nl_mpaso

@@ -0,0 +1,2 @@
+
+config_cvmix_kpp_use_theory_wave = .true.


@dazlich should this variable be set true only under certain conditions (like FullyCoupled at 120km), or do you think this should be true for any EW mpas-o compset? I can create a PR for the second case easily.

dazlich · 2024-05-17T15:06:08Z

I’ve got an outside opinion that it should be generally true.

gdicker1 · 2024-05-17T18:25:14Z

The various config_dt and ATM_NCPL values are set according to this table which comes from emails with Don Dazlich (and some input/context from Brian Dobbins).

Resolution	ocn & ice dt (s)	mpasa dt (s)	ATM_NCPL (per day)	Radiation Long & Short Wave
120km	1800.0	600	48	1hr (iradlw,iradsw=-1)
60km	900	300	96	1hr (iradlw,iradsw=-1)
30km	450	150	192	1hr (iradlw,iradsw=-1)
15km	240	80	360	40 min (iradlw,iradsw=10)
7.5km	120	40	720	20min (iradlw,iradsw=10
3.75km	60	20	1440	10min (iradlw,iradsw=10

gdicker1 · 2024-05-17T18:52:42Z

To test these changes:

Clone this branch and run manage_externals
Run create_test and point to the testlist_earthworks.xml file. ALSO ensure that you point to the EarthWorks input data and supply a suitable (EarthWorks) project/account key. Choose the test category from ew-pr, ew-ver, and ew-rel.
1. NOTE: by default the builds will try to occur on the login node, running the command using the qcmd utility on Derecho is recommended to not violate fair use policies.
2. NOTE: ew-ver and ew-rel can consume a lot of core hours. For now, please add --no-build to the create_test to ensure these tests only run their setup steps for now
Once the cases have been submitted (or finished their appropriate step) you can check the status by going into the directory the tests were created in and running the cs.status.* script (the final part of the script name varies). You can grep for the "OVERALL" lines or for "FAIL" to limit output.

Example commands to run the ew-pr on Derecho and put the tests in <your_scratch>/ew_pr_tests

git clone -b feature/ew-cime-testing https://github.com/gdicker1/EarthWorks
cd EarthWorks && ./manage_externals/checkout_externals
qcmd -A <ew_proj_key> -- ./cime/scripts/create_test--xml-testlist cime_config/testlist_earthworks.xml \
 -i /glade/campaign/univ/ucsu0085/inputdata/ --project <ew_proj_key> \
 --test-root ${SCRATCH}/ew_pr_tests  --xml-category ew-pr
# This will block while builds and other steps finish
cd ${SCRATCH}/ew_pr_tests
./cs.status.*

These are initial tests that serve as a starting point. More should be added in the future. The 'ew-ver' test added should work for what is described on the CESM website for held-suarez. This is a 1200 day forward run of the FHS94 compset with the 120km 32 level MPAS-A grid. See "https://www.cesm.ucar.edu/models/simple/held-suarez" The 'ew-rel' tests are starting with FullyCoupled-EW runs on 120, 60, 30, and 15km grids. These tests are "exact restart" tests that run forward for 11 days, write a restart on day 6, and then restarts with half the number of tasks to reach the same 11 day mark.

dazlich

@gdicker1 - I've tested this with
./create_test --xml-testlist ../../cime_config/testlist_earthworks.xml --xml-compiler intel --xml-category ew-pr --project UCSU0085

This creates two cases, FHS94 and FULLYCOUPLED. Both build through compilation. FHS94 also runs, FULLYCOUPLED fails trying to find seaice/ocean data that doesn't exist in the campaign inputdata directory.

gdicker1 · 2024-05-20T21:31:05Z

FULLYCOUPLED fails trying to find seaice/ocean data that doesn't exist in the campaign inputdata directory.

Judging by your command and comment I assume you mean the CESM input data on Campaign? Since I haven't figured out how to add EW as a model within CIME, we will keep getting that (CESM) location by default.

@dazlich, could you try adding -i /glade/campaign/univ/ucsu0085/inputdata and try again?

(Or go into the specific test, run ./xmlchange DIN_LOC_ROOT='/glade/campaign/univ/ucsu0085/inputdata' and then run ./test.submit)

dazlich · 2024-05-20T21:39:51Z

@gdicker1 Doh, of course - I am still getting up to speed on 'create_test'

dazlich · 2024-05-20T21:45:23Z

B2000-EW

I would have liked to called this CHAOS2000, CHAOS standing for Coupled Hexagonal Atmosphere Ocean and Seaice since that's what the coupled mpas components on a quasi-uniform grid essentially is. ;-)

dazlich

Ok, the test I tried before works now adding the --inputdata option.

gdicker1 · 2024-05-20T22:01:12Z

B2000-EW

I would have liked to called this CHAOS2000, CHAOS standing for Coupled Hexagonal Atmosphere Ocean and Seaice since that's what the coupled mpas components on a quasi-uniform grid essentially is. ;-)

Huh, I thought B2000 was already a compset (could still be, for all I know). I think CHAOS2000 is such a better name! If this is a vote, mine goes to CHAOS2000 instead :D

dazlich · 2024-05-20T22:08:29Z

Actually, I don’t see B2000 when I do a query_config, but other B-cases have CICE and either POP or MOM. I choose the B2000-EW name since I set the bcs like GHG to cyclic 2000. On May 20, 2024, at 4:01 PM, G. Dylan Dickerson ***@***.***> wrote: ** Caution: EXTERNAL Sender ** B2000-EW I would have liked to called this CHAOS2000, CHAOS standing for Coupled Hexagonal Atmosphere Ocean and Seaice since that's what the coupled mpas components on a quasi-uniform grid essentially is. ;-) Huh, I thought B2000 was already a compset (could still be, for all I know). I think CHAOS2000 is such a better name! If this is a vote, mine goes to CHAOS2000 instead :D — Reply to this email directly, view it on GitHub<#45 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADS3XWHEHB6DCHGW44A7WYLZDJXD3AVCNFSM6AAAAABHRB5AB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGI4DMMZXGI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

dazlich

Just to add, I ran the entire ew-ver and ew-rel suites with the --no-build option and they all pass.

My intel/fullycoupled ew-pr ran out of time.

gdicker1 · 2024-05-20T23:33:24Z

Just to add, I ran the entire ew-ver and ew-rel suites with the --no-build option and they all pass.

My intel/fullycoupled ew-pr ran out of time.

@dazlich, thank you. I'll update the FullyCoupled category, it'll only hurt the queue-length for these tests. When you have a sec, could you just make sure the sub-PRs make sense (i.e. pass visual inspection)?

dazlich · 2024-05-21T03:42:21Z

@gdicker1 I've had a look at the sub-PRs and they all look good to me!

areanddee · 2024-05-21T13:21:29Z

I think B2000 is a compset, rings a bell. I vote for CHAOS2000!

…

On Mon, May 20, 2024 at 4:01 PM G. Dylan Dickerson ***@***.***> wrote: B2000-EW I would have liked to called this CHAOS2000, CHAOS standing for Coupled Hexagonal Atmosphere Ocean and Seaice since that's what the coupled mpas components on a quasi-uniform grid essentially is. ;-) Huh, I thought B2000 was already a compset (could still be, for all I know). I think CHAOS2000 is such a better name! If this is a vote, mine goes to CHAOS2000 instead :D — Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZ2GWBNFBDYN7E76JXTSX43ZDJXD3AVCNFSM6AAAAABHRB5AB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGI4DMMZXGI> . You are receiving this because your review was requested.Message ID: ***@***.***>

Remove the hyphen characters from the names (because test names will be confusing enough, already). Also, since there is no B2000 compset, rename to the Coupled, Hexagonal, Atmosphere, Ocean, and Seaice 2000 (CHAOS2000) compset.

dazlich · 2024-05-21T15:02:52Z

Big Boss rejected that name when I proposed it for our CSU model 20 years ago On May 21, 2024, at 7:21 AM, areanddee ***@***.***> wrote: I think B2000 is a compset, rings a bell. I vote for CHAOS2000!

Enable use of create_test workflow. Add definitions of compsets and tests so that typical EarthWorks cases can be run as CIME tests instead. Also include changes to externals that helps auto-select timesteps and coupling intervals based on resolution.

gdicker1 added 3 commits May 9, 2024 09:40

Add 'FullyCoupled-EW' entry to config_compsets.xml

b56dd0e

This begins steps to use the workflow in CIME. A section now exists for EarthWorks specific tests and such compsets should include the '-EW' postfix.

Make first attempt at adding test for FullyCoupled compset

c275d1f

This first test is a 5 day (default length) "smoke-test", using 1 Derecho node, nuopc coupled, on the 480km grid.

Change grid to 120km for SMS FullyCoupled test

6cfd51c

gdicker1 self-assigned this May 10, 2024

gdicker1 added enhancement New feature or request help wanted Extra attention is needed labels May 10, 2024

gdicker1 mentioned this pull request May 16, 2024

Add config_dt for 60km to 3.75km oQU grids EarthWorksOrg/mpas-seaice#11

Merged

gdicker1 added 6 commits May 16, 2024 16:17

Incorporate changes from Externals for cime testing

c38de6a

Add z32 to grid for FHS94 and FKESSLER tests

a6b2901

This makes use of some syntatic sugar to make sure the FHS94 and FKESSLER don't try to use 'vcoords' files for ncdata. Asking for 'mpasa120z32_mpasa120' sets 'nlev' to 32 (instead of 30 which is the default).

Reduce wallclock requested for ew-pr tests

8523938

All 120km runs came in under these new limits. This change should help ew-pr tests get through the Derecho queue more quickly.

Update externals with dt and cpl interval changes

4627564

gdicker1 force-pushed the feature/ew-cime-testing branch from 9e53029 to 4627564 Compare May 16, 2024 22:24

Add aliases for other custom EW compsets

6efecc4

Add EarthWorks equivalents of the B2000 and F2000climo compsets that use MPAS-O and MPAS-SI instead.

gdicker1 commented May 16, 2024

View reviewed changes

This was referenced May 17, 2024

Change config_dt for to match new NCPL intervals EarthWorksOrg/mpas-ocean#8

Merged

Set coupling intervals for MPAS compsets EarthWorksOrg/CMEPS#6

Merged

Update EW mpas-a time steps and set radiation intervals EarthWorksOrg/CAM#17

Merged

gdicker1 changed the title ~~(WIP) Enable use of create_test workflow~~ Enable use of create_test workflow May 17, 2024

gdicker1 added 2 commits May 17, 2024 13:51

Remove FullyCoupled-EW testmods, already in mpas-ocean

f8589b9

gdicker1 marked this pull request as ready for review May 20, 2024 18:37

gdicker1 requested review from sherimickelson and dazlich May 20, 2024 18:37

gdicker1 requested a review from areanddee May 20, 2024 18:37

dazlich reviewed May 20, 2024

View reviewed changes

dazlich approved these changes May 20, 2024

View reviewed changes

Increase walltime req for FullyCoupled test

f42ef24

gdicker1 removed the request for review from sherimickelson May 21, 2024 00:06

Rename EW specific compsets

7e8bd39

Remove the hyphen characters from the names (because test names will be confusing enough, already). Also, since there is no B2000 compset, rename to the Coupled, Hexagonal, Atmosphere, Ocean, and Seaice 2000 (CHAOS2000) compset.

gdicker1 force-pushed the feature/ew-cime-testing branch from 674173a to 7e8bd39 Compare May 21, 2024 14:46

Update Externals.cfg with new tags from sub-PRs

9e4437d

gdicker1 merged commit f0f2c17 into EarthWorksOrg:develop May 21, 2024

gdicker1 added a commit that referenced this pull request May 21, 2024

Merge branch 'feature/ew-cime-testing' into 'develop' (PR #45)

7bd2f8f

gdicker1 mentioned this pull request May 30, 2024

Add EW compsets with MPAS-ocean/seaice #48

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable use of create_test workflow #45

Enable use of create_test workflow #45

gdicker1 commented May 10, 2024 •

edited

Loading

gdicker1 commented May 10, 2024 •

edited

Loading

gdicker1 commented May 10, 2024

gdicker1 May 16, 2024

dazlich commented May 17, 2024 via email •

edited by gdicker1

Loading

gdicker1 commented May 17, 2024

gdicker1 commented May 17, 2024 •

edited

Loading

dazlich left a comment

gdicker1 commented May 20, 2024 •

edited

Loading

dazlich commented May 20, 2024

dazlich commented May 20, 2024

dazlich left a comment

gdicker1 commented May 20, 2024

dazlich commented May 20, 2024 via email

dazlich left a comment

gdicker1 commented May 20, 2024

dazlich commented May 21, 2024

areanddee commented May 21, 2024 via email

dazlich commented May 21, 2024 via email

Enable use of create_test workflow #45

Enable use of create_test workflow #45

Conversation

gdicker1 commented May 10, 2024 • edited Loading

gdicker1 commented May 10, 2024 • edited Loading

gdicker1 commented May 10, 2024

gdicker1 May 16, 2024

Choose a reason for hiding this comment

dazlich commented May 17, 2024 via email • edited by gdicker1 Loading

gdicker1 commented May 17, 2024

gdicker1 commented May 17, 2024 • edited Loading

dazlich left a comment

Choose a reason for hiding this comment

gdicker1 commented May 20, 2024 • edited Loading

dazlich commented May 20, 2024

dazlich commented May 20, 2024

dazlich left a comment

Choose a reason for hiding this comment

gdicker1 commented May 20, 2024

dazlich commented May 20, 2024 via email

dazlich left a comment

Choose a reason for hiding this comment

gdicker1 commented May 20, 2024

dazlich commented May 21, 2024

areanddee commented May 21, 2024 via email

dazlich commented May 21, 2024 via email

gdicker1 commented May 10, 2024 •

edited

Loading

gdicker1 commented May 10, 2024 •

edited

Loading

dazlich commented May 17, 2024 via email •

edited by gdicker1

Loading

gdicker1 commented May 17, 2024 •

edited

Loading

gdicker1 commented May 20, 2024 •

edited

Loading