Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Seaice prescribed mode #7

Closed
wants to merge 10 commits into from
Closed

Seaice prescribed mode #7

wants to merge 10 commits into from

Conversation

dazlich
Copy link
Contributor

@dazlich dazlich commented Feb 20, 2024

This PR implements the prescribed seaice cover mode in mpas-seaice. This was already an option in E3SM. The hooks in the nuopc cap had to be uncommented, and routines brought over from the E3SM source and translated to their cesm and nuopc equivalents.

With the PR, users will able to specify MPASSI%PRES in the F2000climo compset in place of CICE. In create_newcase use:
--compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV

The seaice cover produced by MPASSI and CICE is identical in test runs, the climates differ slightly.

This PR should permit us to bypass NVHPC issues with CICE.

Caveat: this compset will be susceptible to mpas framework issues just like the fully coupled model.

dazlich added 10 commits February 20, 2024 10:35
@dazlich
Implement prescribed mode
31ba351 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
90213d9 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
377a8cc 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
345e14d 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
74f4f50 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
616fb01 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
944db02 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
2718230 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
83fdf04 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich
Implement prescribed mode
bd5e5da 
    The prescribed seaice cover mode has been enabled in mpas-seaice. This is intended as an EarthWorks-specific alternative to CICE%PRES in the F2000climo compset. The necessary code was imported from the E3SM version of prescribed mode and the necessary translations made to the cesm/nuopc environment.
    
    To specify this compset, in create_newcase:
            --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV
@dazlich dazlich self-assigned this Feb 20, 2024
@gdicker1 gdicker1 mentioned this pull request Feb 21, 2024
Merged
gdicker1
gdicker1 requested changes Feb 27, 2024
View reviewed changes
Copy link
Contributor

@gdicker1 gdicker1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a general concern with the commit history as well as an issue when I try to run a case with --compset2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV .

  1. There's a few commits in this branch where typos are introduced and then fixed in the next commit. I suppose it depends on how we (as EarthWorksOrg developers) feel about the importance of the commit history in EarthWorksOrg repos. The branch could be rebased and re-pushed to address this.
  2. A case that I created with the following create_newcase command failed during the submission step due to file name substitutions. I may have gotten some step(s) wrong!

Create the compset:

/glade/work/gdicker/EarthWorks/EWRepo_PullRequests/2024Feb22_MPASSI7/EarthWorks/cime/scripts/create_newcase --case /glade/work/gdicker/EarthWorks/EWRepo_PullRequests/2024Feb22_MPASSI7/cases/2024Feb22_185324_EWMTst_MPASSI7-2_F2000climoEW.mpasa120.derecho.nvhpc --project ${PROJ_KEY} --compiler nvhpc --res mpasa120_mpasa120 --compset 2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV --driver nuopc --run-unsupported -i /glade/campaign/univ/ucsu0085/inputdata -- pecount 128

Then after configuring files (e.g. user_nl_cam), setting up the case, and building the case, the ./check_input_data script failed with this output:

Loading input file list: 'Buildconf/ctsm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
Loading input file list: 'Buildconf/mpassi.input_data_list'
Model mpassi missing file mesh = '/glade/campaign/univ/ucsu0085/inputdata/ice/mpas-cice/gx1v7/..nc'
Model mpassi missing file graph128 = '/glade/campaign/univ/ucsu0085/inputdata/ice/mpas-cice/gx1v7/.part.128'
Loading input file list: 'Buildconf/docn.input_data_list'
Loading input file list: 'Buildconf/cism.input_data_list'
Loading input file list: 'Buildconf/mosart.input_data_list'
Loading input file list: 'Buildconf/cam.input_data_list'
ERROR: check_input_data failed

@dazlich
Copy link
Contributor Author

dazlich commented Feb 27, 2024 via email

@gdicker1
Copy link
Contributor

Regarding point 2: you need '--res mpasa120_oQU120' to go along with using MPASSI

Of course! Re-running now.

Regarding point 1: that was before I fixed my 403 issues and I worked directly in the branch on GitHub - my bad. I’d be happy to rebase and repush - I’ll learn something in the process.

Let me know if you run into any issues. I think this should be fine to force push. Do whatever works best for you!

@dazlich
Copy link
Contributor Author

dazlich commented Feb 27, 2024 via email

@gdicker1
Copy link
Contributor

gdicker1 commented Mar 1, 2024

@dazlich I had to make one edit to the case1 but this worked ok for intel-oneapi compilers on Derecho for me. This isn't an approval yet due to the commit history.

Footnotes

  1. The 120km test case was created with a 75 second CAM timestep which isn't appropriate for MPAS with that mesh. This is because the NCPL_BASE_PERIOD was set to "hour". Running ./xmlchange NCPL_BASE_PERIOD="day" fixed it. ↩

@gdicker1
Copy link
Contributor

gdicker1 commented Mar 1, 2024

Just in case this is important, the end of my run from this comment had this output at the end of its atm.log.*. I expected a MPAS-A timing table instead of this text (but this may not be an error).

File on Derecho: "/glade/derecho/scratch/gdicker/2024Feb27_170459_EWMTst_MPASSI7-2_F2000climoEW.mpasa120.derecho.intel-oneapi/run/atm.log.3681619.desched1.240301-092539.gz"

Excerpt:

 ---------------------------------------
  QNEG3 from D_P_COUPLING:num_a1 Min. mixing ratio violated at  10220455 points. Worst =    0.1E-04
  QNEG3 from D_P_COUPLING:num_a2 Min. mixing ratio violated at  10417037 points. Worst =    0.1E-04
  QNEG3 from D_P_COUPLING:num_a3 Min. mixing ratio violated at   2162517 points. Worst =    0.1E-04
  QNEG3 from D_P_COUPLING:num_a4 Min. mixing ratio violated at   9867731 points. Worst =    0.1E-04
  QNEG3 from clubb:Q Min. mixing ratio violated at     40838 points. Worst =   -0.9E-04
  QNEG3 from nucleatei:num_a3 Min. mixing ratio violated at     75448 points. Worst =    0.2E-05
  QNEG3 from ndrop:num_a1 Min. mixing ratio violated at      6367 points. Worst =    0.2E-05
  QNEG3 from ndrop:num_a2 Min. mixing ratio violated at     75809 points. Worst =    0.6E-05
  QNEG3 from ndrop:num_a3 Min. mixing ratio violated at    258354 points. Worst =    0.8E-06
  QNEG3 from ndrop:num_a4 Min. mixing ratio violated at     32629 points. Worst =    0.7E-05
  QNEG3 from cldwat:num_a1 Min. mixing ratio violated at      6367 points. Worst =    0.2E-05
  QNEG3 from cldwat:num_a2 Min. mixing ratio violated at     75809 points. Worst =    0.6E-05
  QNEG3 from cldwat:num_a3 Min. mixing ratio violated at    320210 points. Worst =    0.8E-06
  QNEG3 from cldwat:num_a4 Min. mixing ratio violated at     32629 points. Worst =    0.7E-05
  QNEG3 from aero_model_wetdep:num_a1 Min. mixing ratio violated at  32376209 points. Worst =    0.1E-04
  QNEG3 from aero_model_wetdep:num_a2 Min. mixing ratio violated at  39968441 points. Worst =    0.1E-04
  QNEG3 from aero_model_wetdep:num_a3 Min. mixing ratio violated at   2003268 points. Worst =    0.8E-05
  QNEG3 from aero_model_wetdep:num_a4 Min. mixing ratio violated at  38446713 points. Worst =    0.1E-04
  QNEG3 from chemistry:H2O2 Min. mixing ratio violated at     10450 points. Worst =   -0.2E-11
  QNEG3 from chemistry:SO2 Min. mixing ratio violated at    147864 points. Worst =   -0.6E-11
  QNEG3 from chemistry:SOAG Min. mixing ratio violated at     31357 points. Worst =   -0.1E-10
  QNEG3 from vertical diffusion:H2O2 Min. mixing ratio violated at     11764 points. Worst =   -0.4E-11
  QNEG3 from vertical diffusion:num_a1 Min. mixing ratio violated at   2423009 points. Worst =    0.1E-04
  QNEG3 from vertical diffusion:num_a2 Min. mixing ratio violated at   3350811 points. Worst =    0.1E-04
  QNEG3 from vertical diffusion:num_a3 Min. mixing ratio violated at    153931 points. Worst =    0.1E-04
  QNEG3 from vertical diffusion:num_a4 Min. mixing ratio violated at   2771518 points. Worst =    0.1E-04
  QNEG3 from vertical diffusion:SO2 Min. mixing ratio violated at     25636 points. Worst =   -0.3E-10
  QNEG3 from vertical diffusion:SOAG Min. mixing ratio violated at     23443 points. Worst =   -0.2E-11
  QNEG3 from aero_model_drydep:num_a1 Min. mixing ratio violated at  32591972 points. Worst =    0.1E-04
  QNEG3 from aero_model_drydep:num_a2 Min. mixing ratio violated at  36886848 points. Worst =    0.9E-05
  QNEG3 from aero_model_drydep:num_a3 Min. mixing ratio violated at   9180373 points. Worst =    0.9E-05
  QNEG3 from aero_model_drydep:num_a4 Min. mixing ratio violated at  31727005 points. Worst =    0.1E-04
  QNEG3 from Gravity wave drag:num_a1 Min. mixing ratio violated at    472216 points. Worst =    0.1E-04
  QNEG3 from Gravity wave drag:num_a2 Min. mixing ratio violated at    766223 points. Worst =    0.1E-04
  QNEG3 from Gravity wave drag:num_a3 Min. mixing ratio violated at    331097 points. Worst =    0.1E-04
  QNEG3 from Gravity wave drag:num_a4 Min. mixing ratio violated at    558293 points. Worst =    0.1E-04
  nstep, te       49   0.26104838293015733E+10   0.26102354605471535E+10  -0.13764896816987702E-01   0.98436861464107264E+05   0.13755154299778766E+03

  -----------------------------------------
  Total log messages printed:
     Output messages =                  833
     Warning messages =                 887
     Error messages =                     0
     Critical error messages =            0
  -----------------------------------------
  Logging complete.  Closing file at 2024/03/01 09:28:29


 Number of completed timesteps:    48
 Time step     49 partially done to provide convectively adjusted and time filtered values for history tape.

  ******* END OF MODEL RUN *******
 (atm_comp_nuopc) -------------------------------------------------------------------------
 (atm_comp_nuopc) CAM: end of main integration loop
 (atm_comp_nuopc) -------------------------------------------------------------------------
# <EOF> Here

@dazlich
Copy link
Contributor Author

dazlich commented Mar 1, 2024 via email

@dazlich
Copy link
Contributor Author

dazlich commented Mar 1, 2024 via email

@dazlich
Copy link
Contributor Author

dazlich commented Mar 1, 2024 via email

@dazlich
Copy link
Contributor Author

dazlich commented Mar 1, 2024 via email

@gdicker1
Copy link
Contributor

gdicker1 commented Mar 1, 2024

Actually, F2000climo with seaice has similar output near the end:

Ah good, we're fine then. I just hadn't paid attention before! I just wanted to get this in text somewhere besides my scratch directory.

@dazlich
Copy link
Contributor Author

dazlich commented Mar 1, 2024 via email

@gdicker1
Copy link
Contributor

gdicker1 commented Mar 1, 2024

Here’s the answer (I think). Mpas_timer.F is part of the framework. The atm framework is build with -DMPAS_NATIVE_TIMERS, the ocean/seaice framework is not. It appears the atmosphere is calling the file in the ocean/seaice framework when both frameworks get built - both frameworks have the same external reference name. So ocean/seaice buildlib should include -DMPAS_NATIVE_TIMERS in the preprocessor flag list.

This problem should get resolved with the framework work in the mpas-* framework-ext-refs branches.

Ah that makes sense; and we're back to the multi-frameworks problem! Thanks for looking at that; I agree we should see this go away once EarthWorks clones have incorporated EWOrg/mpas-framework #6

@dazlich dazlich closed this Mar 8, 2024
@dazlich dazlich deleted the seaice-prescribed-mode branch March 8, 2024 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gdicker1 gdicker1 gdicker1 requested changes

@sherimickelson sherimickelson Awaiting requested review from sherimickelson

@supreethms1809 supreethms1809 Awaiting requested review from supreethms1809

Assignees

@dazlich dazlich

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dazlich @gdicker1