Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GFDL to main candidate branch (2022-05-16) #1570

Merged

Conversation

marshallward
Copy link
Collaborator

This patch includes several new features, as well as the usual crop of bugfixes
and aggressive refactoring.

  • Extensive support for the new HybGen/HYCOM vertical grids
  • Laplace diffusion of internal heights
  • Internal tide: Residual loss term from reflection/transmission
  • Ice shelf: surface mass flux (fluxes%shelf_sfc_mass_flux)

New parameters

  • VELOCITY_REMAPPING_SCHEME
  • PARTIAL_CELL_VELOCITY_REMAP
  • REMAP_VEL_MASK_BBL_THICK
  • REMAP_VEL_MASK_H_THIN
  • DATA_OVERRIDE_SHELF_FLUXES
  • THICKNESS_TOLERANCE
  • INTERNAL_TIDE_RESIDUAL_DRAG
  • GME_NUM_SMOOTHINGS
  • TRANS_FILE
  • KH_ETA_CONST
  • KH_ETA_VEL_SCALE
  • HybGen
    • USE_HYBGEN_UNMIX
    • HYBGEN_MIN_THICKNESS
    • HYBGEN_N_SIGMA
    • HYBGEN_COORD_FILE
    • HYBGEN_DEEP_DZ_PR0FILE
    • HYBGEN_SHALLOW_DZ_PR0FILE
    • HYBGEN_DEEP_DZ_VAR
    • HYBGEN_SHALLOW_DZ_VAR
    • HYBGEN_TGT_DENSITY_VAR
    • HYBGEN_ISOPYCNAL_DZ_MIN
    • HYBGEN_MIN_ISO_DEPTH
    • HYBGEN_RELAX_PERIOD
    • HYBGEN_BBL_THICKNESS
    • HYBGEN_REMAP_DENSITY_MATCH
    • HYBGEN_REMAP_MAX_ZSTAR_DILATE
  • Legacy reproducibility
    • ODA_2018_ANSWERS

New diagnostics

  • RK2 (split) dynamic core
    • deta_dt
  • Horizontal viscosity
    • d[uv]d[xy]_bt
  • Ice shelf
    • sfc_mass_flux
  • Internal tides
    • ITide_tot_residual_loss
    • trans
    • residual

Due to the removal of unused variables and whitespace cleanup, this PR is
expected to contain a very large number of changes in total.

Note that the large number of commits (~180) is due to re-introduction of
incorrecltly squashed commits from previous PRs. Hopefully this will get
everything back on track.

Features

Bugfixes and calculation improvements

Refactoring and testing

Contributors:

Philip Pegion and others added 30 commits January 28, 2022 10:58
remove conflict with dev/emc
further resolve conflict
put id_sppt_wts, etc back.
* remove white space and fix comment

* Update MOM_oda_incupd.F90

remove unused index bounds, and fix sum_h2 loop.

Co-authored-by: pjpegion <Philip.Pegion@noaa.gov>
Co-authored-by: Marshall Ward <marshall.ward@noaa.gov>
- Pointing to OBC wiki file from the lateral parameterizations doc.

- Using the MOM6 verbosity to control the time_interp verbosity.

- Making the check for negative water depths more informative.
  Revised the external/drifters code to bring it into closer alignment with the
MOM6 style guide at https://github.com/mom-ocean/MOM6/wiki/Code-style-guide.
This includes using standard syntax to document variable units, the addition of
'implicit none ; private', explicit public statements, and a licensing statement
to the files that were missing one, and modifications to follow the MOM6 2-point
indenting convention.  All answers are bitwise identical.
  Revised the external/ODA_hooks code to bring it into closer alignment with the
MOM6 style guide at https://github.com/mom-ocean/MOM6/wiki/Code-style-guide.
This includes using standard syntax to document variable units, the addition of
'implicit none ; private', and modifications to follow the MOM6 2-point
indenting convention.  All answers are bitwise identical.
  Corrected unbalanced indentation levels and impose MOM6-standard 2-point
indentation in the config_src/driver codes, including both the FMS and mct caps
and the solo_driver.  The nuopc_cap is excluded from this commit because of the
huge number (over 1000) of lines there that use irregular indentation.  All
answers are bitwise identical, and only white space changes are included.
  Corrected unbalanced indentation levels and impose MOM6-standard 2-point
indentation in the config_src/driver/nuopc_cap code.  There were over 1000 lines
that used non-standard or unbalanced indentation, so when reviewing this commit
the use of the "ignore whitespace" option might be very useful.  All answers are
bitwise identical, and only white space changes are included.
  Avoids using hard-coded dimensional tolerances in horizontal_regridding and
improves the documentation of the variables and their units in this file.
Without this change, calls to horiz_interp_and_extrap_tracer with a non-unity
conversion factor will exhibit unexpected changes in answers, and may have
wildly inappropriate amounts of smoothing of the interpolated values in arrays
that are initialized from z-space file.  However, the defaults are carefully
chosen to avoid changing answers in any cases that do not use rescaling of
tracer concentrations.  The specific changes include:

 - Add a new optional argument tr_iter_tol to the horiz_interp_and_extrap_tracer
   routines to set an appropriate dimensional threshold for when the post-fill
   smoothing of tracers should be considered to be adequate.

 - Make the dimensional crit argument to fill_miss_2d non-optional, and fill it
   with appropriate values in calls from the horiz_interp_and_extrap_tracer
   routines when tr_iter_tol is not present.  There should never be a hard-coded
   non-zero default for a dimensional variable!

 - Add a new optional scale argument to myStats

 - Eliminate the unused and unnecessary smooth logical argument to
   fill_miss_2d; the same functionality is obtained by setting the num_pass
   argument to 0.

 - Added to comments to document the units or unitlessness of the real
   variables in MOM_horizontal_regridding.F90.

 - Made minor changes to make the duplicated portions of the two
   horiz_interp_and_extrap_tracer similar.

By default, all answers are bitwise identical, but this corrects a subtle bug
with the use of the conversion factor argument in horiz_interp_and_extrap_tracer
calls.  There are new optional arguments to publicly visible routines.
@marshallward
Copy link
Collaborator Author

This time with the right branch (Thanks @jiandewang)

Apologies to everyone, those who approved will need to re-approve it.

@codecov
Copy link

codecov bot commented May 19, 2022

Codecov Report

Merging #1570 (0e8acd9) into main (bac8031) will decrease coverage by 0.15%.
The diff coverage is 18.96%.

@@            Coverage Diff             @@
##             main    #1570      +/-   ##
==========================================
- Coverage   28.93%   28.78%   -0.16%     
==========================================
  Files         242      249       +7     
  Lines       71606    72991    +1385     
==========================================
+ Hits        20720    21010     +290     
- Misses      50886    51981    +1095     
Impacted Files Coverage Δ
...g_src/drivers/solo_driver/MESO_surface_forcing.F90 0.00% <0.00%> (ø)
config_src/drivers/solo_driver/MOM_driver.F90 67.33% <0.00%> (-0.55%) ⬇️
...ig_src/drivers/solo_driver/MOM_surface_forcing.F90 21.05% <0.00%> (ø)
...g_src/external/GFDL_ocean_BGC/FMS_coupler_util.F90 0.00% <ø> (ø)
...fig_src/external/GFDL_ocean_BGC/generic_tracer.F90 0.00% <ø> (ø)
...c/external/GFDL_ocean_BGC/generic_tracer_utils.F90 0.00% <ø> (ø)
config_src/external/ODA_hooks/ocean_da_types.F90 0.00% <ø> (ø)
config_src/external/ODA_hooks/write_ocean_obs.F90 0.00% <ø> (ø)
config_src/external/drifters/MOM_particles.F90 0.00% <0.00%> (ø)
...nfig_src/external/drifters/MOM_particles_types.F90 0.00% <ø> (ø)
... and 255 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@gustavo-marques
Copy link
Collaborator

This candidate is changing answers for some of our tests. We will need some time to investigate this further.
cc'ing @alperaltuntas

The candidate merged to dev/ncar can be found here. There were a lot of conflicts that had to be solved manually.

@Hallberg-NOAA
Copy link
Collaborator

@gustavo-marques, if find that you are having trouble tracking down why the answers are changing, I may be able to help.

If you could point us to a git repository with any simple test cases (i.e., not requiring access to huge data volumes or specific access to NCAR machines) where the answers are changing, I might have some time this week to help you track them down. Moreover, I periodically carry out additional tests beyond those in the standard GFDL pipeline tests, especially before committing some of the more widespread code changes. I would be happy to consider other test cases to add to this list.

@jiandewang
Copy link
Collaborator

no answer change in UFS and N. Atlantic region setting. I am holding on approval at this moment as @gustavo-marques reported there is answer change in dev/ncar

@gustavo-marques
Copy link
Collaborator

Thanks, @Hallberg-NOAA. Unfortunately, we do not have any simple test cases that work with the MCT or NUOPC caps. However, we could put together an ocean-only global_ALE version with the NCAR physics enabled. This is not ideal but could help in identifying changes in answers related to the choice of physics.

@alperaltuntas
Copy link
Collaborator

@gustavo-marques, unless you have one ready, I have a tx0.66v1 standalone experiment in the following cheyenne directory. I haven't updated it in the last year or so, so you might want to update MOM_input and MOM_override to be able to reproduce the answer change.

/glade/work/altuntas/mom6.standalone.runs/cesm/c.t061.common

And here are the inputs:
/glade/work/altuntas/mom6.standalone.runs/cesm/INPUT/tx0.66v1

Copy link
Collaborator

@abozec abozec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My global ODA test case has passed without a problem but my regional test case has problems with the OBCs and I don't know why it is unhappy about it ... Need to dig a little bit.

@jiandewang
Copy link
Collaborator

@abozec : following is one of the netcdf file header used by OBC run, it works fine in this PR code
netcdf obc_ts_south {
dimensions:
nx_segment_002 = 2271 ;
ny_segment_002 = 1 ;
nz_segment_002_temp = 50 ;
nz_segment_002_salt = 50 ;
time = UNLIMITED ; // (6 currently)
variables:
int nx_segment_002(nx_segment_002) ;
nx_segment_002:cartesian_axis = "X" ;
int ny_segment_002(ny_segment_002) ;
ny_segment_002:cartesian_axis = "Y" ;
int nz_segment_002_temp(nz_segment_002_temp) ;
nz_segment_002_temp:cartesian_axis = "Z" ;
int nz_segment_002_salt(nz_segment_002_salt) ;
nz_segment_002_salt:cartesian_axis = "Z" ;
double time(time) ;
time:units = "days since 1900-12-31 00:00:00" ;
time:calendar = "gregorian" ;
double lon_segment_002(ny_segment_002, nx_segment_002) ;
double lat_segment_002(ny_segment_002, nx_segment_002) ;
double ilist_segment_002(ny_segment_002, nx_segment_002) ;
ilist_segment_002:orientation = 0 ;
double jlist_segment_002(ny_segment_002, nx_segment_002) ;
jlist_segment_002:orientation = 0 ;
double temp_segment_002(time, nz_segment_002_temp, ny_segment_002, nx_segment_002) ;
double vc_temp_segment_002(time, nz_segment_002_temp, ny_segment_002, nx_segment_002) ;
double dz_temp_segment_002(time, nz_segment_002_temp, ny_segment_002, nx_segment_002) ;
double salt_segment_002(time, nz_segment_002_salt, ny_segment_002, nx_segment_002) ;
double vc_salt_segment_002(time, nz_segment_002_salt, ny_segment_002, nx_segment_002) ;
double dz_salt_segment_002(time, nz_segment_002_salt, ny_segment_002, nx_segment_002) ;

@gustavo-marques
Copy link
Collaborator

I was able to track down where answers start to diverge. It's in the checksum call for Ah_h. The tests passed when LEITH_AH = False. Answers are changing with LEITH_AH = True because @Hallberg-NOAA correctly change Del2vort_h to be rotationally-invariant in NOAA-GFDL@ab4d226

In addition to changing the ocean state, this PR also changes Total age (or age stocks) because of the improvements introduced in NOAA-GFDL@1bf8220

Both changes are correct and should be retained. However, I will check with the rest of the NCAR team on how to proceed before approving this PR. I will report back here before Friday EOD.

@gustavo-marques gustavo-marques self-requested a review May 27, 2022 20:20
@gustavo-marques
Copy link
Collaborator

We approve this PR.

Copy link
Collaborator

@abozec abozec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COAPS approves

@marshallward
Copy link
Collaborator Author

@sanAkel Are you or Matt able to review this PR?

@sanAkel
Copy link
Collaborator

sanAkel commented Jun 1, 2022

@sanAkel Are you or Matt able to review this PR?

For some reason, I didn't get notified of this PR until now. Will get back in a few hours. Thanks!

@marshallward
Copy link
Collaborator Author

Everyone has approved this, and it seems like any answer changes are understood and there is a plan to resolve them, so I will now merge this.

Thanks as always to everyone. Time to update our branches!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.