Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swath grid for MAPL_HistoryGridComp #2458

Merged
merged 48 commits into from
Dec 1, 2023
Merged

Conversation

metdyn
Copy link
Contributor

@metdyn metdyn commented Nov 29, 2023

Description

Creating Swath grid from observation files such as modis_MOD04_L2 data. In history grid component, use ESMF to regrid from cubed sphere grid to this logically rectangular grid.

Coauthor with Tom, Ben and Arlindo. Epoch time frame is introduced to select a chunk of observation based on user specification. Code can regrid and accumulate at each model run step, then use griddedIO to stage the data to the oserver.

Swath grid is regenerated for each Epoch time (e.g., 6 hour).

Code allows for observation file name template with wild-character (e.g., modis_*.nc4) and can skip missing observation file.

I should add Tom also solved the issue with NAG netCDF (lons/lats output problem in this PR): #2380

Example input:

  SwathGrid.GRID_TYPE: Swath
  SwathGrid.GRID_FILE: 'flk_modis_MOD04_2017_090/MOD04_L2.A%y4%D3.%h2%n2.*.h5'
  SwathGrid.NX:  1
  SwathGrid.NY:  6
  SwathGrid.LM:  2

  SwathGrid.index_name_lon: 'Cell_Across_Swath:mod04'
  SwathGrid.index_name_lat: 'Cell_Along_Swath:mod04'
  SwathGrid.var_name_lon:   'mod04/Geolocation Fields/Longitude'
  SwathGrid.var_name_lat:   'mod04/Geolocation Fields/Latitude'
  SwathGrid.var_name_time:  'mod04/Data Fields/Scan_Start_Time'
  SwathGrid.tunit:          'seconds since 1993-01-01 00:00:00'

  SwathGrid.obs_file_begin: '2017-03-31T02:00:00'
  SwathGrid.obs_file_end:   '2017-03-31T23:00:00'
  SwathGrid.obs_file_interval:   '000000 000500'     # yymmdd  hhmmss
  SwathGrid.Epoch:           010000                  #
  SwathGrid.Epoch_init:      '2017-03-31T00:00:00'   # redundant, default: obs_file_begin

Output:
var2d and var3d in
modis_swath.20170331_0300.nc4

Related Issue

Motivation and Context

How Has This Been Tested?

  1. Checked output on Mac with NAG and GCC.

var2d = 100*cos(lons)*cos(lats)

swath_6hr
  1. Run gcm_run.j on discover with control (GEOSgcm.git main branch) vs gcm with this MAPL PR for C12.
    Checked the history output with nccmp and ncdump results.
    They are identical. This means the modifications to GriddedIO.F90 and other codes do not introduce output difference in history.

diff d.c12_control_Nov1.geosgcm_prog.20000415_1800z d.c12_swath_PR2_test.geosgcm_prog.20000415_1800z.

> netcdf c12_control_Nov1.geosgcm_prog.20000415_1800z
< netcdf c12_swath_PR2_test.geosgcm_prog.20000415_1800z
204,205c204,205
> :Source = "GEOSgcm-v11.3.2 experiment_id: c12_control_Nov1" ;
< :Title = "a" ;

> :Source = "GEOSgcm-v11.3.2 experiment_id: c12_swath_PR2_test" ;
> :Title = "test_SWATH_PR2_with_C12" ;

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Trivial change (affects only documentation or cleanup)

Checklist:

  • I have tested this change with a run of GEOSgcm (if non-trivial)
  • I have added one of the required labels (0 diff, 0 diff trivial, 0 diff structural, non 0-diff)
  • I have updated the CHANGELOG.md accordingly following the style of Keep a Changelog

Modifications made to GriddedIO.F90, MAPL_AbstractRegridder.F90, and MAPL_EsmfRegridder.F90.
nf90_getvar from  mod04/Data Fields/Scan_Start_Time
netCDF error: NetCDF: Start+count exceeds dimension bound
- I seem to find some config problems related to the position history.rc entries are laid out in file.
- File names needs to be in template,
- Stick to every 5 minute inteval for template
 - DOY
 - cmd=bash -c 'ls /Users/yyu11/ModelData/earthData/flk_modis_MOD04_2017_090/MOD04_L2.A2017090.0010.*.hdf' &> zzz_MAPL
Fatal Error: /Users/yyu11/repos/br_mapl_traj_all_compilers/MAPL/base/FileMetadataUtilities.F90, line 5: Cannot find module MAPL_GRIDMANAGERMOD
             detected at MAPL_GRIDMANAGERMOD@<end-of-statement>
Next merge develop to see if this can solve the problem.
@metdyn metdyn requested review from a team as code owners November 29, 2023 22:10
base/MAPL_ObsUtil.F90 Outdated Show resolved Hide resolved
base/MAPL_ObsUtil.F90 Outdated Show resolved Hide resolved
Copy link
Collaborator

@tclune tclune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot here so I mostly did spot checks. There are 1-2 items that should be addressed and maybe 1-2 more that could be, but are not as crucial.

tclune
tclune previously approved these changes Nov 30, 2023
@mathomp4 mathomp4 merged commit 960ffc8 into develop Dec 1, 2023
25 checks passed
@mathomp4 mathomp4 deleted the feature/ygyu/PR_swath_2_epoch branch December 1, 2023 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants