-
Notifications
You must be signed in to change notification settings - Fork 1
remap_restarts package
The remap_restarts package is a tool for remapping (or “regridding”) GEOS restart files to a different resolution and/or a different version of the associated model boundary conditions.
The main script (remap_restarts.py) and the associated scripts and executables can be found in the install/bin directory of a GEOSgcm build. See the GEOSgcm wiki for help with building GEOSgcm.
To run remap_restarts.py, load the necessary modules (incl. python3) as follows:
source g5_modules.sh [bash]
source g5_modules [csh]
remap_restarts.py can be run in four different ways:
- Use the interactive questionnaire (recommended):
./remap_restarts.py
The questionnaire concludes with the option to submit the remapping job. It also creates a yaml configuration file (remap_params.yaml) and a matching command line argument string (remap_restarts.CMD), which can be edited manually and used in the next two run options.
- Use an existing yaml config file*:
./remap_restarts.py config_file -c my_config.yaml
- Use command line arguments*:
./remap_restarts.py command_line -ymdh 2004041421 ...
- For GEOSldas: Remap land (catch[cn]) restart only; global domain only; ens0000 only:
./remap_restarts.py land_only
*NOTE: The yaml and command-line interfaces may not be backward compatible across releases. If existing yaml files or command-line strings do not work, use the questionnaire option to generate updated versions.
./remap_restarts.py -h
./remap_restarts.py config_file -h
./remap_restarts.py command_line -h
To create the configuration parameters, the interactive questionnaire asks a set of questions. Brief explanatory comments, pull-down menus, lists of possible answers, and/or defaults are provided as appropriate to assist the user.
To avoid unnecessary questions, the questions asked depend on the preceding answers. For example, the script asks about date/time only if the input restart directory contains restarts for multiple date/time subdirectories. Moreover, the script examines fvcore_internal_rst (if present) and extracts basic information about the horizontal resolution and number of levels.
Subsets of restarts can be remapped separately by running:
-
remap_upper.py(for atmospheric model restarts) -
remap_catchANDcn.py(for Catchment[CN] model restarts), -
remap_lake_landice_saltwater.py(for lake, landice, and [data] ocean restarts), and/or -
remap_analysis.py(for atmospheric analysis restarts).
Each of these scripts reads its configuration inputs from remap_params.yaml. See the Appendix below for a list of restart files in each subset.
remap_restarts.py must know the model boundary conditions (BCs) associated with the input and new restart files. The script uses BCs in the revised layout (directory structure and file naming convention) introduced in late 2023. Recent and commonly used older versions of the BCs are provided in the revised layout in the following locations:
NCCS: /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles/
NAS: /nobackup/gmao_SIteam/ModelData/bcs_shared/fvInput/ExtData/esm/tiles/
For more information on BCs, see GEOSgcm_GridComp Wiki.
remap_restarts.py replaces the (deprecated) legacy perl script regrid.pl. Results are binary identical when remap_restarts.py and regrid.pl are from the same build.
Note that regrid.pl uses BCs in the “legacy” layout (directory structure and file naming convention). If the BCs of interest are not available in the revised layout (see above), contact the GEOSgcm_GridComp gatekeepers for assistance.
The ./tests sub-directory includes a script and several yaml configuration files that serve as examples and for use in 0-diff testing. The tests are set up for NCCS only. An overview of the tests is in the file test_remap_cases.yaml. To run the tests, source g5_modules (see above) and execute
./test_remap_restarts.py
The test script automatically compares the output against the archived baseline results. Individual test cases without the comparison can be run as described above, for example:
./remap_restarts.py config_file -c c180Toc360.yaml
(Note: The string '$NOBACKUP' in the sample file c180Toc360.yaml should be replaced manually with the user's desired output path.)
remap_restarts.py is a python wrapper that uses helper scripts (named remap_*.py) and the following Fortran executables from a build of GEOSgcm:
Atmospheric model:
fvrst.x interp_restarts.x interp_restarts_bin.x
Catchment[CN] land model:
mk_catchANDcnRestarts.x
Lake, landice, and (data) ocean:
SaltIntSplitter.x mk_LakeLandiceSaltRestarts.x
Atmospheric analysis:
dyn2dyn.x mkdrstdate.x
Atmospheric model:
fvcore_internal_rst moist_internal_rst agcm_import_rst
agcm_internal_rst carma_internal_rst achem_internal_rst
geoschemchem_internal_rst gmichem_internal_rst gocart_internal_rst
hemco_internal_rst mam_internal_rst matrix_internal_rst
stratchem_internal_rst pchem_internal_rst tr_internal_rst
ss_internal_rst du_internal_rst cabr_internal_rst
cabc_internal_rst caoc_internal_rst ni_internal_rst
su_internal_rst
Catchment[CN] land model:
catch_internal_rst catchcnclm[XX]_internal_rst
Lake, landice, and (data) ocean:
lake_internal_rst landice_internal_rst openwater_internal_rst
seaice_internal_rst saltwater_internal_rst
Atmospheric analysis:
[bkg] lcv
remap_restarts.py remaps whichever of the above restart files are present in the input restart directory.
Like regrid.pl, remap_restarts.py does not properly remap the [bkg.eta] atmospheric analysis files to the new ADAS grid. Rather, the remapping provides [bkg.eta] files on some grid. Because the ADAS can ingest [bkg.eta] files on any grid, the usual practice is to run one throw-away ADAS cycle to get the correct [bkg.eta] files.
When the MERRA-2 default configuration is used (first option in Step 2), all the necessary data files are made available to the users and the remapping of restart files is done seamlessly. Running:
./remap_restarts.py
presents an interface where users are prompted to answer questions about the the input restarts and output restarts. Most users will only need to focus on:
- the restart date (YYYYMMDDHH)
- The output directory for the new restart files
- Horizontal resolution
- Vertical resolution
- Type of ocean used (data, MOM5, MOM6)
In most cases, beyond the requested horizontal resolution, the defaults presented by the tool are a good place to start. Here are the questions that are asked:
? Remap archived MERRA-2 restarts? (NCCS/Discover only; elsewhere, select 'N' and complete full config; requires nc4 restarts.
? Enter restart date: (Must be 10 digits: yyyymmddhh; hour = 03, 09, 15, or 21 [z].)
? Enter output directory for new restarts:
? Remap to a stretched cubed-sphere grid? (y/N)
? Enter atmospheric grid for new restarts:
C12 C180 C1440
C24 C360 C2880
C48 C720 C5760
C90 C1000
? Select ocean model for new restarts:
(Use arrow keys)
» data
MOM5
MOM6
? Select data ocean grid/resolution for new restarts:
(Use arrow keys)
360x180 (Reynolds)
1440x720 (MERRA-2)
2880x1440 (OSTIA)
» CS (same as atmosphere OSTIA cubed-sphere grid)
? Enter number of atmospheric levels for new restarts: (71 72 91 127 132 137 144 181)
72
? Select boundary conditions (BCs) version for new restarts:
BCs version | ADAS tags | GCM tags typically used with BCs version
-----------------|----------------------|-----------------------------------------
GM4: Ganymed-4_0 | 5_12_2 ... 5_16_5 | Ganymed-4_0 ... Heracles-5_4_p3
ICA: Icarus | 5_17_0 ... 5_24_0_p1 | Icarus, Jason ... 10.18
NL3: Icarus-NLv3 | 5_25_1 ... present | Icarus_NL, 10.19 ... present
----------------------------------------------------------------------------------
Other: Additional choices used in model or DAS development.
(Use arrow keys)
» NL3
ICA
GM4
Other
? Select BCs base directory for new restarts:
(Use arrow keys)
» NCCS/Discover : /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles
NAS : /nobackup/gmao_SIteam/ModelData/bcs_shared/fvInput/ExtData/esm/tiles
Custom
? Remap upper air restarts? (Y/n)
? Remap agcm_import_rst (a.k.a. IAU) file needed for REPLAY runs?
(NOTE: Preferred method is to regenerate IAU file,
but IF requested, remapping will be performed.) (y/N)
? Remap surface restarts? (Y/n)
? Remap bkg files? (Required by ADAS but not mapped onto ADAS grid; run one ADAS cycle to spin up.) (y/N)
? Write lcv file? (Required by ADAS.) (y/N)
? Enter value of WEMIN (min. snow water equivalent parameter) to be used in simulation with new restarts.
26
? Enter value of zoom parameter for surface restarts [1-8]? (Search radius, smaller value means larger radius.)
2
? Enter experiment ID for new restarts: (Added as prefix to new restart file names; can leave blank.)
? Add labels for BCs version and atm/ocean resolutions to restart file names? (y/N)
? SLURM quality-of-service (qos)? (If on NCCS and atm resolution is c1440 or higher, enter allnccs.) debug
? SLURM account?
The dialogue process continues with a summary of the settings (displayed on the screen) and the question:
? Above is the YAML config file, would you like to continue? (Y/n)
Finally, users are asked if they want submit the job:
? Submit the jobs now ? (Y/n)
If the answer is yes, the remap process automatically begins.
When the MERRA-2 option is selected, it is important to note that only restart files at 03z, 09z, 15z and 21z can be created. This means that the 10-digit date/time should end with the numbers 03, 09, 15, 21 (hh).
The remap_restarts.CMD below was generated by answering the above questionnaire:
remap_restarts.py command_line -merra2 -ymdh 2015050521 -grout C90 -levsout 72 -newid TEST -ocnmdlin data -ocnmdlout data -oceanin 1440x720 -oceanout 1440x720 -bcvin GM4 -bcvout ICA -rst_dir FULL_PATH_TO_WHERE_TO_STORE_NEW_FILES/merra2_tmp_2015050521 -out_dir FULL_PATH_TO_WHERE_TO_STORE_NEW_FILES -in_bc_base /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles -out_bc_base /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles -catch_model catch -zoom 2 -in_wemin 26 -out_wemin 26 -nolcv -rs 3 -account k3002 -qos debug -partition compute