Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Homme V2 integration updates #3406

Merged
merged 9 commits into from
Jan 23, 2020
Merged

Conversation

ambrad
Copy link
Member

@ambrad ambrad commented Jan 18, 2020

  • Call prim_finalize from EAM, and call Kokkos::finalize when using SL transport (indirectly) from prim_finalize.
  • Final tool for physgrid topography file.
  • Fix reading of GLL PHIS_d when reading the new physgrid topo file format.

[BFB]

ambrad and others added 9 commits January 8, 2020 20:11
In the new physgrid topography toolchain, we need to run this utility twice. On
ne30pg2 (for example), it would previously issue a message and abort. However,
with the stop statement removed, the resulting topo dataset appears fine. So
comment it out.

Abort immediately if --smooth-topography is an invalid file.

Reduce the number of progress messages.
There are two new topography tools to support physgrid.

The first is a simple converter. The input is an old GLL topography file. The
output is a topography file in the new GLL-physgrid format. SGH, SGH30,
LANDFRAC, and LANDM_COSLAT are not quite as good as with second tool, but this
converter works in one step:

$ cat input.nl
&ctl_nl
ne = 30
/
&vert_nl
/
&analysis_nl
tool = 'topo_convert'
infilenames = 'USGS-gtopo30_ne30np4_16xdel2-PFC-consistentSGH.nc', 'USGS-gtopo30_ne30np4pg2_16xdel2-PFC-consistentSGH_converted'
/

mpirun -np 8 homme_tool < input.nl

The second tool provides the best-quality GLL-physgrid file. It has five steps:

1. TempestRemap:

${tempest_root}/bin/GenerateCSMesh --alt --res 30 --file topo2/ne30.g
${tempest_root}/bin/GenerateVolumetricMesh --in topo2/ne30.g --out topo2/ne30pg2.g --np 2 --uniform
${tempest_root}/bin/GenerateVolumetricMesh --in topo2/ne30.g --out topo2/ne30pg4.g --np 4 --uniform
${tempest_root}/bin/ConvertExodusToSCRIP --in topo2/ne30pg4.g --out topo2/ne30pg4_scrip.nc

2. cube_to_target, run 1:

${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \
  --target-grid topo2/ne30pg4_scrip.nc \
  --input-topography topo2/USGS-topo-cube3000.nc \
  --output-topography topo2/ne30pg4_c2t_topo.nc

3. homme_tool:

$ cat input.nl
&ctl_nl
ne = 30
smooth_phis_numcycle = 16
smooth_phis_nudt = 28e7
hypervis_scaling = 0
hypervis_order = 2
se_ftype = 2 ! actually output NPHYS; overloaded use of ftype
/
&vert_nl
/
&analysis_nl
tool = 'topo_pgn_to_smoothed'
infilenames = 'ne30pg4_c2t_topo.nc', 'ne30np4pg2_smoothed_phis'
/

mpirun -np 8 ./homme_tool < input.nl

4. cube_to_target, run 2:

${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \
  --target-grid ne30pg2_scrip.nc \
  --input-topography USGS-topo-cube3000.nc \
  --smoothed-topography ne30np4pg2_smoothed_phis1.nc \
  --output-topography USGS-gtopo30_ne30np4pg2_16xdel2.nc

5. ncks

ncks -A ne30np4pg2_smoothed_phis1.nc USGS-gtopo30_ne30np4pg2_16xdel2.nc

USGS-gtopo30_ne30np4pg2_16xdel2.nc is the final GLL-physgrid topography file.
In particular, clean up Kokkos. This is called from prim_finalize.
@ambrad ambrad added BFB PR leaves answers BFB HOMME HOMME standalone issues with the standalone HOMME code that dont impact E3SM labels Jan 18, 2020
@ambrad ambrad requested a review from mt5555 January 18, 2020 22:42
@ambrad
Copy link
Member Author

ambrad commented Jan 18, 2020

e3sm_developer passes on sandiatoss3.

@mt5555 mt5555 self-assigned this Jan 18, 2020
mt5555 added a commit that referenced this pull request Jan 22, 2020
Call prim_finalize from EAM, and call Kokkos::finalize when using SL transport (indirectly) from prim_finalize.
Final tool for physgrid topography file.
Fix reading of GLL PHIS_d when reading the new physgrid topo file format.

[BFB]
mt5555 added a commit that referenced this pull request Jan 23, 2020
Call prim_finalize from EAM, and call Kokkos::finalize when using SL transport (indirectly) from prim_finalize.
Final tool for physgrid topography file.
Fix reading of GLL PHIS_d when reading the new physgrid topo file format.

[BFB]
@mt5555 mt5555 merged commit 7259724 into E3SM-Project:master Jan 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFB PR leaves answers BFB HOMME standalone issues with the standalone HOMME code that dont impact E3SM HOMME
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants