Skip to content

Commit

Permalink
Replace input.geos with geoschem_config.yml; assorted minor updates
Browse files Browse the repository at this point in the history
Signed-off-by: Lizzie Lundgren <elundgren@seas.harvard.edu>
  • Loading branch information
lizziel committed Jun 15, 2022
1 parent f4947d7 commit b6e1b4d
Show file tree
Hide file tree
Showing 9 changed files with 41 additions and 47 deletions.
18 changes: 10 additions & 8 deletions docs/source/getting-started/quick-start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,8 @@ You therefore need to run it to actually apply the settings:

.. code-block:: console
$ vim setCommonRunSettings.sh # edit simulation settings here
$ ./setCommonRunSettings.sh # applies the updated settings
$ vim setCommonRunSettings.sh # edit simulation settings here
$ ./setCommonRunSettings.sh # applies the updated settings
Simulation start date is set in :file:`./cap_restart`.
Run directories come with this file filled in based on date of the initial restart file in subdirectory :literal:`./Restarts`.
Expand All @@ -136,12 +136,14 @@ You need to set this symbolic link before running:

.. code-block:: console
$ ./setRestartLink.sh # sets symbolic link gchp_restart.nc4
$ ./setRestartLink.sh # sets symbolic link to target file in Restarts
If you used an environment file to load libraries prior to building GCHP then you should load that file prior to running GCHP. A simple way to make sure you always use the correct combination of libraries is to set the GCHP environment symbolic link :literal:`./gchp.env` in the run directory:
If you used an environment file to load libraries prior to building GCHP then you should load that file prior to running. A simple way to make sure you always use the correct combination of libraries is to set the GCHP environment symbolic link :literal:`./gchp.env` in the run directory:

$ ./setEnvironment.sh /path/to/env/file # sets symbolic link gchp.env
$ source gchp.env
.. code-block:: console
$ ./setEnvironment.sh /path/to/env/file # sets symbolic link gchp.env
$ source gchp.env # applies the environment settings
6. Run GCHP
Expand All @@ -163,10 +165,10 @@ This means that you write a script (usually bash) that configures and runs your
Example job scripts are provided in subdirectory :literal:`./runScriptSamples` in the run directory.
That folder also includes an example script for running GCHP from the command line.

Several steps beyond running GCHP are included in the example run scripts. These include loading the environment, updating common settings, and setting the restart file link based on start time and grid resolution. In addition, the output restart file is moved and renamed to the :literal:`./Restarts` subdirectory upon successful completion of the run.
Several steps beyond running GCHP are included in the example run scripts. These include loading the environment, updating commonly changed run settings, and setting the restart file based on start time and grid resolution. In addition, the output restart file is moved and renamed to the :literal:`./Restarts` subdirectory upon successful completion of the run.

.. note::
When GCHP runs to completion it updates the start date in :file:`cap_restart` to be the end date of your run. This is so that you can easily submit a new GCHP run starting off where your last run left off. If your GCHP run failed to complete this file will not be updated. In addition, GCHP outputs a restart file to your run directory called :file:`gcchem_internal_checkpoint`. Upon successful completion of a run this file should be moved and renamed to subdirectory :literal:`./Restarts` to include the date and grid resolution. This is good for archiving and enables the :file:`./setRestartLink.sh` script to work for the new start time. This step is included in all sample GCHP run scripts.
File :file:`cap_restart` is over-written to contain the run end date upon successful completion of a GCHP run. This is done within GCHP and not by the run script. You can then easily submit a new GCHP run starting off where your last run left off. In addition, GCHP outputs a restart file to your run directory called :file:`gcchem_internal_checkpoint`. This file is moved to subdirectory :literal:`Restarts` and renamed to include the date and grid resolution. This is done by the run script and technically is optional. We recommend doing this since it is is good for archiving (restart files will contain date and grid res) and enables use of the :file:`./setRestartLink.sh` script to set the :file:`gchp_restart.nc4` symbolic link.

Those are the basics of using GCHP!
See the user guide, step-by-step guides, and reference pages for more detailed instructions.
25 changes: 9 additions & 16 deletions docs/source/stretched-grid.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,21 @@ Stretched-Grid Simulations
==========================

.. note::
Stretched-grid simulations are described in :cite:`Bindle_et_al._2021`. The paper also discusses things
you should consider, and offers guidance for choosing appropriate stretching parameters.
Stretched-grid simulations are described in :cite:`Bindle_et_al._2021`. This paper also discusses related topics of consideration and offers guidance for choosing appropriate stretching parameters.

A stretched-grid is a cubed-sphere grid that is "stretched" to enhance its resolution in a region.
To set up a stretched-grid simulation you need to do the following:

#. Choose stretching parameters, including stretch factor and target latitude and longitude.
#. Create a stretched grid restart file for your simulation using your chosen stretch parameters.
#. Update :file:`setCommonRunSettings.sh` to specify the stretched grid used in the restart file.
#. Configure using the stretched grid restart file
#. Configure the GCHP run directory to specify stretched grid parameters in :file:`setCommonRunSettings.sh` and use your stretched grid restart file.

Before starting you will need to choose stretching parameters.

Choose stretching parameters
----------------------------

The :term:`target face` is face of a stretched-grid that shrinks so that the grid resolution is
The :term:`target face` is the face of a stretched-grid that shrinks so that the grid resolution is
finer. The target face is centered on a target point, and the degree of stretching is controlled by
a parameter called the stretch-factor. Relative to a normal cubed-sphere, the resolution of the
target face is refined by approximately the stretch-factor. For example, a C60 stretched-grid with a
Expand Down Expand Up @@ -75,7 +73,7 @@ GCPy documentation for this program's exact usage, and for installation instruct
Description of arguments:

.. option:: -i initial_GEOSChem.Restart.20190701_000000z.c90.nc
.. option:: -i GEOSChem.Restart.20190701_000000z.c90.nc

Specifies the input restart file is :file:`GEOSChem.Restart.20190701_000000z.c90.nc4` (in the current working directory).

Expand Down Expand Up @@ -103,8 +101,8 @@ Description of arguments:
Once you have created a restart file for your simulation, you can move on to updating your
simulation's configuration files.

Update your configuration files
-------------------------------
Configure the run directory
---------------------------

Modify the section of :file:`setCommonRunSettings.sh` that controls the simulation grid. Turn
:envvar:`STRETCH_GRID` to :literal:`ON` and update :envvar:`CS_RES`, :envvar:`STRETCH_FACTOR`,
Expand Down Expand Up @@ -150,11 +148,6 @@ Execute :program:`./setCommonRunSettings.sh` to update to update your run direct
if your effective resolution is greater than or equal to C180-equivalent.


Configure the restart file
--------------------------

Update :file:`cap_restart` to match the date of your restart file which will also be the start date of the run.
Copy or symbolically link your restart file in the :literal:`Restarts` subdirectory with the proper filename format. The format includes global resolution but not stretched grid resolution so it is a good idea to symbolically link it if you want to preserve the original file's specification of stretched grid in its name.
Run :literal:`setRestartLink.sh` to set symbolic link :file:`gchp_restart.nc4` to point to your restart file based on start date in :file:`cap_restart` and global grid resolution in :file:`setCommonRunSettings.sh`.

You will also need to configure the run directory to use the stretched grid restart file. Update :file:`cap_restart` to match the date of your restart file. This will also be the start date of the run.
Copy or symbolically link to your restart file in the :literal:`Restarts` subdirectory with the proper filename format. The format includes global resolution but not stretched grid resolution so it is a good idea to symbolically link to the original if you want to preserve the original file's specification of stretched grid in its name.
Run :literal:`setRestartLink.sh` to set symbolic link :file:`gchp_restart.nc4` to point to your restart file based on start date in :file:`cap_restart` and global grid resolution in :file:`setCommonRunSettings.sh`.
2 changes: 1 addition & 1 deletion docs/source/supplement/caching-input-data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Make sure to set :literal:`FIND_PATH` to ExtData and :literal:`REPLACE_PATH` to
gcuser:/MyRunDirectory$ swap_extdata_link ChemDir
gcuser:/MyRunDirectory$ swap_extdata_link HcoDir
gcuser:/MyRunDirectory$ swap_extdata_link MetDir
gcuser:/MyRunDirectory$ sed -i "s#${FIND_PATH}#${REPLACE_PATH}#g" HEMCO_Config.rc input.geos
gcuser:/MyRunDirectory$ sed -i "s#${FIND_PATH}#${REPLACE_PATH}#g" HEMCO_Config.rc geoschem_config.yml
Now your GCHP simulation will use input data from ExtDataCache.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/supplement/config-files.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ See the individual subsections on each file for additional information.
:file:`GCHP.rc`
Controls high-level aspects of the simulation, including grid type and resolution, core distribution, stretched-grid parameters, timesteps, and restart file configuration.

:file:`input.geos`
:file:`geoschem_config.yml`
Primary config file for GEOS-Chem.
Same function as in GEOS-Chem Classic except grid, simulation start/end, met field source, timers, and data directory information are ignored.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/supplement/config-files/HEMCO.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ HEMCO_Config.rc, HEMCO_Diagn.rc
HEMCO_Config.rc
-----------------------

Like :file:`input.geos`, information about the :file:`HEMCO_Config.rc` file is the same as for GEOS-Chem Classic with a few exceptions.
Like :file:`geoschem_config.yml`, information about the :file:`HEMCO_Config.rc` file is the same as for GEOS-Chem Classic with a few exceptions.
Refer to the HEMCO documentation for an overview of the file.

Some content of the :file:`HEMCO_Config.rc` file is ignored by GCHP.
Expand Down
14 changes: 14 additions & 0 deletions docs/source/supplement/config-files/geoschem_config.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
input.geos
=================

Information about the :file:`geoschem_config.yml` file is the same as for GEOS-Chem Classic with a few exceptions.
See the :file:`geoschem_config.yml` file wiki page for an overview of the file.

The :file:`geoschem_config.yml` file used in GCHP is different in the following ways:

* Start/End datetimes are ignored. Set this information in :file:`CAP.rc` instead.
* Root data directory is ignored. All data paths are specified in :file:`ExtData.rc` instead with the exception of the FAST-JX data directory which is still listed (and used) in :file:`geoschem_config.yml`.
* Met field is ignored. Met field source is described in file paths in :file:`ExtData.rc`.
* GC classic timers setting is ineffectual. GEOS-Chem Classic timers code is not compiled when building GCHP.

Other parts of the GEOS-Chem Classic :file:`geoschem_config.yml` file that are not relevant to GCHP are simply not included in the file that is copied to the GCHP run directory.
14 changes: 0 additions & 14 deletions docs/source/supplement/config-files/input_geos.rst

This file was deleted.

5 changes: 2 additions & 3 deletions docs/source/supplement/containers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,7 @@ Performance
-----------

Because we do not include optimized infiniband libraries within the provided Docker images, container-based GCHP is currently not as fast as other setups.
Container-based benchmarks on Harvard's Cannon cluster up to 360 cores and c90 (~1x1.25) resolution averaged 15% slower than equivalent non-container runs,
and may perform worse at a higher core count and resolution.
Container-based benchmarks deployed on Harvard's Cannon cluster using up to 360 cores at c90 (~1x1.25) resolution averaged 15% slower than equivalent non-container runs. Performance may worsen at a higher core count and resolution.
If this performance hit is not a concern, these containers are the quickest way to setup and run GCHP.


Expand Down Expand Up @@ -124,7 +123,7 @@ Luckily we can use a GC Classic container to execute a dry-run that matches the
$ #get pre-compiled GC Classic executable
$ singularity exec -B .:/classic_rundir ../gcc.sif cp /opt/geos-chem/bin/gcclassic /classic_rundir
Make sure to tweak dates of run in input.geos as needed, following info `here <http://wiki.seas.harvard.edu/geos-chem/index.php/Downloading_data_with_the_GEOS-Chem_dry-run_option#Executing_GEOS-Chem_in_dry-run_mode>`__.
Make sure to tweak dates of run in geoschem_config.yml as needed, following info `here <http://wiki.seas.harvard.edu/geos-chem/index.php/Downloading_data_with_the_GEOS-Chem_dry-run_option#Executing_GEOS-Chem_in_dry-run_mode>`__.

.. code-block:: console
Expand Down
6 changes: 3 additions & 3 deletions docs/source/supplement/rundir-config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ GCHP is controlled using a set of configuration files that are included in the G
1. :file:`CAP.rc`
2. :file:`ExtData.rc`
3. :file:`GCHP.rc`
4. :file:`input.geos`
4. :file:`geoschem_config.yml`
5. :file:`HEMCO_Config.rc`
6. :file:`HEMCO_Diagn.rc`
7. :file:`input.nml`
Expand Down Expand Up @@ -115,7 +115,7 @@ Turn on/off model components
""""""""""""""""""""""""""""

You can toggle all primary GEOS-Chem components, including type of mixing, from within :file:`setCommonRunSettings.sh`.
The settings in that file will update :file:`input.geos` and :file:`HEMCO_Config.rc` automatically.
The settings in that file will update :file:`geoschem_config.yml` and :file:`HEMCO_Config.rc` automatically.
Look for section :literal:`Turn Components On/Off`.
Please note that you can turn on/off all base emissions from :file:`setCommonRunSettings.sh` but you cannot toggle HEMCO extensions or individual base emissions. For this you must manually edit :file:`HEMCO_Config.rc`.

Expand Down Expand Up @@ -278,7 +278,7 @@ Enable maximum print output
Besides compiling with :literal:`CMAKE_BUILD_TYPE=Debug`, there are a few settings you can configure to boost your chance of successful debugging.
All of them involve sending additional print statements to the log files.

1. Set Turn on debug printout? in input.geos to T to turn on extra GEOS-Chem print statements in the main log file.
1. Set Turn on debug printout? in :file:`geoschem_config.yml` to T to turn on extra GEOS-Chem print statements in the main log file.
2. Set the Verbose and Warnings settings in :file:`HEMCO_Config.rc` to maximum values of 3 to send the maximum number of prints to :file:`HEMCO.log`.
3. Set :literal:`CAP.EXTDATA` option :literal:`root_level` in :file:`logging.yml` to :literal:`DEBUG` to send root thread MAPL ExtData (input) prints to :file:`allPEs.log`.
4. Set :literal:`CAP.EXTDATA` option :literal:`level` in :file:`logging.yml` to :literal:`DEBUG` to send all thread MAPL ExtData (input) prints to :file:`allPEs.log`.
Expand Down

0 comments on commit b6e1b4d

Please sign in to comment.