Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
drewpendergrass committed Dec 7, 2023
1 parent da69019 commit aec0a55
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 14 deletions.
12 changes: 9 additions & 3 deletions docs/Debugging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,15 @@ Updates to scale factors but not emissions

If SCALEFACTOR.nc files are changing over time, but ensemble GC simulations do not show any emission changes, there is likely a disconnect between CHEEREIO and GEOS-Chem. This is diagnosed most easily by comparing control emissions with ensemble emissions by using the relevant HEMCO diagnostics (e.g. total methane emissions for a methane inversion are identical between control and ensemble simulations). If this happens to you, it is usually because of a problem with the HEMCO_Config.rc file. In th Check to see that CHEEREIO successfully linked individual emissions with SCALEFACTOR.nc files in your template run directory. CHEEREIO scalefactor codes begin at 700 and increment by 1 for each emission you adjust.

Here is an example emission file adjusted by CHEEREIO in a methane assimilation simulation: ``0 GEPA_GAS_PRODUCTION $ROOT/CH4/v2017-10/GEPA/GEPA_Annual.nc emissions_1B2b_Natural_Gas_Production 2012/1/1/0 C xy molec/cm2/s CH4 1008/700 2 100``

This emissions file is scaled correctly farther down in the the HEMCO_Config.rc file: ``700 ASSIM_CH4 /n/holyscratch01/jacob_lab/dpendergrass/GC-LETKF/METHANE_scale_v06/template_run/CH4_SCALEFACTOR.nc Scalar 2018-2020/1-12/1-31/0-23 RF xy 1 1``
Here is an example emission file adjusted by CHEEREIO in a methane assimilation simulation:
::
0 GEPA_GAS_PRODUCTION $ROOT/CH4/v2017-10/GEPA/GEPA_Annual.nc emissions_1B2b_Natural_Gas_Production 2012/1/1/0 C xy molec/cm2/s CH4 1008/700 2 100

This emissions file is scaled correctly farther down in the the HEMCO_Config.rc file:
::
700 ASSIM_CH4 /n/holyscratch01/jacob_lab/dpendergrass/GC-LETKF/METHANE_scale_v06/template_run/CH4_SCALEFACTOR.nc Scalar 2018-2020/1-12/1-31/0-23 RF xy 1 1``

This tells GEOS-Chem that it should scale natural gas production emissions by the CHEEREIO posterior scaling factor values stored in the CH4_SCALEFACTOR.nc file. Sometimes, especially in cases where emissions are grouped into chemical families, CHEEREIO does not successfully update HEMCO_Config.rc on its own and requires manual user correction.

Expand Down
18 changes: 9 additions & 9 deletions docs/Existing-observation-operators.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ To activate TROPOMI observations, list "TROPOMI" as an observation type in the `
TROPOMI fields available to save and plot in CHEEREIO
~~~~~~~~~~~~~

All observation operators save standard data from observations and GEOS-Chem and pass it on to CHEEREIO (including latitude, longitude, observation values, GC simulated observation values, and timestamps). However, individual operators might also be able to save additional data, such as albedo in remote sensing cases or site IDs in surface observation cases. Users can list the additional data they would like to save in the ``EXTRA_OBSDATA_FIELDS_TO_SAVE_TO_BIG_Y`` setting in ``ens_config.json``, following the instructions on the :ref:`postprocessing settings` page. **With the TROPOMI operator, all data read in by the ``read_tropomi`` function are available to be saved and/or plotted.** Below is a subset of the supported fields which users can save in TROPOMI:
All observation operators save standard data from observations and GEOS-Chem and pass it on to CHEEREIO (including latitude, longitude, observation values, GC simulated observation values, and timestamps). However, individual operators might also be able to save additional data, such as albedo in remote sensing cases or site IDs in surface observation cases. Users can list the additional data they would like to save in the ``EXTRA_OBSDATA_FIELDS_TO_SAVE_TO_BIG_Y`` setting in ``ens_config.json``, following the instructions on the :ref:`postprocessing settings` page. **With the TROPOMI operator, all data read in by the read_tropomi function are available to be saved and/or plotted.** Below is a subset of the supported fields which users can save in TROPOMI:

.. option:: qa_value

Expand Down Expand Up @@ -80,35 +80,35 @@ The remaining three utility functions are very similar. They read TROPOMI level

.. py:function:: read_tropomi(filename, species, filterinfo=None, includeObsError = False)
**Designed for the TROPOMI operational level 2 product.** A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``DEFAULT`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.
**Designed for the TROPOMI operational level 2 product.** A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``DEFAULT`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.

:param str filename: NetCDF file containing TROPOMI observations to be loaded. Expects standard level 2 data.
:param str species: Name of species to be loaded. Currently, only "CH4" is supported, though an "NO2" operator is partially written.
:param dict filterinfo: A dictionary of information about data filtering which is passed to a standard observation operator utility function. See :ref:`Observation filters` for more information
:param bool includeObsError: True or False, read the errors associated with individual observations.
:return: A dictionary containing observation values and metadata, ready for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class.
:return: A dictionary containing observation values and metadata, ready for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class.
:rtype: dict

.. py:function:: read_tropomi_acmg(filename, species, filterinfo=None, includeObsError = False)
**Designed for the Harvard/ACMG version of the TROPOMI operational level 2 product.** A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``ACMG`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.
**Designed for the Harvard/ACMG version of the TROPOMI operational level 2 product.** A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``ACMG`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.

:param str filename: NetCDF file containing TROPOMI observations to be loaded. Expects standard level 2 data.
:param str species: Name of species to be loaded. Currently, only "CH4" is supported, though an "NO2" operator is partially written.
:param dict filterinfo: A dictionary of information about data filtering which is passed to a standard observation operator utility function. See :ref:`Observation filters` for more information
:param bool includeObsError: True or False, read the errors associated with individual observations.
:return: A dictionary containing observation values and metadata, ready for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class.
:return: A dictionary containing observation values and metadata, ready for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class.
:rtype: dict

.. py:function:: read_tropomi_gosat_corrected(filename, species, filterinfo=None, includeObsError = False)
**Designed for the Belasus et al., 2023 version of the TROPOMI operational level 2 product,** but also works for the TROPOMI science product. A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``BLENDED`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.
**Designed for the Belasus et al., 2023 version of the TROPOMI operational level 2 product,** but also works for the TROPOMI science product. A utility function which loads TROPOMI observations from file, filters them, and returns a dictionary of important data formatted for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class. This function is selected when users supply the ``BLENDED`` value to the ``WHICH_TROPOMI_PRODUCT`` setting in the ``TROPOMI_CH4_extension.json`` file.

:param str filename: NetCDF file containing TROPOMI observations to be loaded. Expects standard level 2 data.
:param str species: Name of species to be loaded. Currently, only "CH4" is supported, though an "NO2" operator is partially written.
:param dict filterinfo: A dictionary of information about data filtering which is passed to a standard observation operator utility function. See :ref:`Observation filters` for more information
:param bool includeObsError: True or False, read the errors associated with individual observations.
:return: A dictionary containing observation values and metadata, ready for input into the `gcCompare`` method of the ``TROPOMI_Translator`` class.
:return: A dictionary containing observation values and metadata, ready for input into the ``gcCompare`` method of the ``TROPOMI_Translator`` class.
:rtype: dict

.. _OMI tools:
Expand All @@ -134,13 +134,13 @@ The OMI observation operator calls two utility functions to process observations

.. py:function:: read_omi(filename, species, filterinfo=None, includeObsError = False)
A utility function which loads OMI observations from file, filters them, and returns a dictionary of important data formatted for input into the `gcCompare`` method of the ``OMI_Translator`` class.
A utility function which loads OMI observations from file, filters them, and returns a dictionary of important data formatted for input into the ``gcCompare`` method of the ``OMI_Translator`` class.

:param str filename: NetCDF file containing OMI observations to be loaded. Expects standard level 2 data.
:param str species: Name of species to be loaded. Currently, only "NO2" is supported.
:param dict filterinfo: A dictionary of information about data filtering which is passed to a standard observation operator utility function. See :ref:`Observation filters` for more information
:param bool includeObsError: True or False, read the errors associated with individual observations.
:return: A dictionary containing observation values and metadata, ready for input into the `gcCompare`` method of the ``OMI_Translator`` class.
:return: A dictionary containing observation values and metadata, ready for input into the ``gcCompare`` method of the ``OMI_Translator`` class.
:rtype: dict

.. py:function:: clearEdgesFilterByQAAndFlatten(met)
Expand Down
4 changes: 2 additions & 2 deletions docs/The-ensemble-configuration-file.rst
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ The first section of the ``ens_config.json`` file (i.e. between the first two co

If ``AMPLIFY_ENSEMBLE_SPREAD_FOR_FIRST_ASSIM_PERIOD`` is set to "true", then this is the factor with which CHEEREIO will multiply the ensemble standard deviation at the end of the ensemble spinup period. For more information on the burn-in period, see :ref:`Burn in period`.

..option:: DO_BURN_IN
.. option:: DO_BURN_IN

Should CHEEREIO do a burn-in period? "true" or "false." A burn-in period is a time period where full LETKF assimilation is being applied, but the results will be discarded from final analysis. The idea of a burn in period is to allow CHEEREIO's emissions to "catch up" with the system, as it takes time for the updated emissions in CHEEREIO to become consistent with observations. For more information on the burn-in period, see :ref:`Burn in period`.

Expand Down Expand Up @@ -515,7 +515,7 @@ Observation settings

.. attention::

*Note: Lee Murray reports that NOAA doesn't use consistent fields across ObsPack versions. If you get an error the preprocessing step (performed during ensemble run directory creation in the installation workflow), or you already have ObsPack files processed, you should set the ``preprocess_raw_obspack_files`` to ``false`` and supply an already populated directory of manually preprocessed files. Details for how to do this are provided in the ObsPack documentation for GEOS-Chem. (DP, 2023/12/06).*
*Note: Lee Murray reports that NOAA doesn't use consistent fields across ObsPack versions. If you get an error the preprocessing step (performed during ensemble run directory creation in the installation workflow), or you already have ObsPack files processed, you should set the preprocess_raw_obspack_files variable to false and supply an already populated directory of manually preprocessed files. Details for how to do this are provided in the ObsPack documentation for GEOS-Chem. (DP, 2023/12/06).*

.. option:: obspack_gc_input_file

Expand Down

0 comments on commit aec0a55

Please sign in to comment.