Merge pull request #693 from ssec/dockathys

Update documentation for G2G Version 1.2 release

NEWS_GEO2GRID.rst

@@ -1,9 +1,24 @@
 Release Notes
 =============
 
-Version 1.1.0 (2022-12-12)
+Version 1.2.0 (2023-05-10)
 --------------------------
+* Preliminary GOES-19 ABI reader support added
+* Preliminary EUMETSAT MTG FCI (fci_l1c_nc) reader support added
+* Additional ABI Product readers support added:
+
+  * Aerosol Optical Depth (AOD)
+  * Low Cloud and Fog (FLS)
+  * Land Surface Temperature (LST)
+
+* New 3.9 micron band scaling
+* Added ABI AOD product example to documenation
+* Support for additional RGBs
+* Optimizations
+* Bug fixes
 
+Version 1.1.0 (2022-12-12)
+--------------------------
 * GOES-18 ABI reader support added
 * ABI Level 2 (abi_l2_nc) reader added
 * Gridded GLM (glm_l2) reader added

doc/source/compositors.rst

@@ -173,7 +173,7 @@ go over some basic examples of how to make your own composites.
             - night_microphysics
           standard_name: day_night_mix
 
-    Once the .yaml files has been updated, the composite can be generated using the following
+    Once the .yaml file has been updated, the composite can be generated using the following
     command:
 
     .. code-block:: bash
@@ -183,7 +183,7 @@ go over some basic examples of how to make your own composites.
     The image created by executing the command on a GOES-16 ABI Full Disk dataset from 12:30 UTC,
     12 November 2018 is shown below.
 
-    .. figure:: _static/example_images/GOES-16_ABI_RadF_true_color_night_microphysics_20181112_123034_GOES-East_new.png
+    .. figure:: _static/example_images/GOES-16_ABI_RadF_true_color_night_microphysics_20181112_123034_GOES-East_newenhancement.png
         :width: 90%
         :align: center
 

doc/source/conf.py

@@ -84,19 +84,24 @@
     "https://bin.ssec.wisc.edu/pub/CSPP/p2g_v_3_0_examples/amsr2/gcom-w1_amsr2_btemp_89.0ah_20220910_233500_wgs84_lcc_fit_rescaled_and_colorized.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/ahi/HIMAWARI-8_AHI_true_color_20181112_233020_perth_example.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/ahi/HIMAWARI-8_AHI_true_color_20181112_233020_perth.png",
+    "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/abi_20181219_1745_montage.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/abi_20181219_1745_montage.jpg",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadC_natural_color_20181219_174215_GOES-East.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-18_ABI_RadF_natural_color_20221115_183020_GOES-West_cutout.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadC_true_color_20190104_195718_GOES-East.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadF_true_color_night_20181112_123034_GOES-East.jpg",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadF_true_color_night_20181112_123034_GOES-East_new.png",
+    "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadF_true_color_night_microphysics_20181112_123034_GOES-East_newenhancement.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadF_true_color_night_microphysics_20181112_123034_GOES-East.jpg",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/GOES-16_ABI_RadF_true_color_night_microphysics_20181112_123034_GOES-East_new.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/my_goes16_abi_naturalcolor.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/my_goes18_abi_naturalcolor.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/GOES-17_ABI_TEMP_20221123_183117_GOES-West_original.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/goes17_overlay_true_color_cloud_temperature.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/my_goes17_abi_ctt.png",
+    "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/my_goes17_abi_ctt1.png",
+    "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/GOES-16_ABI_AOD_20240429_225020_GOES-East.png",
+    "https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/GOES-16_ABI_AOD_20240429_225020_GOES-East_woverlays.png",
     "https://bin.ssec.wisc.edu/pub/CSPP/p2g_v_2_1_examples/flood/Flood_Legend.png",
 )
 script_path = os.path.dirname(os.path.realpath(__file__))
@@ -192,8 +197,8 @@ def setup(app):
 # General information about the project.
 if is_geo2grid:
     project = "Geo2Grid"
-    version = "1.1"
-    release = "1.1"
+    version = "1.2"
+    release = "1.2"
 else:
     project = "Polar2Grid"
 
@@ -202,10 +207,10 @@ def setup(app):
     # built documents.
     #
     # The short X.Y version.
-    version = "3.0"
+    version = "3.1"
 
     # The full version, including alpha/beta/rc tags.
-    release = "3.0"
+    release = "3.1"
 
 # Tell Polar2Grid scripts which system we are running
 os.environ["USE_POLAR2GRID_DEFAULTS"] = str(int(not is_geo2grid))

doc/source/examples/abi_l2_example.rst

@@ -1,11 +1,8 @@
-Working with ABI Level 2 Cloud Product Files
---------------------------------------------
+Working with ABI Level 2 Product Files
+--------------------------------------
 
 This example walks through the creation of GOES ABI
-Level 2 GeoTIFF product image files and adding overlays.
-
-The Basics of Geo2Grid ABI L2 GeoTIFF File Creation
-***************************************************
+Level 2 Product GeoTIFF image files and adding overlays.
 
 Find the options available when creating GOES-16, -17 and -18
 ABI Level 2 GeoTIFFs:
@@ -16,9 +13,17 @@ List the products that can be created from your ABI dataset:
 
     ``geo2grid.sh -r abi_l2_nc -w geotiff --list-products -f <path_to_files>``
 
-Geo2Grid now supports two different ABI Cloud Products, cloud top
-height (HT) and cloud top temperature (TEMP).  The height files use product
-names ``ACHA`` while the temp files using ``ACHT``.  For instance
+Geo2Grid now supports a number of different CSPP Geo ABI Science Products, including
+cloud top height (HT), cloud top temperature (TEMP), low cloud and fog
+(IFR_Fog_Prob,LIFR_Fog_Prob,MVFR_Fog_Prob,Fog_Depth), Land
+Surface Temperature (LST) and aerosol optical depth (AOD).
+
+Creating ABI L2 GeoTIFF Cloud Product Images
+********************************************
+
+Geo2Grid supports two cloud ABI cloud NetCDF products; cloud height which uses
+a file product name of ``ACHA`` and  cloud temperature files use which use
+``ACHT``.  For instance
 
     ``CG_ABI-L2-ACHAC-M6_G17_s20223271831172_e20223271833556_c20223271834370.nc``
 
@@ -35,7 +40,7 @@ By default the products are color enhanced using the colormap:
 
     ``$GEO2GRID_HOME/colormaps/abi_l2_modified_cloud_top.cmap``.
 
-Create just a Cloud Top Height Geotiff image:
+Create just a Cloud Top Height GeoTIFF image:
 
     ``geo2grid.sh -r abi_l2_nc -w geotiff -p HT -f <path_to_abi_files>``
 
@@ -56,28 +61,29 @@ The resulting image is displayed below.
     :width: 100%
     :align: center
 
-    CSPP GEO ABI AIT GOES-17 Cloud Top Temperature GeoTIFF image (GOES-17_ABI_TEMP_20221123_183117_GOES-West.tif)
+    CSPP Geo ABI AIT GOES-17 Cloud Top Temperature GeoTIFF image (GOES-17_ABI_TEMP_20221123_183117_GOES-West.tif)
 
 .. raw:: latex
 
     \newpage
 
-Add a color table, coastlines, borders and latitude/longitude grid lines to the image, and write the output
-to the file ``my_goes17_abi_ctt.png`` using the ``add_coastlines.sh`` script. The script provides many options,
-including the ability to add a colorbar title using the font of your choice (provide the path
-to the font location on your local machine):
+Add a color table, coastlines, borders and latitude/longitude grid lines
+to the image, and write the output to the file ``my_goes17_abi_ctt.png``
+using the ``add_coastlines.sh`` script. The script provides many options,
+including the ability to add a colorbar title using the font of your
+choice (provide the path to the font location on your local machine):
 
 .. code-block:: bash
 
     add_coastlines.sh GOES-17_ABI_TEMP_20221123_183117_GOES-West.tif --add-colorbar --colorbar-text-color="black"   --colorbar-title="GOES-17 ABI Cloud Top Temperature (°K)  23 November 2022  18:30 UTC"   --add-coastlines --coastlines-outline "black" --coastlines-level 1 --coastlines-resolution=i --add-borders --borders-level 2 --borders-outline gray --coastlines-width 2  --colorbar-tick-marks 10 --colorbar-font /usr/share/fonts/gnu-free/FreeSerifBold.ttf -o my_goes17_abi_ctt.png
 
-.. figure:: ../_static/example_images/my_goes17_abi_ctt.png
+.. figure:: ../_static/example_images/my_goes17_abi_ctt1.png
     :width: 100%
     :align: center
 
-    CSPP GEO ABI AIT GOES-17 Cloud Top Temperature GeoTIFF image with overlays (my_goes17_abi_ctt.png).
+    CSPP Geo ABI AIT GOES-17 Cloud Top Temperature GeoTIFF image with overlays (my_goes17_abi_ctt.png).
 
-Users can aslo overlay Level 2 images onto other GeoTIFFS. In the example execution below, we
+Users can aslo overlay Level 2 images onto other GeoTIFFs. In the example execution below, we
 overaly the Cloud Top Temperature GeoTIFF image on top of the GOES-17 true color image from
 the same time and name the output GeoTIFF "goes17_overlay_true_color_cloud_temperature.tif".
 
@@ -87,8 +93,84 @@ the same time and name the output GeoTIFF "goes17_overlay_true_color_cloud_tempe
 
 The new combined GeoTIFF is displayed below.
 
+.. raw:: latex
+
+    \newpage
+
 .. figure:: ../_static/example_images/goes17_overlay_true_color_cloud_temperature.png
     :width: 100%
     :align: center
 
     CSPP GOES-17 ABI cloud top temperatures overlaid on the coincident true color image from 23 November 2022, 18:31 UTC (goes17_overlay_true_color_cloud_temperature.tif).
+
+Creating ABI L2 GeoTIFF AOD Product Images
+******************************************
+
+One new supported Level 2 is the NOAA GOES-16 Aersol Optical Depth (AOD).
+This product is a satellite retrieval of the extinction of light due to aerosols
+in the atmosphere (for example, dust, smoke and pollution). The NetCDF product
+files include ``AOD`` in the filenames.  For example:
+
+    ``CG_ABI-L2-AODF-M6_G16_s20241202250202_e20241202259524_c20241202305210.nc``
+
+This is a full disk AOD retrieval file from day 120 (April 29), 2024, 22:50 UTC.
+
+You can list the products available from  the file using this command:
+
+.. code-block:: bash
+
+    geo2grid.sh -r abi_l2_nc -w geotiff --list-products -f CG_ABI-L2-AODF-M6_G16_s20241202250202_*.nc
+
+Which shows one product option, ``AOD`` available. This is the array AOD within the
+NetCDF product file which is the *ABI L2+ Aerosol Optical Depth at 550 nm*.
+
+To create a full disk GeoTIFF image output file from this input data set, you can execute
+the following command:
+
+.. code-block:: bash
+
+     geo2grid.sh -r abi_l2_nc -w geotiff -p AOD -f CG_ABI-L2-AODF-M6_G16_s20241202250202_*.nc
+
+By default the products are color enhanced using a rainbow colormap, with values
+ranging from 0.0 to 1.0.  Values above 1.0 are enhanced to dark red.
+
+For this example, we want to focus in on higher aerosol values for this day in Mexico
+and the United States. So I will use the ``--ll-bbox`` option to create a cut out
+region bounded by ``10 to 40`` North Latitude and ``-110 to -80`` West Longitude.
+To create an image covering this domain, I can use this command:
+
+.. code-block:: bash
+
+     geo2grid.sh -r abi_l2_nc -w geotiff --ll-bbox -110 10 -80 40 --num-workers 8 -p AOD -f  CG_ABI-L2-AODF-M6_G16_s20241202250202_*.nc
+
+The resulting image is displayed below:
+
+.. figure:: ../_static/example_images/GOES-16_ABI_AOD_20240429_225020_GOES-East.png
+    :width: 80%
+    :align: center
+
+    CSPP Geo ABI AIT GOES-16 Aersol Optical Depth GeoTIFF image (GOES-16_ABI_AOD_20240429_225020_GOES-East.tif).
+
+Now add an annotated color table, coastlines, borders and latitude/longitude grid lines
+to the image:
+
+.. code-block:: bash
+
+     add_coastlines.sh GOES-16_ABI_AOD_20240429_225020_GOES-East.tif --add-colorbar --colorbar-text-color="black"   --colorbar-title="GOES-16 ABI Aersol Optical Depth  20:52 UTC 29 April 2024"   --add-coastlines --coastlines-outline "black" --coastlines-level 1 --coastlines-resolution=i --add-borders --borders-level 2 --borders-outline gray --borders-width 1 --coastlines-width 2 --colorbar-tick-marks 0.1 --colorbar-minor-tick-marks 0.05
+
+The result is a PNG image with overlays as shown below.
+
+.. raw:: latex
+
+    \newpage
+
+.. figure:: ../_static/example_images/GOES-16_ABI_AOD_20240429_225020_GOES-East_woverlays.png
+    :width: 80%
+    :align: center
+
+    CSPP Geo ABI AIT GOES-16 Aersol Optical Depth GeoTIFF image with added borders, coastlines and an annotated colorbar. The retrievals were created from April 29, 2024, 22:50 UTC observations.
+
+To see the movement of the aersols during this day, an MP4 animation was made of the ABI Full
+Disk daytime AOD overlay files as similarily described in :doc:`creating_animations_example`.
+That animation is availble to view by following
+`this link <https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi_l2/my_aod_animation.mp4>`_.

doc/source/examples/creating_animations_example.rst

@@ -3,7 +3,7 @@ Using Geo2Grid to Create Animations
 
 The advantage of Geostationary Satellites is the temporal resolution of the
 observations.  Geo2Grid offers an easy interface for creating animations from
-Geo2Grid GeoTIFF files.  The following example demonstrates how
+Geo2Grid GeoTIFF and PNG files. The following example demonstrates how
 Geo2Grid software can be used to create an animation of
 files from a latitude/longitude subset of GOES-16 ABI CONUS GeoTIFF images
 located over the Southeastern United States.
@@ -20,7 +20,7 @@ natural color images from all time periods that are available.
 
 	# Set GEO2GRID environment variables
 
-	export GEO2GRID_HOME=/home/g2g/geo2grid_v_1_1
+	export GEO2GRID_HOME=/home/g2g/geo2grid_v_1_2
 	export PATH=$PATH:$GEO2GRID_HOME/bin
 
 	# Get input list of files/times based upon ABI Band 1 files
@@ -64,7 +64,7 @@ filename of ``my_true_color_animation.mp4``. The software can also
 create animations from input ``.png`` files.
 
 The figure below is the last image in the 120 loop sequence.  The
-output MP4 animation is available for viewing at `this site <ftp://ftp.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/my_true_color_animation.mp4>`_.
+output MP4 animation is available for viewing at `this site <https://bin.ssec.wisc.edu/pub/CSPP/g2g_examples/abi/my_true_color_animation.mp4>`_.
 
 .. figure:: ../_static/example_images/GOES-16_ABI_RadC_true_color_20190104_195718_GOES-East.png
     :width: 100%

doc/source/getting_started.rst

@@ -95,7 +95,7 @@ are always available:
         -g <grid_name>        Specify the output grid to use. Default is the Platte Carrée projection, also
                               known as the wgs84 coordinate system. See :doc:`grids` and :doc:`custom_grids`
                               for information on possible values.
-        --num-workers NUM_WORKERS   Specify number of worker threads to use (Default: 4).
+        --num-workers num_workers   Specify number of worker threads to use (Default: 4).
         --progress            Show processing progress bar (Not recommended for logged output).
         -v                    Print detailed log information.
 
@@ -114,7 +114,7 @@ are always available:
         -r 	 	      Instrument input files to read from.
         -w  		      Output format to write to (Currently only option is geotiff).
         -h                    Print helpful information.
-        --list-products       List all possible product options to use with -p from the given input data and exit.
+        --list-products       List all possible geo2grid product options to use with -p from the given input data and exit.
         --list-products-all   List available geo2grid products options and custom/Satpy products and exit.
         -p                    List of products you want to create.
         -f                    Input files and paths.
@@ -123,7 +123,7 @@ are always available:
                               See :doc:`grids` and :doc:`custom_grids` for information on other possible values.
         --cache-dir <dir>     Directory to store resampling intermediate results between executions.
                               Not used with 'native' resampling method.
-        --num-workers NUM_WORKERS   Specify number of worker threads to use (Default: 4).
+        --num-workers num_workers   Specify number of worker threads to use (Default: 4).
         --progress            Show processing progress bar (Not recommended for logged output).
 
         --ll-bbox <lonmin latmin lonmax latmax>    Subset input data to the bounding coordinates specified.
@@ -145,8 +145,12 @@ are always available:
 
         geo2grid.sh -r ami_l1b -w geotiff -p IR112 VI006 --num-workers 12 -f /ami/gk2a_ami_l31b*.nc
 
+        geo2grid.sh -r fci_l1c_nc -w geotiff -p ir_38 true_color -f /fci/MTG-FCI-L1C/*_N_JLS_C_0072*
+
         geo2grid.sh -r agri_fy4a_l1 -w geotiff -p C07 natural_color --progress -f /fy4a/FY4A-_AGRI--*.HDF
 
+        geo2grid.sh -r agri_fy4b_l1 -w geotiff -p C01 C02 C03 -f FY4B-_AGRI--_N_DISK_1330E_L1*_V0001.HDF
+
         geo2grid.sh -r abi_l2_nc -w geotiff -f /cloud_products/CG_ABI-L2-ACH?C-M6_G16*.nc
 
         geo2grid.sh -r glm_l2 -w geotiff -p flash_extent_density -f CG_GLM-L2-GLMF-M3_G18*.nc
@@ -208,20 +212,24 @@ To access these features provide the "reader" and "writer" names to the
         create high quality reprojections in the lowest latency possible
         thanks to the dask python library. Dask splits data arrays in to
         multiple "chunks" and processes them in parallel. The creation of
-        these RGBs includes the following steps, which are performed by
+        these two RGBs includes the following steps, which are performed by
         default with each execution:
 
         * Check for required spectral bands used in RGB creation among input files.
-        * Upsample and sharpen composite bands to the highest spatial resolution (500m).
-        * Creation of pseudo "green" band for the ABI and AGRI instruments.
+        * Upsample and sharpen composite bands to the highest spatial resolution.
+        * Creation of pseudo "green" band if necessary (for instance ABI).
         * Reflectance adjustment (dividing by cosine of the solar zenith angle).
         * Removal of atmospheric Rayleigh scattering (atmospheric correction).
         * Nonlinear scaling before writing data to disk.
 
         Geo2Grid also supports the creation of other RGBs (this varies depending on
         the instrument), however these files are not produced by default.  The
-        recipes for creating these RGBs come from historical EUMETSAT recipes that
-        have been adjusted to work with the data being used in |project|.
+        recipes for creating these RGBs come from recipes that have been
+	decided upon by different satellite governing Agencies. For more
+	information on RGB recipes, please see `this site
+        <https://rammb2.cira.colostate.edu/training/visit/quick_reference/#tab17>`_.
+        Please `Contact Us <http://cimss.ssec.wisc.edu/contact-form/index.php?name=CSPP%20Geo%20Questions>`__
+        if you have any specific questions regarding RGB creation.
 
 
 Creating Your Own Custom Grids