[FEATURE REQUEST] Skip reading FLASH_DENS and CONV_DEPTH fields when lightning NOx extension is turned off

[msulprizio]

Not all simulations require the lightning NOx extension, and therefore don't need to read in the fields FLASH_DENS and CONV_DEPTH. Those fields are essentially treated as met fields -- they're read into State_Met in GeosCore/flexgrid_read_mod.F90. As of now there is no easy way to turn off reading those fields if the lightning NOx extension is turned off. Instead users need to go into the source code as described on issue #277.

[ltmurray]

There are quite a few met fields that are never used by GEOS-Chem, e.g., the SEAICE{01..90} fields are only used by the Hg simulation, and some are just read in to be re-written as diagnostic output. It would be more generic if GEOS-Chem had the feature to ignore any met field specified in HEMCO_Config.rc with /dev/null, as I believe GCHP allows? Since the speciality simulations that do not use lightning NOx don't call the lightning NOx extension, it doesn't matter what is in the lightning flash State_Met field. This would facilitate coupling with met fields from other external GCMs/CCMs as well.

[msulprizio]

@ltmurray You make a great point. I am planning on doing this work for the methane simulation (i.e. skipping reads of unnecessary met fields). I can think of a way to expand this to all simulations so that only the met fields needed by that simulation are read by HEMCO.

[msulprizio]

This feature will be included in 12.9.0 because it will require changes in HEMCO_Config.rc files and therefore will require the creation of new run directories.

[msulprizio]

In commit 3e2886a, FLASH_DENS and CONV_DEPTH fields are now skipped when the lightning NOx extension is turned off. This will be included in 12.9.0.

The more general request of skipping other met fields that are not needed by simulations can be tracked in the separate issue #91.

[jennyfisher]

Hi @msulprizio
We're running some v12.8.2 tagged CO simulations for early 2020 to simulate the Australian fires. The model is crashing because we don't have the lightning fields (which don't yet exist) - but we don't need them because it's tagged CO. Besides upgrading to v12.9.0, what's the best way around this situation in the interim? Would we be best copying in your code changes from the commit above, or is there something in HEMCO_Config.rc we could change as a stop-gap? Thanks!

[ltmurray]

Hi Jenny, changing the behavior flag to “C” should work. It will then read in 2019.

[msulprizio]

Yes, Lee's suggestion is a quick an easy way to get past the error. The lightning fields for 2019 will be read in that case (and still not used as you point out). If you want to skip the read completely in GEOS-Chem 12.8.2 and earlier, you can comment out those lines in HEMCO_Config.rc:

#* FLASH_DENS $ROOT/OFFLINE_LIGHTNING/v2020-03/$MET/$YYYY/FLASH_CTH_$MET_0.5x0.625_$YYYY_$\
MM.nc4  LDENS 1980-2020/1-12/1-31/0-23/+90minute RFY3 xy  1  * -  1 1
#* CONV_DEPTH $ROOT/OFFLINE_LIGHTNING/v2020-03/$MET/$YYYY/FLASH_CTH_$MET_0.5x0.625_$YYYY_$\
MM.nc4  CTH   1980-2020/1-12/1-31/0-23/+90minute RFY3 xy  1  * -  1 1

and also comment out the corresponding lines in GeosCore/flexgrid_read_mod.F90:

!! Read FLASH_DENS                                                                  
!v_name = "FLASH_DENS"
!CALL Get_Met_2D( State_Grid, Q2, TRIM(v_name) )
!State_Met%FLASH_DENS = Q2
!
!! Read CONV_DEPTH                                                                  
!v_name = "CONV_DEPTH"
!CALL Get_Met_2D( State_Grid, Q2, TRIM(v_name) )
!State_Met%CONV_DEPTH = Q2

You would then need to recompile the code. The fields won't be read and will remain all zeroes.

[jennyfisher]

Thanks both!

[jennyfisher]

Actually changing the flag to C (or CS) didn't work.
HEMCO ERROR: Cannot find file for current simulation time: /g/data/m19/geos-chem/data/ExtData/HEMCO/OFFLINE_LIGHTNING/v2020-03/MERRA2/2019/FLASH_CTH_MERRA2_0.5x0.625_2019_12.nc4 - Cannot get field FLASH_DENS. Please check file name and time (incl. time ra
We'll try Melissa's fix now.

[ltmurray]

Huh, that's weird. Any idea why, Melissa? Melissa's fix should definitely work, but as another quick offline alternative, you can just take one of the previous MERRA2 leap-year files (e.g., 2016), and change the time units with ncatted or ncap2 to be relative to 2020, e.g.,

ncatted -a units,time,m,c,"minutes since 2020-01-01 00:00:00.0" FLASH_CTH_MERRA2_0.5x0.625_2016_01.nc4 FLASH_CTH_MERRA2_0.5x0.625_2020_01.nc4

or

ncap2 -O -s 'time@units="minutes since 2020-01-01 00:00:00.0";' FLASH_CTH_MERRA2_0.5x0.625_2016_01.nc4 FLASH_CTH_MERRA2_0.5x0.625_2020_01.nc4

[jennyfisher]

Thanks Lee. We'll stick with Melissa's as a fake file will be an issue when we start to run full chem for that period (which is fairly imminent). On that note - what do you recommend we do for 2020 runs since there are no lightning fields yet? We're really mostly interested in what's happening near the surface, but it seems not ideal to have lightning turned off completely. Is there a climatology we can use, or some other option? What's the timeframe for 2020 fields?

[msulprizio]

I'm not sure why using 'C' didn't work, but setting verbose and warnings to 3 would tell us why if we wanted to dig further. That would provide information in the HEMCO.log file about what the expected date/time is and what it finds.

For what it's worth, we're planning on generating a lightning climatology file for the 13.0.0 release by running a 10-year HEMCO standalone simulation. The idea is that if a user wants to run for a time period for which lightning data isn't available, then HEMCO will crash with an error letting them know that they have the choice to use the climatology file. We'll do the same for volcanoes, which also does not have 2020 data at this time.

[jennyfisher]

Hi Melissa,
Thanks - it's a student project, and she's finally running, so I don't want to try to do more serious debugging with her. But I'll note we've had the same problem with the WMO_2018 SfcFix files. When we used the dry run we only downloaded a few of them; trying to run a different year caused the model to crash even though the flag is set to C (in this case I'm glad it did because we wouldn't have realised the files were missing!). For example, all lines look like this:
* SfcVMR_CCl4 $ROOT/SfcFix/v2019-12/WMO_2018/surface_VMRs_WMO2018_$YYYY.nc CCl4 1960-2100/1-12/1/0 C xy v/v * 802 1 1

Have seen in 12.8.1 and 12.8.2 out of the box code. So it may be a more pervasive problem with HEMCO...

Thanks for the note about the climatology - that will be really useful!!

Lee - any recommendation on what to do for 2020 lightning in the interim?

Cheers,
Jenny

[ltmurray]

Hi Jenny, I have MERRA-2 lightning generated for Jan-Mar, 2020 so far — send me an offline message if you want me to send them to you. I am presently processing April 2020. Any user who sees this, please always feel free to contact me directly for the most recent available lightning.

BTW, there is no need to use HEMCO to generate a lightning climatology; I generate this using CDO during the process of creating the scaling factors, so it already exists.

Also, I've noticed an issue with dryrun failing to download fields newer in time if a subset of older fields have already been downloaded, including the surface files. E.g., if previously I used dryrun to download and run 2018, but then later use dryrun for a new 2019 simulation, any fields flagged with "C" do not show up as missing in the log file and the downloader skips them. But when the full model runs, it expects the field to exist based on the temporal specification (e.g., 1960-2100), and crashes when it can't find the file for that year, usually showing a random year in the error message. I haven't had time to put together a toy example to confirm that this is what is happening, but was going to probably put together a bug ticket. Just mentioning, in case you want to explore in the meantime, since it sounds potentially related to what Jenny is seeing.

[jennyfisher]

Thanks Lee, I'll follow up offline.

Re dryrun, see #312 (in theory fixed, though I'm now having an issue where the dryrun thinks all my files are missing when they are there!).

[msulprizio]

Thanks, Lee! Can you send us the climatology files you've generated? We'll be adding that as an option in 13.0.0.

[ltmurray]

Hi Melissa, I put lightning.clim.tgz in my ~/share directory on Odyssey. If you untar that from within the OFFLINE_LIGHTNING root directory, it will place the climatologies alongside the time-varying data, including important README disclaimers about the use of lightning climatology versus time-varying fields on the photochemistry. Let me know if you have difficulty accessing!