Feature: Export land types and LAI

[lizziel]

This PR allows CLM to export land types and LAI to CAM so that GEOS-Chem may compute dry deposition velocities. This update was originally developed by Thibaud Fritz (MIT) and submitted as #1377 on a CESM 2.1 base. This PR supercedes #1377.

Please note that the first commit of this PR does not yet make changes to the NUOPC layer. This commit will come later and merge should not happen before then. I will do full testing once the updates to NUOPC are in place.

Original message from @fritzt:

Hello!

I have been working on implementing the GEOS-Chem (version 13.0.0) chemistry module in CESM2.1.1. This new option is designated as CESM2-GC. The implementation of GEOS-Chem chemistry inside CESM2 aims to provide an additional chemistry option alongside CAM-Chem.

This is the second PR out of three (CAM, CLM, cime). The main PR can be found at ESCOMP/CAM#378

Description of changes
This PR focuses on passing land types and leaf area indices (LAI) from CLM to CAM through the coupler. I followed the same workflow as what is done for albedo. The land types and LAI are required in CAM to let GEOS-Chem compute its own dry deposition velocities.

Specific notes
Contributors other than yourself, if any: None

CTSM Issues Fixed (include github issue #): None

Are answers expected to change (and if so in what way)?
No, this PR introduces changes that only aim to pass information from CLM to CAM.

Any User Interface Changes (namelist or namelist defaults changes)?
No.

Testing performed, if any:
I was able to output land types and LAI as archived by CAM.

Let me know if I am targetting the wrong branch. The CLM tag I used for my work is release-clm5.0.25. Attached is the top-level Externals.cfg that I used for my project.

Regards,
Thibaud Fritz
Laboratory for Aviation and the Environment

[lizziel]

I just noticed this message in the conversation in #1377:

@mvertens is correct you should ONLY implement this in the new nuopc cap rather than the mct tag as was done here.

For this PR I implemented in mct with the intention of also implementing in nuopc. I can remove the mct changes but have questions about testing with nuopc. Is the version using it ready for testing or will this delay the merge? I also wonder if this means I should change the target branch to something other than master.

[ekluzek]

@lizziel the NUOPC version is ready to use, and has been tested and approved by the SSC. We currently test mainly with MCT, but have about a dozen NUOPC tests that we do as well. When we change the default to NUOPC we'll do the bulk of our testing with NUOPC and do those dozen tests with MCT. This change is going on for CESM now, and will happen with CTSM soon.

[mvertens]

@lizzie - I am happy to help guide you in putting this into CMEPS, CAM and CLM. Maybe a quick meeting would be helpful when you are ready to start doing this.

[ekluzek]

We had meetings both with some of the CAM developers of this work (and myself), and at the weekly CTSM software meeting. So I'll report on both of those, other's feel free to correct what I say. It also seems like we need to bring Francis Vitt and Louisa Emmons in on some of this as well (they are already commenting on the cime issue that covers some of this workhttps://github.com/ESMCI/cime/pull/3974#issuecomment-911921148), the current CAM issue is here: ESCOMP/CAM#378.

[ekluzek]

The CAM Developer meeting included Steve Goldhaber (NCAR), @lizziel (Harvard), Frizt Thibaud (MIT graduating PhD), and Sesabstian Eastham (MIT). This is the general plan that we have for moving this part of the work forward.

• @lizziel will also implement in NUOPC (we might just leave the MCT version of it there as it doesn't hurt anything to have it). (the one caveat that I add to that is that does require changes go into both cime and share repositories, so we'd only retain the MCT version if that's been done and this is all working)
• Steve will give @lizziel access to izumi.unified.ucar.edu our local CGD cluster for testing.
• I will give @lizziel a test list to run (the drydep list of aux_clm tests) and instructions on how to do that on cheyenne and izumi.
• An important detail that I neglected is that we'll need a new test-mod for turning this on that's different from our current drydep testmod since GEOS-Chem will have a different mechanism to turn it on. If @lizziel gives me collaborator access to this branch, I could go ahead and do that step for her, otherwise I'll instruct on how she can do it.
• For this branch to work the Externals.cfg file will need to be updated with the cime/cmeps version that has the needed changes in it. As above I'll either do that or instruct @lizziel on how to do it (also may include updates of share or other externals).
• @lizziel makes sure the list of tests runs on cheyenne and izumi
• @lizziel responds to any PR reviews we have for changes we want to see
• I will do the full testing of it and shepherd taking it all the way to CTSM main-development as well as scheduling it for when it should come in relative to other CTSM tags.
• We think it's important to bring this in largely as it is so that the GEOS-Chem code can remain untouched which will make it easy to bring in updated versions of it into CESM.
• @lizziel will bring in any medium term changes that we think needs to happen with how this is done
• Likewise @lizziel will bring in any long term changes that we think needs to happen with how this is done
  Once both the cime and ctsm tags are made, Steve will shepherd this to CAM development.

We do have concerns about how this is done in sending CTSM internals to CAM, so we may want to change how this is done both in the medium and long term. @lizziel will be the long term support for this and will be the contact for doing those projects (we'll need to keep in mind her other responsibilities so this has to be manageable for her and the GEOS-Chem project).

[ekluzek]

At our CTSM meeting we talked about this issue and also about how well this will hold up for future changes in CTSM. One issue @billsacks noted was that this code assumes that vegetation types are only on a single column within a gridcell. This assumption is going to be broken in future CTSM versions (such as the hillslope version). Depending on what's desired we'll need to either average LAI for the same veg type, or send additional information for other columns that have the same veg type.

Since, the only information it's sending is LAI for each vegetation type on a gridcell as well as the weights, changes to vegetation itself should be fairly isolated from this connection. So we should be able to add or remove new veg types without trouble. The only thing that will change is the total number of veg patches to send through the coupler. Note, also that we already have "virtual" patches where a patch will be defined for a veg-type, but the weight will be zero. So this implementation already handles that case.

The other concern we had was the hardcoded value of NPatch from cime. Currently it's hardcoded to 80, which is more than enough to handle all our cases including future development of particular crop types. The downside of that is efficiency though (both memory and cpu). For a SP case NPatch would only be up to 17 and right now when crop is on (which allows up to 78 types) we only actually use 5 normally as we merge some crop types together (and irrigated and non-irrigated crops can be handled separately as well depending on how the model is run). We also have the capability of fixing the number of patches to a low number down to 1. So a better way to handle this would be for NPatch to be set by CTSM at initialization and sent to cime/cmeps and then to use that value for the rest of the simulation.

The other thing I wonder on reflection is why you only need vegetation types? Why not information on other landunit types? You do have the landunit weights so that tells you the size. I would think there must be information on the landunit types that's currently embedded into GEOS-Chem that covers these details. It sounds like this also indicates that if we add additional landunits, this code will need to be changed as well. Possibly those details on landunits really should be communicated to GEOS-Chem as well or shared in some fashion? Some of this could be just sent at initialization.

@billsacks was also going to add comments on this, but I wanted to give my notes on both meetings. So go ahead either add a new comment or edit what I have here.

[billsacks]

Thank you for submitting this PR. The details of the coding look good to me, but I have some big-picture comments / questions:

(1) How is NPatch (and NLUse, but that is less important) set in seq_drydep_mod? It seems like this is supposed to be the maximum number of patches per grid cell, but this can differ for different CTSM configurations (crop on vs. off, and whether FATES is active), so we're wondering if it can / should be set dynamically based on the configuration. (You could hard-code it to some value that is always large enough, but then you'll incur an unnecessary communication cost in configurations where it is significantly smaller, which may or may not be important.) (@billsacks this is currently a hardcoded constant set to 80).

(2) It looks like the assumption here is that, in a given grid cell, there is only one patch with any given patch%itype value. This currently is not true for patch%itype == 0 (non-vegetated): I believe that all patches in non-vegetated landunits, in addition to the bare ground patch in the vegetated landunit, all have itype == 0. Even though LAI will be 0 for all of these, it seems like this might be a problem for the pwtgcell variable: this will somewhat arbitrary be set to the patch weight of the last patch on the grid cell that has itype == 0, whichever one that happens to be. Perhaps more importantly, in the near future CTSM will commonly be run with more complex subgrid configurations, including multiple vegetated columns per grid cell with hillslope hydrology and complex vegetation structure with FATES. In these configurations, a given vegetated type could occur in multiple patches in a given grid cell (and, if I remember correctly, with FATES the relationship between patches and vegetated type is completely different than in the current big-leaf version of CTSM). I can't tell if what you really need are the LAI values for every patch (corresponding to the patch weights on the gridcell), the LAI values for each vegetation type (perhaps averaged over all patches of that type within a given grid cell), or both. In any case, I think the current implementation will need to be reworked somewhat to handle the additional flexibility that will be coming soon.

(3) I'm curious why you need the landunit weights in addition to the patch weights, given that the only subgrid state variable you pass (LAI) is at the patch level. If you need them, it's fine (perhaps this is related to point (2) in some way), but I just want to confirm that you really need this.

[billsacks]

Oops, I was in the midst of writing my comment when @ekluzek 's came through, and I didn't see his until I had already posted mine. We say mostly the same thing but in different ways.

[lizziel]

Thanks all for the detailed notes.

• Collaborator access for @ekluzek is now granted
• I will work on bringing the CTSM changes into the relevant nuopc file (UPDATE: done)
• I am also concurrently working on CIME/CMEPS updates needed to test the CTSM changes
• I will keep the changes in mct in this PR for now. However, if I deem the changes needed in CIME too complicated or extensive I may just remove them here to avoid the hassle of updating something slated for retirement
• @fritz, could you weigh in on the implementation questions above from @ekluzek and @billsacks? You may have ready answers about what guided the design decisions.

[fritzt]

Hello! Let me respond to the implementation questions as listed out by @billsacks:

(1) I remember setting NPatch and NLUse to the maximum number of patches and land units in all configurations. It would definitely be ideal if these were set from the actual number of patches and land units, passed from CLM. I understand that right now this leads to unnecessary memory and CPU usage, but I wasn't sure how to dynamically set these variables based on the compset.

(2) Thanks for the insight. I wasn't aware of future development of CTSM. I believe that what GEOS-Chem currently needs is LAI for every vegetation type in a given gridcell.

(3) From my initial investigation, I found that passing land units allowed CESM2-GC to identify lakes and deserts. Maybe there is a better way to get this land type information from CTSM?

[mvertens]

@fritzt - with the new nuopc interface - it is very easy to pass all the lai for every vegetation type as a single field with each vegetation type being an ungridded dimension of that field. So this is much easier to implement than in the old cpl7 framework. I am happy to discuss this if it would be helpful.

[ekluzek]

@fritzt thanks for your update:

(1) I remember setting NPatch and NLUse to the maximum number of patches and land units in all configurations. It would definitely be ideal if these were set from the actual number of patches and land units, passed from CLM. I understand that right now this leads to unnecessary memory and CPU usage, but I wasn't sure how to dynamically set these variables based on the compset.

Yes, I think we should discuss this further for the implementation in NUOPC.

(2) Thanks for the insight. I wasn't aware of future development of CTSM. I believe that what GEOS-Chem currently needs is LAI for every vegetation type in a given gridcell.

Could you point us to where the calculation that uses LAI in GEOS-Chem is done?

(3) From my initial investigation, I found that passing land units allowed CESM2-GC to identify lakes and deserts. Maybe there is a better way to get this land type information from CTSM?

Yes, I think we should compile what you actually need and send only it rather than all landunits. The ordering of landunits could change, and things like having shallow and deep lakes can be done. So what we send to GEOS-Chem should be the summed fields of what you actually need. What do you do for deposition over land-units like urban or glacier or wetland? We also should pass things in such a way that we make sure you handle all land-units. This will require putting an interface in CTSM to do this, but this way when CTSM changes it's part of the CTSM code and change it all together.

[fritzt]

Hi @ekluzek!

The LAI are stored in the GEOS-Chem State_Met structure, defined in here, while being used in the dry deposition module (here).

For my implementation of CESM2-GC, I match each CLM land type with a corresponding Olson land type. The GEOS-Chem Wiki provides a lot of documentation on how these are defined.

[billsacks]

Thanks for the additional pointers, @fritzt . I just looked quickly through the corresponding CAM PR. It looks to me like the key here is the getLandTypes module in https://github.com/ESCOMP/CAM/pull/378/files . I'm concerned about the hard-coding of indices here, which would prevent a more flexible use of type indices in CTSM. My initial thinking – which I think is similar to what @ekluzek is suggesting – is that we want the translation from CTSM land cover types to Olson land cover types to be handled in CTSM, not CAM. An alternative would be for CTSM to send values in its native types (as is done in this PR) but also send metadata describing the meaning of each index, and then the CAM code would use that metadata. But that alternative seems significantly more complex, and I think is only worthwhile if different atmosphere schemes want to slice and dice the landcover types in different ways.

This change would be in addition to the change mentioned above to handle the possibility that a given landcover type can appear more than once in a single grid cell.

I would be happy to join a call to discuss this further.

[lizziel]

I'm sorting through the new as I get acquainted with Thibaud's code. For easy reference I'm going to document the relevant places in the code below in case anyone else wants to easily reference it.

The LAI code in GEOS-Chem is a bit confusing due to maintaining backwards compatibility with old dry deposition code. The LAI field used in GEOS-Chem dry deposition is State_Met%XLAI, not the field State_Met%LAI. XLAI for a given surface type is the average LAI for only the area in the grid cell with that surface type. XLAI is calculated from the LAI values stored in State_Met%XLAI_Native divided by land type fraction (State_Met%LandTypeFrac) as shown in geos-chem/GeosCore/modis_lai_mod.F90.

If I am understanding this correctly, State_Met%XLAI and State_Met%LandTypeFrac are set to the CAM imports in the proposed CAM updates with mapping hard-coded in CAM/src/chemistry/geoschem/getLandTypes.F90 (clm40 highlighted only).

That is done only if using online, and there is a separate assignment if using offline instead. The logic for that is in the GEOS-Chem chemistry driver, CAM/src/chemistry/geoschem/chemistry.F90, prior to the call to compute XLAI for use in dry deposition.

[lizziel]

I am closing this PR as we are now going in a different direction for GEOS-Chem dry deposition in CESM. GEOS-Chem will receive only dry deposition velocities directly from CESM, and will use the CLM/CTSM dry deposition velocity calculation codes. I will be working on this and will create a new PR for in the future.

[ekluzek]

Thanks for your work on this @lizziel. I think this will be a better way to go for future development. But we appreciate the work that you did in evaluating this and working with us on it. And we appreciate the work that @fritzt did to get this configuration working as well. It can be helpful to have a working prototype to compare and contrast to evaluate the best ways forward. Looking forward to working with you and the GEOS-Chem community on future work.

Note, I often delete branches after they are closed, but I'll let you decide if you want to do that.