bring soil NOx scheme into CTSM

[mvalmartin]

We would like to bring a soil NOx parameterization into CTSM. This issue is requested by Louisa Emmons, Anthony Wong and @mvalmartin. The code is in Cheyenne, implemented in CESM2.1 with release-clm5.0.25.
May you please advise on what is needed by the CTSM team at this stage?
Thanks!
Maria

[wwieder]

It would be helpful to have some idea what approach you've taken for this. Seems like you have a paper in prep, which is helpful (and I need to look at!). Beyond a broader scientific description of what you're trying to do (which you can add to this issue).

You should also open up a pull request to bring the code to the main development branch of CTSM. It would be most productive if your team can work on bringing your code up to the main CTSM5.1 dev118 code base. We the LMWG software team, can help advise on this, but likely don't have a lot of time to spend on this in the short term. Having a planning meeting to help scope this out would be productive once you open a PR.

Finally, what is your timeline for having this code integrated into CTSM, specifically, is CAM-Chem hoping to have this for CESM3?

[mvalmartin]

Hi Will, Thanks for getting back to me!

The idea is to bring the soil NOx scheme into ctsm to use it with CAM-chem and have it ready for CESM3. I'm tagging Louisa Emmons (@lkemmons) here as she can follow-up about that too.

We could do it in a more simple way that in my GMD draft manuscript, eg independently of soil pH, that is, using the fixed soil pH of 6.5 and leaving soil NH3 out. It is not a massive implementation if we leave NH3 and all the weathering modules out. The scheme follows DayCent (Parton et al 2001) and uses the current N2O denitrification and nitrification schemes.

I don't think that adding the code in CTSM5.1 dev118 code base will be a massive job, I can try to do it, test and evaluate it. Now, the only catch is the coupling (land to atm). I remember trying to couple the scheme in cesm2.2 and gave up as the structure had substantially changed from cesm2.1 (where I had already coupled the soil NOx and NH3 from clm5 to cam-chem). Could the LMWG software team advise on that?

Thanks again!

Maria

[ekluzek]

@mvalmartin yes we should talk about sending this through CMEPS from CAM to CTSM. I assume this would be something would be turned on some of the time -- but not ALL of the time right?

We should probably add it as a namelist to drv_flds_in with at least a logical variable to say it's turned on or not. That can be seen by both CTSM and CAM to know it's being passed.

[mvalmartin]

Yes! What we need is to pass soil NOx from CTSM to CAM. I already have a namelist in my version, with a switch (use_soil_emis=true or false). However, I couldn't manage to implement it in cesm2.2, hence I am still working with cesm2.1 and clm5.0.25.

I will start transfering the soil NOx code into ctsm5.1.

When is the code going to be frozen for cesm3? That will help me get organized here!

Thanks!

[ekluzek]

@mvalmartin there's not a fixed date for CESM3 to be frozen yet. But, the first coupled experiments with updated components are ongoing now. There's talk of CMIP7 experiments starting mid 2024, and I think the desire is for most updates for CESM3 to be in the models by this Summer. As such we'd like to see a PR for this right away if possible. You can start a branch and get the PR started as a draft, but describe it as a Work In Progress. It helps for us to start to see the changes and know the branch exists and can be brought in. That also helps in us reviewing the code and working with you on changes needed.

This is more than you need but these are slides from Gokhan about the current planning on all of this.
cesm_cochairs_20230228.pptx

[wwieder]

Thanks for posting on this, @ekluzek . To confirm, our current plan is to start CMIP7 simulations with CESM3 in the summer of 2024. This timeline is highly dependent on external forcing data from the CMIP community as well as a stable model code that WGs are satisfied with, but it's the most firm info we've had to date.

[mvalmartin]

Thanks @wwieder and @ekluzek for getting back to me. I should be able to transfer the soil NOx scheme into the CTSM5.1 in the next weeks. (now that I submitted the manuscript!).

[mvalmartin]

Hello @wwieder and @ekluzek, I have sometime this week to do the soil NOx implementation and was wondering if the the main base code is still CTSM5.1 dev118. The latest available in the github is ctsm5.1.dev128.
Thanks!
Maria

[wwieder]

Dev128 is the latest. Should we expect or on this before long?-- Will Wieder (he/his) Project Scientist & Land Model Working Group Co-Chair CGD, NCAR 303-497-1352

[ekluzek]

@mvalmartin yes the code base keeps marching forward. So you might as well start with the very latest version at the time you bring it in, so ctsm5.1.dev128 right now. If you get it into a branch it's not usually too hard to keep the branch up to date with the latest (although how easy depends both on the changes coming in and how complex your branch is).

[mvalmartin]

Hello, I finished implementing the soil NOx code in ctsm5.1.dev128. I started a test run and noticed that it takes 10 times more to run ctsm5.1.dev128 than clm5.0.25. I checked the timing without my implementations too.

Overall Metrics: (ctsm5.1.dev128)
Model Cost: 2394.82 pe-hrs/simulated_year
Model Throughput: 18.40 simulated_years/day

Overall Metrics: (clm5.0.25)
Model Cost: 274.58 pe-hrs/simulated_year
Model Throughput: 160.48 simulated_years/day

That is quite of an increase from one version to another and I wanted to double check if that is ok. The timings in ctsm5.1.dev128 are similar with the soil NOx code in.

Thanks!

Maria

[ekluzek]

@mvalmartin can you point us to the two cases? It depends on all the particulars of the cases. I assume the older version was running clm5_0, and the newer clm5_1 physics, and we expect it to be more expensive, but not 10X. It'll also depend on the number of processors given to each, and details of your output.

[mvalmartin]

@ekluzek the cases are
For ctsm5.1.dev128 -->/glade/work/mariavm/cases/i.ctsm5.orig.000
For clm5.0.25 --> /glade/work/mariavm/cases/i.clm5.Cont.104
You are right about the processors. I'm running ctsm5.1.dev128 'out of the box' so I may need to update the env_mach_pes.xml, which I did back in the days for clm5.0.25.
Thanks!
Maria

[samsrabin]

Just wanted to add that this paper is now published in GMD.

[wwieder]

Thanks for highlighting this @samsrabin. @mvalmartin do you think you'll open up a PR for this work?

[mvalmartin]

Hi, apologies for the slow response! The implementation is ready in ctsm5.1.dev128. I added soil NOx and the missing N from SOM turn over term in nitrification, as in Parton et al 2001 Equation 2, following Nevison et al., (2022a) doi:10.1002/eap.2530.

The scheme uses the default, fixed soil pH value in CTSM of 6.5. I haven't coupled soil NO to CAM.

Please let me know if you need further information or if anything else is needed. I guess I just need now to figure out how to open a PR for that! Github and I are not exactly in good terms :)

Thanks a lot!

Maria

@lkemmons

[wwieder]

Thanks Maria,

At this point we're pressed for SE resources to bring in new science, but will do our best. It will help that your code base is pretty current.

How critical is this work for CAM-CHEM priorities for CESM3?

[mvalmartin]

Perfect, I'll start opening an PR then and get back to you if i don't manage it. My understanding is that the CAM-chem folks wanted to have soil NOx for CESM3. Luisa (@lkemmons) may you please correct me if I'm mistaken? Thanks!

[lkemmons]

Yes, Simone @tilmes and I agree that having this in CESM3 will be extremely valuable. Thanks.

[mvalmartin]

Hi,

I'm trying to open a PR but consistently failing. I'm not very familiar with GitHub, and after watching several tutorials online, and the "Quck start to CTSM development with Git" I'm encountering errors when trying to start a pull request!

The code is in Glade, /glade/work/mariavm/ctsm_soilnox. I was able to issue the checkout on my directory and commit my changes using git commit. I also created a fork. However, I can't push the changes to it. I can't see what I'm doing wrong, but obviously, I'm doing something. I got a few warning messages about access rights and someone doing something nasty! :) I'll keep trying but starting the pull request is not as straightforward as I expected, unfortunately.

Maria

[ekluzek]

@mvalmartin there are some wiki pages and CTSM specific tutorial's that I could point you to. But, if you've already been watching github tutorials and still have problems, I think it would be more productive to work with you directly to see what's going on and to help.

I think you are likely trying to push to the master branch or a branch on the ESCOMP fork, rather than a branch on your fork. From looking at your clone, you are clearly on your own branch. Looking at your remotes it looks like that's the issue. You have a remote that points to a new repository under your github account, and the others besides escomp aren't functional.

So let's figure out a time to meet. Let me know what would work for you. I'll also add the CTSM specific helps that will help you out.

By, the way, git is a very complex tool with a difficult to understand conceptual model. It's also a critically important tool that we rely on. But, I totally understand having problems with it. I'm not going to just say "Oh it's easy you just _____". The available options to git are complex and different repositories work differently, so everyone needs some guidance here....

[ekluzek]

Some help:

https://github.com/ESCOMP/CTSM/wiki/Tutorials (This is a tutorial specific to CTSM and includes videos)

https://github.com/ESCOMP/CTSM/wiki/Quick-start-to-CTSM-development-with-git
https://github.com/ESCOMP/CTSM/wiki/Recommended-git-setup

[mvalmartin]

Hi @ekluzek, Thanks a lot! That's the video tutorial I was looking for yesterday but couldn't find. I knew there was a recording. Apologies for missing the wiki/Tutorial site. I'll watch the video and see if I can make the push to the remote site work. I'm based in the UK, with a 7-hour difference, so in terms of meetings, perhaps an early meeting for you and an afternoon time for me may work. Your 11:30 am (Wednesday) and 8:30 am (Thursday) work for me this week. Thanks a lot!

[ekluzek]

Yeah, hopefully those links help. In terms of meeting this week, I have meetings that conflict both this Wednesday and next. Thursday and Friday this week is a US holiday (Thanksgiving), so that doesn't work. I could make an 8:30am if needed. So what are some times for next week?

[mvalmartin]

I forgot about Thanksgiving! I'll look at the recording in the meanwhile. Nov 30th (Thursday) at your 8:30 am works for me. I hope to have that figure out before so! :) If it works for you, I'll send you a calendar invitation with a google meet link to block our calendars, we can always move or cancel it. Enjoy the US holiday! Thanks a lot.

[lkemmons]

@ekluzek I will try to move this forward with a PR. Maria's current code is in: /glade/work/mariavm/derecho_ctsm_soilnox/
I don't know what version this is in. Should I move her new code into a newer version?