Always send Nitrogen-deposition to surface components

[ekluzek]

In standard configurations both CTSM and the ocean model need to have Nitrogen-deposition used. The system is currently configured so that CTSM and POP assume that they have to provide ndep, but in some cases CAM sends ndep down to them (when WACCM is being used) -- and when it does they use it rather than providing it themselves. However, at the script level they don't know that it's being sent down so normally will abort if they don't have a file to read ndep setup. In the case when CAM IS sending ndep to the surface components they don't need to abort, and we'd like a way for them to know they don't need to abort when setting up their namelists. The other bad thing about the current situation is that the surface components will set control variables for the ndep file -- that won't actually be used. In this setup they are dummy variables that a user might think matters -- but actually doesn't. So it's bad behavior (to abort) and it's bad documentation of the case. The surface components could query the compset -- but since whether ndep is sent is a run-time configurable variable for CAM -- that's NOT a good solution.

The long term best answer to this to ALWAYS send ndep down to the surface components from CAM. When WACCM is on it'll be prognostic, but for other configuration's it'll be prescribed from datasets coming from previous simulations.

Note this relates to: #102

CTSM issue that relates to this: ESCOMP/CTSM#946

[ekluzek]

A short term solution we propose is that an XML variable be set to signal that ndep is being sent from CAM. I propose something like:

ATM_SENDING_NDEP=TRUE

I propose using "ATM" rather than "CAM", because this same variable could be used by DATM to signal that it's sending ndep.

[ekluzek]

A longer term solution would be for CAM to ALWAYS send ndep no matter what configuration it's running. This likely would be to adopt the calls to shr_streams to read in the same data that CTSM has been using. This would also require that datm do the same thing. If something like ATM_SENDING_NDEP XML variable was added it could be used while we were in transition.

The philosophy behind this is that inherently Nitrogen-deposition is a physical flux from the atmosphere, so the atmosphere component should be the one that handles it -- rather than surface components.

[gold2718]

At the level where CAM sends ndep to the surface models, it is merely responding to what in in the driver namelist. That is, CAM has no idea that it is sending ndep until runtime. Therefore, I believe that the simplest solution is for the surface models to imitate what CAM does. The first step is to determine whether the nitrogen species are in the MCT attribute vector (cam_cpl_indices.F90):

    index_a2x_Faxa_nhx      = mct_avect_indexra(a2x,'Faxa_nhx',perrWith='quiet')
    index_a2x_Faxa_noy      = mct_avect_indexra(a2x,'Faxa_noy',perrWith='quiet')

Then, the model can make decisions on the result. For instance (from atm_import_export.F90):

          if (index_a2x_Faxa_nhx > 0 ) then
             !! Expect nhx from atmosphere model
          else
             !! Surface model needs to read nhx
          end if

I do not see how to implement @ekluzek's recommendation because the information is in the driver namelist and CAM does not read that.

[ekluzek]

@gold2718 I don't think that's quite true. The thing is that CAM is what SETS the drv_flds_in namelist. An awkward aspect of drv_flds_in is that both CAM and CLM can set things in it -- and then the coupler puts them together.

The suggestion you have for the code IS what we are doing inside of CTSM. But, CTSM is stopping in preview_namelists because it doesn't know it doesn't need to set the ndep file.

Does that explain it better?

[gold2718]

@ekluzek, yes, I understand that this property is added to CAM's version of drv_flds_in (line 3955 in build_namelist). CLM also does this. Then, the driver compares and combines these files (in write_drv_flds_in_file of buildnml) to produce the run-time versions.
So, I maintain that, before CAM's build-namelist is run (via buildnml), CAM has no idea if those fields will be included. Since we cannot have any order dependence on which component's buildnml goes first, I still think that the only way forward in the short term is for the surface models to decide at run time whether or not to read an emissions file (the else clause above).

Since CLM needs a filename if is to read a file (duh), I think that CLM's build-namelist will always have to specify one -- even if it may not be used.

[ekluzek]

OK, so I think what you are saying is that the conditions for which CAM is going to decide it's going to send ndep to the coupler are complex enough that we can't know it beforehand. So we have to treat it as a black-box and just see the end result, rather than being able to predict it beforehand. If we treat it as a blackbox there then isn't a way to signal to anything else that it's going to be sent.

As you point out, the only way around it would be for CTSM check on the drv_flds_in file from CAM, but that requires that CAM's buildnml goes first. Relying on order dependence would be fragile and would break anytime the order changed. The order has been that CAM goes first right now -- but we can't robustly depend on that order.

So at this point we don't have a proposal for a short term solution. The only workable solution is the long term one, of having CAM always send ndep in some form (either calculated or from data files).

[ekluzek]

And since there is no short term solution, that does tell me that the right solution is the long term one of ndep always coming from CAM. Otherwise we are stuck with the system having this odd behavior of aborting when it doesn't really need to as well as listing files that it doesn't really need to read.

So what are the chances of the long term solution happening? The good thing about it would be that it could just adopt the same stream reading that CTSM is currently using which is only a few hundred lines of code.

[gold2718]

OK, so I think what you are saying is that the conditions for which CAM is going to decide it's going to send ndep to the coupler are complex enough that we can't know it beforehand.

No, that is not what I am saying. What I am saying is that a user can insert or remove these items from user_nl_cam just before submitting a run, i.e., there is no way for CAM to know what the user did before build-namelist is called.

So at this point we don't have a proposal for a short term solution.

But I did make a proposal for a short-term solution! My first post in this conversation has a specific, actionable proposal. If this solution is not viable (e.g., because build-namelist cannot always configure in an ndep file as a backup), please address that.

[ekluzek]

@gold2718 OK one question and one restatement then.

First, if someone sets ndep_list to send nhx and noy in their user_nl_cam (for a case without chemistry) -- will it abort and tell the user it can't do that? Or will it just send zero's or NaNs? If so that is something that you might want to address in CAM. Note that CTSM doesn't check the values that's it's being sent from CAM. So it would happily run with zero's -- but that would affect the simulation. It would abort with NaN's though. So if it's sending NaN's I think that would be an OK way to operate.

Second, the restatement. As I had said we have adopted what you suggested. CTSM checks the nhx and noy indices and if set -- it sets the logical variable ndep_from_cpl to true. If true it does NOT read in the ndep file, but uses what's sent from the coupler. If false it DOES read the ndep file that was set on the namelist and uses it. So we've already implemented everything I see in your suggestion, right? Is there anything that I've missed?

[gold2718]

@ekluzek, responding to your second piece, if you have implemented everything I suggested, why is there still a problem? In the opening of the issue, you said:

However, at the script level they don't know that it's being sent down so normally will abort if they don't have a file to read ndep setup.

My suggestion includes CLM (and POP?) having a default ndep file that is always configured. If that is not currently the case, can it be added? If not, can the code set the fluxes to zero if there is no file and the data is not coming from the coupler?

[gold2718]

@ekluzek, to address your second point, yes, the nitrogen quantities are initialized to zero. If chemistry is not running, those quantities are not modified so the surface should receive zero.
While that may not result in a valid simulation, we cannot control everything the user is doing.

In the long term, the plan is to solve issues such as this with a combination CCPP analysis and run-time NUOPC negotiation, however, this is a CESM3 target.

We may be able to implement your 'longer term solution' in CESM2, however, that will have to be scheduled and prioritized in AMP.

[fvitt]

It appears that if ndep_list is set and the atm chemistry does not include the required nitrogen species (or no chemistry) then the nitrogen fluxes sent to the cplr will be zeros.

[ekluzek]

@gold2718 that's something we explicitly decided that we would NOT do. I actually originally configured CTSM so that it would use the closest ndep file it knew about. But, we decided that's problematic -- because it's silently doing the wrong thing and not telling you about it. So, for example, you ask for SSP5-3.4 and under the covers it gives you the ndep file for SSP5-8.5. Now maybe it tells you that in log files or the namelist -- but if it doesn't do something drastic (like abort) it's easy to miss. I could also configure it so that it's given a dummy file that it knows it should never read (call it something like never_read_this_ndep_file.nc). In that case it would abort if it's trying to read it and ndep_from_cpl==F. But, then it's aborting when the model is running rather than being warned before hand. A good goal in behavior is for the model to let you know something's wrong as soon as it can be detected. So it's best if it tells you at case setup or preview_namelist time.

[ekluzek]

@gold2718 and @fvitt thanks for clarifying the behavior. Possibly CTSM should also check the values then and if they are all identically zero we should abort.

[ekluzek]

And @gold2718 thanks for working on scheduling the long term solution. I know this isn't likely to be a high priority, but it does seem this discussion brings out that it's the right one.

[gold2718]

@ekluzek, What if the user simply wants to see the difference in running without nitrogen fluxes and uses user_nl_cam as a way to do that? By aborting, you prevent that experiment.
Also, in place of a default ndep file, you could just set the fluxes to zero if there is no file and the data is not coming from the coupler (as I suggested above).

If TSS decides that running CLM with zero nitrogen fluxes is always an invalid experiment, then go ahead and abort. However, please realize that if (when?) we implement your proposed long-term solution, there will be configurations where CAM still sends CLM zero fluxes through the coupler!. How will you check for that?

[ekluzek]

@gold2718 good comments. I'm checking with the CTSM people on how bad zero ndep is to CTSM (and if a single point being zero is a possibility). Note, if you want to turn nitrogen fluxes off, you could setup a ndep file that CTSM reads that's identically zero. You'd potentially have to both set it in user_nl_clm AND turn it off in user_nl_cam, but you could do that (we currently don't check the values being read from the ndep file).

I think the point isn't that zero fluxes is an invalid experiment -- it's that getting identically zero fluxes from the CPL is the way for CTSM to know it's NOT getting valid data from CAM. So I think that, short or long term, checking that if all of the nhx and noy array sent from the cpl is zeroed -- CTSM aborting is the right thing to do. Now, then someone has to figure out why CAM is sending zero ndep to CTSM, but it at least lets you know by aborting there is something strange about this configuration. So your argument convinces me even more that we need to check on ndep sent from CAM -- unless I'm missing something.

[gold2718]

So your argument convinces me even more that we need to check on ndep sent from CAM ...

As I said, I think this is a science decision for TSS. Is setting the nitrogen flux to zero an invalid CLM experiment? If so, go ahead and check, however, CAM does allow that setting.

If zero flux is invalid for CLM, it leaves us with a much more complicated (and therefore less likely to be implemented) long-term fix since CAM will somehow have to know it is dealing with CLM and not some other configuration (e.g., an aquaplanet).

[fvitt]

In my opinion, if ndep_list is set then CTSM should assume CAM is sending valid fluxes rather they are zeros or not. If CAM is not configured to send valid nitrogen fluxes to the coupler then CAM will not set ndep_list

[brian-eaton]

I agree with Francis regarding use of ndep_list.

This entire discussion is a repeat of the one we had years ago with respect to the dust and carbon deposition fluxes, as Dave mentioned in the meeting yesterday. I believe the right solution then and the right solution now is that CAM should be responsible for providing atmospheric fluxes.

Short term solution: CLM should always provide a default dataset until CAM is set up to always provide these fluxes. I think Erik's argument above for not doing this is based on a false assumption that the default could be wrong. No one is suggesting that a single default dataset is appropriate. That's what use case files are for. The use case file for SSP5-3.4 won't contain the same dataset as the file for SSP5-8.5. That's how CAM deals with providing the correct prescribed carbon and dust deposition, and all sorts of other emissions.

Long term solution: CAM should always provide this data. I don't know that adopting shr_streams code to deal with the prescribed data is the best solution. Maybe it is. But CAM provides lots of prescribed data (like dust and carbon deposition) without the use of the shr_streams code, and I imagine that extending what we already do would not be a huge task.

[cacraigucar]

Reviewing this at a cursory level. Is this an issue we need to keep open?

[ekluzek]

@cacraigucar I reviewed the above discussion. This issue served to provide a discussion on both a short term and long term solution. We nixed the idea of a short term solution -- but agreed the long term solution is to have CAM always send ndep to surface components. So we should either create a new issue that's just about that long term solution -- or change the title of this one. Which of those two should we do (rename this issue or create a new long term one)?

@gold2718 @brian-eaton @fvitt does that sound correct to you?

[brian-eaton]

I wasn't in on the discussion not to have CLM provide correct default datasets in the short term. But I agree that CAM should always be responsible for providing this data in the longer term. I also agree the issue should remain open since it hasn't been resolved. I don't feel strongly one way or the other regarding changing the name of the issue vs creating a new one (as long as the new one refers back to the original).

[cacraigucar]

I assume that this is not something for the upcoming CESM2.2 release, but rather is a longer-term issue?

…

On Tue, May 26, 2020 at 5:05 PM Brian Eaton ***@***.***> wrote: I wasn't in on the discussion not to have CLM provide correct default datasets in the short term. But I agree that CAM should always be responsible for providing this data in the longer term. I also agree the issue should remain open since it hasn't been resolved. I don't feel strongly one way or the other regarding changing the name of the issue vs creating a new one (as long as the new one refers back to the original). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRX2XZ6KX5YZVGXFJEMB5DRTRDMVANCNFSM4LTAKVSQ> .

-- Cheryl Craig cacraig@ucar.edu NCAR Voice: (303) 497-2901 P.O. Box 3000 Fax: (303) 497-1324 Boulder, CO 80307

[ekluzek]

Yes, exactly, this won't be for CESM2.2.0, but a longer term issue. @cacraigucar should I just rename this one, or open a new one? Which do you prefer?

[cacraigucar]

Why don’t you just rename it. That way we keep the discussion all in one issue.

On Tue, May 26, 2020 at 5:27 PM Erik Kluzek ***@***.***> wrote: Yes, exactly, this won't be for CESM2.2.0, but a longer term issue. @cacraigucar <https://github.com/cacraigucar> should I just rename this one, or open a new one? Which do you prefer? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRX2XZ3PAGFW2JIJNPFDM3RTRF6XANCNFSM4LTAKVSQ> .

-- Cheryl Craig cacraig@ucar.edu NCAR Voice: (303) 497-2901 P.O. Box 3000 Fax: (303) 497-1324 Boulder, CO 80307

[ekluzek]

For reference the CIME issue that makes datm send ndep to surface components is here: ESMCI/cime#3549 This doesn't really matter as far as CAM is concerned, but the idea that CIME ATM components are required to send ndep to surface components does outline expected behavior for CAM.

[mvertens]

With the CDEPS functionality to read in stream data directly via function alls to the stream share code, I think the right solution is to have the CAM nuopc cap read in ndep from the relevant stream file if it is not going to calculate it internally. I am happy to implement this.

[ekluzek]

Note a group of us talked about this and other similar issues yesterday. @fvitt @mvertens

See https://docs.google.com/document/d/1iVOSLyDeO5-U0vxHV7CeT4lRSjLL4Klx5Or7kwothAU

[ekluzek]

It sounds like @mvertens got a start at this, but there's more work that needs to happen with it.

See the latest updates here...

ESCOMP/CTSM#1844