Dynamics tendencies

[ericaligo-NOAA]

Following the instructions here: https://ccpp-techdoc.readthedocs.io/en/latest/ParamSpecificOutput.html
I was able to output and plot various physics and dynamics tendencies. I noticed that the x and y wind tendencies from the dynamics have reasonable values until the first output time after my restart. I see values such as Contouring: -555556 to -555556 interval 0.01. I have hourly output until my restart time when I have output every 3 minutes. My fhzero is set to 3 hours. All other tendencies seem okay. Is this a bug?

[jkbk2004]

@grantfirl @dustinswales I am not sure if you can suggest an idea to isolate this contouring issue of dynamics tendency.

[grantfirl]

@ericaligo-NOAA The x and y wind dynamics tendencies are calculated in the FV3/atmos_model.F90/update_atmos_radiation_physics subroutine. They are calculated between the timestep_init and run phase of the radiation group and are not touched by the CCPP at all. These are so-called "non-physics" tendencies.

As far as I can tell, these variables are part of the Diag DDT in GFS_typedefs and are not needed for restart, so I'm guessing this has something to do with what you're seeing. I'm guessing it has something to do with the values of statein%ugrs/vgrs and stateout%gu0/gv0 for the first time update_atmos_radiation_physics is called after a restart. From what I can tell, the array "dtend" holding the x and y wind dynamics tendencies is properly initialized to zero before the first call, so my guess is for this calculation, one of ugrs/vgrs or ug0/vg0 is not set to something meaningful yet.

@SamuelTrahanNOAA would be the best expert to help debug what you're seeing since he was intimately involved with the coding of the tendencies.

[grantfirl]

It's weird that you're also NOT seeing weird values of the non-physics tendencies of T and q since they're handled identically as far as I can tell.

[ericaligo-NOAA]

Exactly. The temperature, qv and o3 'non-physics' tendencies look fine as shown by the profiles on the top row here.

[gspetro-NOAA]

Note: There is an open issue in CCPP physics to look into this problem, but
@dustinswales and @grantfirl were thinking the issue is likely somewhere above the physics, possibly related to restart.
@ericaligo-NOAA If you have not been able to troubleshoot this successfully with help from @SamuelTrahanNOAA or others, I can convert this discussion into an issue. If the problem is solved, I'd like to close the discussion as resolved. Please let me know how to proceed.

[SamuelTrahanNOAA]

If something isn't working properly, it should be a bug report issue. Eric's hypothesis that uninitialized variables are behind this is a good one. We need a self-contained test case so others can reproduce this bug by compiling the code and submitting a test script. (No fifty step process or workflows.) Through the magic of print and write statements, we'll be able to trace where the bad data originates.

[gspetro-NOAA]

I have opened a bug report and transferred the information from this discussion to Issue #2222.