Fixed the warm bias in stratosphere from MERRA2 coupled microphysics (branch Maie) #1815

AnningCheng-NOAA · 2023-06-30T02:50:22Z

Description

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics replacing PR 1788 closed accidently and addressing Isue 1786. It will modified the results from merra2_thompson and atmaero_control_p8_rad_micro whenever mraerosol=.true. No library updates.

Includes entire PR #1838 (changes one line from compile.sh), and code change from PR #1794 (one line from detect_machine.sh).

Input data additions/changes

No changes are expected to input data.
Changes are expected to input data:
- New input data.
- Updated input data.

Anticipated changes to regression tests:

No changes are expected to any regression test.
Changes are expected to the following tests: merra2_thompson and atmaero_control_p8_rad_micro

RegressionTests_hera.log

Subcomponents involved:

Library Updates/Changes

Not Needed
Create separate issue in JCSDA/spack-stack asking for update to library. Include library name, library version.
Add issue link from JCSDA/spack-stack following this item

Combined with PR's (If Applicable):

Commit Queue Checklist:

Link PR's from all sub-components involved in section below
Confirm reviews completed in ALL sub-component PR's
Add all appropriate labels to this PR.
Run full RT suite on either Hera/Cheyenne AND attach log to a PR comment.
Add list of any failed regression tests to "Anticipated changes to regression tests" section.

Linked PR's and Issues:

Testing Day Checklist:

This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR.
Move new/updated input data on RDHPCS Hera and propagate input data changes to all supported systems.

Testing Log (for CM's):

RDHPCS
- Hera
- Orion
- Jet
- Gaea
- Cheyenne
WCOSS2
- Dogwood/Cactus
- Acorn
CI
- Completed
opnReqTest
- N/A
- Log attached to comment

…sol=T

AnningCheng-NOAA · 2023-06-30T03:05:19Z

All RTs completed successfully as expected at /scratch1/NCEPDEV/global/Anning.Cheng/tmp1/ufs-weather-model/tests. log files for the two tests needing new baselines attached.
run_176_atmaero_control_p8_rad_micro_intel.log
run_051_merra2_thompson_intel.log

.gitmodules

jkbk2004 · 2023-06-30T12:59:48Z

@AnningCheng-NOAA Can you add/link dependency PR and Issues in the PR description? Complete PR description is always helpful.

AnningCheng-NOAA · 2023-06-30T13:06:39Z

Denise: I think it is suggested by Jong to point CMEPS to ***@***.*** <NOAA-EMC/CMEPS@cec8db8>

…

On Fri, Jun 30, 2023 at 8:55 AM Denise Worthen ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In .gitmodules <#1815 (comment)> : > @@ -16,8 +16,8 @@ branch = develop [submodule "CMEPS"] path = CMEPS-interface/CMEPS - url = https://github.com/NOAA-EMC/CMEPS - branch = emc/develop + url = https://github.com/AnningCheng-NOAA/CMEPS Why are you pointing to a branch of CMEPS? — Reply to this email directly, view it on GitHub <#1815 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQPMINUGKLRHYXZS5MU3WTXN3EDPANCNFSM6AAAAAAZZIRM4E> . You are receiving this because you authored the thread.Message ID: ***@***.***>

jkbk2004 · 2023-06-30T13:10:56Z

Denise: I think it is suggested by Jong to point CMEPS to @.*** <NOAA-EMC/CMEPS@cec8db8>
…
On Fri, Jun 30, 2023 at 8:55 AM Denise Worthen @.> wrote: @.* commented on this pull request. ------------------------------ In .gitmodules <#1815 (comment)> : > @@ -16,8 +16,8 @@ branch = develop [submodule "CMEPS"] path = CMEPS-interface/CMEPS - url = https://github.com/NOAA-EMC/CMEPS - branch = emc/develop + url = https://github.com/AnningCheng-NOAA/CMEPS Why are you pointing to a branch of CMEPS? — Reply to this email directly, view it on GitHub <#1815 (review)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQPMINUGKLRHYXZS5MU3WTXN3EDPANCNFSM6AAAAAAZZIRM4E . You are receiving this because you authored the thread.Message ID: @.***>

CMEPS hash is correct in your fork branch. So, you don't any change for cmeps in gitmodules and only need fv3 changes. You can revert lines changed for CMEPS in gitmodules.

AnningCheng-NOAA · 2023-06-30T13:15:38Z

@jkbk2004 done

DeniseWorthen · 2023-06-30T15:16:15Z

@AnningCheng-NOAA The RT test log you posted attached to the PR request shows many, many failed tests. If you have a RT test log that shows only two failed cases, you need to provide it.

grantfirl · 2023-06-30T15:16:19Z

@DeniseWorthen @jkbk2004 @AnningCheng-NOAA I recreated PRs for fv3atm (NOAA-EMC/fv3atm#674) and ccpp-physics (ufs-community/ccpp-physics#88) that were accidentally closed. Should I recreate this one too, or should we ask @AnningCheng-NOAA just to point to NOAA-EMC/fv3atm#674?

grantfirl · 2023-06-30T15:19:57Z

@AnningCheng-NOAA The RT test log you posted shows many, many failed tests.

Agreed. The hera/intel RT log file is not consistent with the claim of changing the baselines for just those 2 tests. I can re-run them using my branches that I created for the re-opened PRs if you'd like.

DeniseWorthen · 2023-06-30T15:26:11Z

@grantfirl I've updated the linked the updated PRs in the PR information, thanks. I think this PR is already the new, replacement PR for #1788

AnningCheng-NOAA · 2023-06-30T15:27:02Z

Yes, please! I will rerun the cases again. My RTs are at /scratch1/NCEPDEV/global/Anning.Cheng/tmp1/ufs-weather-model/tests

…

On Fri, Jun 30, 2023 at 11:20 AM Grant Firl ***@***.***> wrote: @AnningCheng-NOAA <https://github.com/AnningCheng-NOAA> The RT test log you posted shows many, many failed tests. Agreed. The hera/intel RT log file is not consistent with the claim of changing the baselines for just those 2 tests. I can re-run them using my branches that I created for the re-opened PRs if you'd like. — Reply to this email directly, view it on GitHub <#1815 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQPMIKCNZDULJITDPQELC3XN3VCTANCNFSM6AAAAAAZZIRM4E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

grantfirl · 2023-06-30T16:28:46Z

Here is the log from Anning's testing that was added to the repo instead of attached in the comments. It is showing the two failed tests are the expected ones (only).

RegressionTests_hera.log

@AnningCheng-NOAA Please delete the link to the log in this PR description, since that seems to be in error. I'm going to re-run the RTs to double-check as soon as I'm able (I'm currently running RTs on Hera for a different PR).

grantfirl · 2023-06-30T16:36:05Z

Also, @AnningCheng-NOAA we need to point your ufs-weather-model PR branch to the PR branch that I opened for fv3atm. Here is how I would do this:

On your machine where you have your code, cd into the top level ufs-weather-model directory and double-check that you're on your maie branch: git branch.
cd FV3
We need to add my fork as a new remote for you to checkout my PR branch: git remote add grant-fork https://github.com/grantfirl/fv3atm
git fetch grant-fork
git checkout strat_warm_bias_fix_cheng
If this was all successful, cd back up to the ufs-weather-model directory
git add FV3
edit .gitmodules so that
[submodule "FV3"]
path = FV3
url = https://github.com/AnningCheng-NOAA/fv3atm
branch = maie
is replaced with
[submodule "FV3"]
path = FV3
url = https://github.com/grantfirl/fv3atm
branch = strat_warm_bias_fix_cheng
git add .gitmodules
git commit -m "update FV3 submodule pointer and .gitmodules for testing"
git push

AnningCheng-NOAA · 2023-06-30T16:44:43Z

Grant:

I received the following error:

/scratch1/NCEPDEV/global/Anning.Cheng/tmp1/ufs-weather-model/FV3% git add remote grant-fork https://github.com/grantfirl/fv3atm
fatal: pathspec 'remote' did not match any files

grantfirl · 2023-06-30T16:54:36Z

Grant:

I received the following error:

/scratch1/NCEPDEV/global/Anning.Cheng/tmp1/ufs-weather-model/FV3% git add remote grant-fork https://github.com/grantfirl/fv3atm fatal: pathspec 'remote' did not match any files

Sorry, the command should have been:
git remote add grant-fork https://github.com/grantfirl/fv3atm

AnningCheng-NOAA · 2023-06-30T17:13:54Z

Grant, I encounter an error in last step. Please advise.

grantfirl · 2023-06-30T18:07:29Z

Grant, I encounter an error in last step. Please advise.

You'll get this error if your local branch is behind what is on GitHub. When I accidentally commit something locally to a branch that is missing commits from a remote, I typically revert the last local commit git reset HEAD~1, pull in the commits from GitHub that are missing git pull, then redo the new commit and try to push.

It looks like you were able to finish the instructions, though, because commit 79ca754 shows up here and it looks like .gitmodules is pointing to the right place and the submodule pointer for fv3atm is correct. I think we're good to go now.

AnningCheng-NOAA · 2023-06-30T18:12:19Z

Grant and Denise: I cloned my branch and made a new set of RTs at /scratch1/NCEPDEV/climate/Anning.Cheng/tmp/ufs-weather-model/tests. I got the same results as yesterday and RTs are all successful except two. Below is a series of commands that clones my branch, Please let me know if anything wrong git clone -b develop --recursive https://github.com/ufs-community/ufs-weather-model cd ufs-weather-model git submodule update --init --recursive git remote rename origin upstream git remote add origin https://github.com/AnningCheng-NOAA/ufs-weather-model git remote update git checkout maie cd FV3 git submodule update --init --recursive git remote rename origin upstream git remote add origin https://github.com/AnningCheng-NOAA/fv3atm git remote update git checkout maie cd ccpp/physics git submodule update --init --recursive git remote rename origin upstream git remote add origin https://github.com/AnningCheng-NOAA/ccpp-physics git remote update git checkout maie ssh hecflow01 cd /scratch1/NCEPDEV/climate/Anning.Cheng/tmp/ufs-weather-model/tests ./rt.sh -a fv3-cpu -e >output 2>&1 &

…

On Fri, Jun 30, 2023 at 11:20 AM Grant Firl ***@***.***> wrote: @AnningCheng-NOAA <https://github.com/AnningCheng-NOAA> The RT test log you posted shows many, many failed tests. Agreed. The hera/intel RT log file is not consistent with the claim of changing the baselines for just those 2 tests. I can re-run them using my branches that I created for the re-opened PRs if you'd like. — Reply to this email directly, view it on GitHub <#1815 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQPMIKCNZDULJITDPQELC3XN3VCTANCNFSM6AAAAAAZZIRM4E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

AnningCheng-NOAA · 2023-06-30T18:26:40Z

Grant, done

grantfirl · 2023-06-30T20:38:34Z

@AnningCheng-NOAA Looks good. @jkbk2004 I think that we're in good shape to add this to the commit queue after #1775, #1777, and #1797. I'm sure that we'll need to merge in the top of the develop as things get merged, but it's in good shape right now.

I'm running one last "pre-test" RT set on Hera right now to verify @AnningCheng-NOAA 's tests.

on-behalf-of @ufs-community <ecc.platform@noaa.gov>

zach1221 · 2023-07-21T18:41:31Z

Ok, testing is now complete. I'll follow up on the ufs-community/ccpp-physics/pull/88 so we can begin the merging process.

BrianCurtis-NOAA · 2023-07-24T16:50:00Z

fv3 hash: NOAA-EMC/fv3atm@b32981f

grantfirl · 2023-07-24T16:53:33Z

@zach1221 @jkbk2004 @BrianCurtis-NOAA OK, ready for review/merge.

zach1221 · 2023-07-24T16:54:34Z

Thanks @grantfirl ! Sending out final reviews.

zach1221 · 2023-07-24T16:56:25Z

Disregard the assignments I sent out, that was a mistake.

AnningCheng-NOAA added 4 commits June 28, 2023 23:09

establish maie

89705e0

optimized the code and decrease the stratosphere warm bias for mraero…

badbf95

…sol=T

updated and passed the regression tests

8120d2a

all RTs completed successfully

bfca985

AnningCheng-NOAA mentioned this pull request Jun 30, 2023

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics #1788

Closed

36 tasks

DeniseWorthen reviewed Jun 30, 2023

View reviewed changes

.gitmodules Outdated Show resolved Hide resolved

change CMEPS back

eb9fe95

update FV3 submodule pointer and .gitmodules for testing

79ca754

zach1221 added the Baseline Updates Current baselines will be updated. label Jul 17, 2023

jkbk2004 mentioned this pull request Jul 19, 2023

Ignore empty fv3atm output bundle #1825

Merged

39 tasks

jkbk2004 assigned zach1221 Jul 19, 2023

zach1221 added the jet-RT Run AutoRT Jet regression testing label Jul 21, 2023

epic-cicd-jenkins removed the jet-RT Run AutoRT Jet regression testing label Jul 21, 2023

zach1221 and others added 5 commits July 21, 2023 09:04

add orion RT logs: passed

8495f64

Acorn RT Log

332af4c

WCOSS2 RT Log

92dae2c

[AutoRT] jet Job Completed.

779cc73

on-behalf-of @ufs-community <ecc.platform@noaa.gov>

add gaea RT logs: passed

32424f7

zach1221 mentioned this pull request Jul 21, 2023

fixed stratosphere warm bias and code optimization for MERRA2 ufs-community/ccpp-physics#88

Merged

update FV3 submodule pointer and revert .gitmodules

00df71c

zach1221 requested a review from jkbk2004 July 24, 2023 16:54

zach1221 assigned SadeghTabas-NOAA, DeniseWorthen, BrianCurtis-NOAA and zach1221 and unassigned SadeghTabas-NOAA, DeniseWorthen, BrianCurtis-NOAA and zach1221 Jul 24, 2023

zach1221 requested review from DeniseWorthen, SadeghTabas-NOAA and BrianCurtis-NOAA July 24, 2023 16:55

BrianCurtis-NOAA approved these changes Jul 24, 2023

View reviewed changes

SadeghTabas-NOAA approved these changes Jul 24, 2023

View reviewed changes

zach1221 merged commit 4d05445 into ufs-community:develop Jul 24, 2023

grantfirl mentioned this pull request Oct 30, 2023

UFS-dev PR88 NCAR/ufs-weather-model#106

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics (branch Maie) #1815

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics (branch Maie) #1815

AnningCheng-NOAA commented Jun 30, 2023 •

edited by zach1221

Loading

AnningCheng-NOAA commented Jun 30, 2023

jkbk2004 commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

jkbk2004 commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023

DeniseWorthen commented Jun 30, 2023 •

edited

Loading

grantfirl commented Jun 30, 2023

grantfirl commented Jun 30, 2023

DeniseWorthen commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

grantfirl commented Jun 30, 2023

grantfirl commented Jun 30, 2023 •

edited

Loading

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

zach1221 commented Jul 21, 2023 •

edited by BrianCurtis-NOAA

Loading

BrianCurtis-NOAA commented Jul 24, 2023

grantfirl commented Jul 24, 2023 •

edited

Loading

zach1221 commented Jul 24, 2023

zach1221 commented Jul 24, 2023

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics (branch Maie) #1815

Fixed the warm bias in stratosphere from MERRA2 coupled microphysics (branch Maie) #1815

Conversation

AnningCheng-NOAA commented Jun 30, 2023 • edited by zach1221 Loading

Description

Input data additions/changes

Anticipated changes to regression tests:

Subcomponents involved:

Library Updates/Changes

Combined with PR's (If Applicable):

Commit Queue Checklist:

Linked PR's and Issues:

Testing Day Checklist:

Testing Log (for CM's):

AnningCheng-NOAA commented Jun 30, 2023

jkbk2004 commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

jkbk2004 commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023

DeniseWorthen commented Jun 30, 2023 • edited Loading

grantfirl commented Jun 30, 2023

grantfirl commented Jun 30, 2023

DeniseWorthen commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

grantfirl commented Jun 30, 2023

grantfirl commented Jun 30, 2023 • edited Loading

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

AnningCheng-NOAA commented Jun 30, 2023 via email

AnningCheng-NOAA commented Jun 30, 2023

grantfirl commented Jun 30, 2023

zach1221 commented Jul 21, 2023 • edited by BrianCurtis-NOAA Loading

BrianCurtis-NOAA commented Jul 24, 2023

grantfirl commented Jul 24, 2023 • edited Loading

zach1221 commented Jul 24, 2023

zach1221 commented Jul 24, 2023

AnningCheng-NOAA commented Jun 30, 2023 •

edited by zach1221

Loading

DeniseWorthen commented Jun 30, 2023 •

edited

Loading

grantfirl commented Jun 30, 2023 •

edited

Loading

zach1221 commented Jul 21, 2023 •

edited by BrianCurtis-NOAA

Loading

grantfirl commented Jul 24, 2023 •

edited

Loading