Script to clone, build workflow and generate experiment for prototypes #470

CoryMartin-NOAA · 2023-05-12T15:38:10Z

As the title says.

Closes #467

Also includes fixes for new global-workflow conventions.

… into feature/gen_proto

CoryMartin-NOAA · 2023-05-12T15:39:39Z

@RussTreadon-NOAA @guillaumevernieres this is a draft PR for now. I'm curious to hear your thoughts on variable names, conventions, etc. (like do we need to adhere to ALL CAPS, or lowercase, or a mix) or does it not matter.

Also, @guillaumevernieres the OCNVAR is "NO" because it fails when I try to run the setup_xml with it. I'm not sure why yet, will have to investigate.

guillaumevernieres · 2023-05-12T15:50:31Z

@RussTreadon-NOAA @guillaumevernieres this is a draft PR for now. I'm curious to hear your thoughts on variable names, conventions, etc. (like do we need to adhere to ALL CAPS, or lowercase, or a mix) or does it not matter.

Also, @guillaumevernieres the OCNVAR is "NO" because it fails when I try to run the setup_xml with it. I'm not sure why yet, will have to investigate.

submitted the fix as a pr into your branch @CoryMartin-NOAA

prototypes/configs/cp0.sh

prototypes/gen_prototype.sh

… into feature/gen_proto

CoryMartin-NOAA · 2023-05-12T17:12:18Z

Marking this as ready to review. Turns out you need app to not be ATM but rather something like S2S if you want DO_JEDIOCNVAR="YES"... who'd have thunk it!

guillaumevernieres

👍 ... Assuming the ci is OK.

emcbot · 2023-05-12T21:48:16Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri May 12 20:34:58 UTC 2023 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri May 12 21:42:58 UTC 2023
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri May 12 21:48:16 UTC 2023
Tests: 82% tests passed, 9 tests failed out of 49
	1451 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP (Failed)
	1452 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT (Failed)
	1453 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY (Failed)
	1454 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1455 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1456 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
	1472 - test_gdasapp_atm_jjob_ens_init (Failed)
	1473 - test_gdasapp_atm_jjob_ens_run (Failed)
	1474 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/470/global-workflow/sorc/gdas.cd/build/log.ctest

RussTreadon-NOAA · 2023-05-13T01:30:25Z

test_gdasapp_atm_jjob_ens* failed because test/atm/global-workflow/jjob_ens_init.sh uses NMEM_ENKF in the loop to populate ROTDIR with member backgrounds. Change to NMEM_ENS

 # Link tiled ges and atmf006 files to ROTDIR
 dpath=enkfgdas.$gPDY/$gcyc
-for imem in $(seq 1 $NMEM_ENKF); do
+for imem in $(seq 1 $NMEM_ENS); do
     memchar="mem"$(printf %03i $imem)

After this change test_gdasapp_atm_jjob_ens* pass.

(gdasapp) Hera(hfe12):/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/work/sorc/gdas.cd/build$ ctest -R test_gdasapp_atm_jjob_ens
Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/work/sorc/gdas.cd/build
    Start 1472: test_gdasapp_atm_jjob_ens_init
1/3 Test #1472: test_gdasapp_atm_jjob_ens_init ....   Passed   43.96 sec
    Start 1473: test_gdasapp_atm_jjob_ens_run
2/3 Test #1473: test_gdasapp_atm_jjob_ens_run .....   Passed  173.34 sec
    Start 1474: test_gdasapp_atm_jjob_ens_final
3/3 Test #1474: test_gdasapp_atm_jjob_ens_final ...   Passed   42.66 sec

100% tests passed, 0 tests failed out of 3

Total Test time (real) = 260.30 sec

Commit change to jjob_ens_init.sh at 6c72f5a.

RussTreadon-NOAA · 2023-05-13T02:34:01Z

Found that with the following change to ush/soca/run_jjobs.py

         self.f.write("\n")
         self.f.write("# Make a copy of config\n")
-        self.f.write(f"cp -r {origconfig} .\n")
+        self.f.write(f"mkdir -p config\n")
+        self.f.write(f"cp -r {origconfig} config/\n")

         # Dump the configs in a separate yaml file

all soca ctests pass

      Start 1458: test_gdasapp_soca_obsdb
20/49 Test #1458: test_gdasapp_soca_obsdb ...................................   Passed    0.80 sec
      Start 1459: test_gdasapp_soca_nsst_increment_to_mom6
21/49 Test #1459: test_gdasapp_soca_nsst_increment_to_mom6 ..................   Passed    2.09 sec
      Start 1460: test_gdasapp_soca_prep
22/49 Test #1460: test_gdasapp_soca_prep ....................................   Passed    0.86 sec
      Start 1461: test_gdasapp_soca_concatioda
23/49 Test #1461: test_gdasapp_soca_concatioda ..............................   Passed    1.72 sec
      Start 1462: test_gdasapp_soca_run_clean
24/49 Test #1462: test_gdasapp_soca_run_clean ...............................   Passed    0.03 sec
      Start 1463: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
25/49 Test #1463: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ........   Passed   42.60 sec
      Start 1464: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
26/49 Test #1464: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ........   Passed   42.82 sec
      Start 1465: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY
27/49 Test #1465: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY ...   Passed   42.96 sec
      Start 1466: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
28/49 Test #1466: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .........   Passed   42.28 sec
      Start 1467: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
29/49 Test #1467: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT .......   Passed   43.75 sec
      Start 1468: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
30/49 Test #1468: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ........   Passed   10.25 sec

I did not commit this change since I don't know if the above change is the right change. It's possible to get the desired behavior, Passed, with the wrong change.

The above was done on Hera in /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/work/sorc/gdas.cd

CoryMartin-NOAA · 2023-05-15T17:14:09Z

thanks @RussTreadon-NOAA ! I do think that is probably the right change also (it was gfs/ instead of config/ like it should be I think) but let's wait for @guillaumevernieres to confirm

… into feature/gen_proto

emcbot · 2023-05-15T20:31:54Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Mon May 15 19:19:18 UTC 2023 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May 15 20:19:40 UTC 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Mon May 15 20:31:54 UTC 2023
Tests: 100% tests passed, 0 tests failed out of 49

RussTreadon-NOAA · 2023-05-15T23:56:57Z

Given success of hera-GW-RT, manually install current head of g-w develop along with head of feature/gen_proto on Orion. Run 49 test_gdasapp tests. All tests pass

(gdasapp) Orion-login-1:/work2/noaa/da/rtreadon/git/global-workflow/test3/sorc/gdas.cd/build$ ctest -R test_gdasapp
Test project /work2/noaa/da/rtreadon/git/global-workflow/test3/sorc/gdas.cd/build
      Start 1427: test_gdasapp_check_python_norms
 1/49 Test #1427: test_gdasapp_check_python_norms ...........................   Passed    1.00 sec
      Start 1428: test_gdasapp_check_yaml_keys
 2/49 Test #1428: test_gdasapp_check_yaml_keys ..............................   Passed    0.42 sec
      Start 1429: test_gdasapp_jedi_increment_to_fv3
 3/49 Test #1429: test_gdasapp_jedi_increment_to_fv3 ........................   Passed    4.68 sec
      Start 1430: test_gdasapp_genYAML_prep
 4/49 Test #1430: test_gdasapp_genYAML_prep .................................   Passed    0.03 sec
      Start 1431: test_gdasapp_genYAML_run
 5/49 Test #1431: test_gdasapp_genYAML_run ..................................   Passed    4.70 sec
      Start 1432: test_gdasapp_genYAML_land_prep
 6/49 Test #1432: test_gdasapp_genYAML_land_prep ............................   Passed    0.03 sec
      Start 1433: test_gdasapp_genYAML_land_run
 7/49 Test #1433: test_gdasapp_genYAML_land_run .............................   Passed    0.76 sec
      Start 1434: test_gdasapp_genYAML_aero_prep
 8/49 Test #1434: test_gdasapp_genYAML_aero_prep ............................   Passed    0.03 sec
      Start 1435: test_gdasapp_genYAML_aero_run
 9/49 Test #1435: test_gdasapp_genYAML_aero_run .............................   Passed    0.70 sec
      Start 1436: test_gdasapp_convert_ewok_yaml
10/49 Test #1436: test_gdasapp_convert_ewok_yaml ............................   Passed    0.22 sec
      Start 1437: test_gdasapp_setup_cycled_exp
11/49 Test #1437: test_gdasapp_setup_cycled_exp .............................   Passed    0.68 sec
      Start 1438: test_gdasapp_convert_bufr_temp_dbuoy
12/49 Test #1438: test_gdasapp_convert_bufr_temp_dbuoy ......................   Passed    3.42 sec
      Start 1439: test_gdasapp_convert_bufr_salt_dbuoy
13/49 Test #1439: test_gdasapp_convert_bufr_salt_dbuoy ......................   Passed    0.24 sec
      Start 1440: test_gdasapp_convert_bufr_temp_mbuoyb
14/49 Test #1440: test_gdasapp_convert_bufr_temp_mbuoyb .....................   Passed    0.25 sec
      Start 1441: test_gdasapp_convert_bufr_salt_mbuoyb
15/49 Test #1441: test_gdasapp_convert_bufr_salt_mbuoyb .....................   Passed    0.25 sec
      Start 1442: test_gdasapp_convert_bufr_tesacprof
16/49 Test #1442: test_gdasapp_convert_bufr_tesacprof .......................   Passed    0.29 sec
      Start 1443: test_gdasapp_convert_bufr_trkobprof
17/49 Test #1443: test_gdasapp_convert_bufr_trkobprof .......................   Passed    0.27 sec
      Start 1444: test_gdasapp_convert_bufr_sfcships
18/49 Test #1444: test_gdasapp_convert_bufr_sfcships ........................   Passed    0.26 sec
      Start 1445: test_gdasapp_convert_bufr_sfcshipsu
19/49 Test #1445: test_gdasapp_convert_bufr_sfcshipsu .......................   Passed    0.27 sec
      Start 1446: test_gdasapp_soca_obsdb
20/49 Test #1446: test_gdasapp_soca_obsdb ...................................   Passed    1.18 sec
      Start 1447: test_gdasapp_soca_nsst_increment_to_mom6
21/49 Test #1447: test_gdasapp_soca_nsst_increment_to_mom6 ..................   Passed   12.59 sec
      Start 1448: test_gdasapp_soca_prep
22/49 Test #1448: test_gdasapp_soca_prep ....................................   Passed    1.47 sec
      Start 1449: test_gdasapp_soca_concatioda
23/49 Test #1449: test_gdasapp_soca_concatioda ..............................   Passed    4.24 sec
      Start 1450: test_gdasapp_soca_run_clean
24/49 Test #1450: test_gdasapp_soca_run_clean ...............................   Passed    0.41 sec
      Start 1451: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
25/49 Test #1451: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ........   Passed   75.02 sec
      Start 1452: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
26/49 Test #1452: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ........   Passed   74.31 sec
      Start 1453: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY
27/49 Test #1453: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY ...   Passed   74.72 sec
      Start 1454: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
28/49 Test #1454: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .........   Passed   42.41 sec
      Start 1455: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
29/49 Test #1455: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT .......   Passed   74.25 sec
      Start 1456: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
30/49 Test #1456: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ........   Passed   10.57 sec
      Start 1457: test_gdasapp_land_create_ens
31/49 Test #1457: test_gdasapp_land_create_ens ..............................   Passed   13.56 sec
      Start 1458: test_gdasapp_land_imsproc
32/49 Test #1458: test_gdasapp_land_imsproc .................................   Passed    5.78 sec
      Start 1459: test_gdasapp_land_apply_jediincr
33/49 Test #1459: test_gdasapp_land_apply_jediincr ..........................   Passed    2.97 sec
      Start 1460: test_gdasapp_land_letkfoi_snowda
34/49 Test #1460: test_gdasapp_land_letkfoi_snowda ..........................   Passed    4.63 sec
      Start 1461: test_gdasapp_convert_bufr_adpsfc
35/49 Test #1461: test_gdasapp_convert_bufr_adpsfc ..........................   Passed    9.82 sec
      Start 1462: test_gdasapp_convert_gsi_satbias
36/49 Test #1462: test_gdasapp_convert_gsi_satbias ..........................   Passed    5.59 sec
      Start 1463: test_gdasapp_store_gsi_satbias
37/49 Test #1463: test_gdasapp_store_gsi_satbias ............................   Passed   11.36 sec
      Start 1464: test_gdasapp_run_jedi_exe_3dhofx
38/49 Test #1464: test_gdasapp_run_jedi_exe_3dhofx ..........................   Passed   40.44 sec
      Start 1465: test_gdasapp_run_jedi_exe_3dvar
39/49 Test #1465: test_gdasapp_run_jedi_exe_3dvar ...........................   Passed   14.53 sec
      Start 1466: test_gdasapp_run_jedi_exe_3denvar
40/49 Test #1466: test_gdasapp_run_jedi_exe_3denvar .........................   Passed   12.55 sec
      Start 1467: test_gdasapp_run_jedi_exe_letkf
41/49 Test #1467: test_gdasapp_run_jedi_exe_letkf ...........................   Passed    7.98 sec
      Start 1468: test_gdasapp_setup_atm_cycled_exp
42/49 Test #1468: test_gdasapp_setup_atm_cycled_exp .........................   Passed    0.78 sec
      Start 1469: test_gdasapp_atm_jjob_var_init
43/49 Test #1469: test_gdasapp_atm_jjob_var_init ............................   Passed   32.56 sec
      Start 1470: test_gdasapp_atm_jjob_var_run
44/49 Test #1470: test_gdasapp_atm_jjob_var_run .............................   Passed   70.04 sec
      Start 1471: test_gdasapp_atm_jjob_var_final
45/49 Test #1471: test_gdasapp_atm_jjob_var_final ...........................   Passed   18.54 sec
      Start 1472: test_gdasapp_atm_jjob_ens_init
46/49 Test #1472: test_gdasapp_atm_jjob_ens_init ............................   Passed   39.86 sec
      Start 1473: test_gdasapp_atm_jjob_ens_run
47/49 Test #1473: test_gdasapp_atm_jjob_ens_run .............................   Passed  234.16 sec
      Start 1474: test_gdasapp_atm_jjob_ens_final
48/49 Test #1474: test_gdasapp_atm_jjob_ens_final ...........................   Passed   28.29 sec
      Start 1475: test_gdasapp_aero_gen_3dvar_yaml
49/49 Test #1475: test_gdasapp_aero_gen_3dvar_yaml ..........................   Passed    0.37 sec

100% tests passed, 0 tests failed out of 49

Total Test time (real) = 944.11 sec

guillaumevernieres

Thanks for fixing my stuff @RussTreadon-NOAA ! I thought I pushed a fix, but apparently not ...
Looks good.

emcbot · 2023-05-16T01:51:32Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Mon May 15 19:01:24 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May 15 20:34:55 CDT 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Mon May 15 20:51:31 CDT 2023
Tests: 100% tests passed, 0 tests failed out of 49

CoryMartin-NOAA added 14 commits May 8, 2023 16:39

Commit to test on Hera

8ae1a83

Update to clone/checkout properly

4222119

Incremental commit

3e40d64

Merge branch 'feature/gen_proto' of https://github.com/NOAA-EMC/GDASApp…

e47acf4

… into feature/gen_proto

incremental commit

0caf649

fix for CI with workflow change

af866cd

actually fix CI

c54c4a0

another attempt to fix CI

40cf983

Small change

08e30ed

end of day commit

3593ab2

Changes to be consistent with g-w

7e8c510

typo fix

3685fcf

Change hashes

89ffbc8

Merge branch 'develop' into feature/gen_proto

0949755

added gfs arg (#471)

ac06b56

guillaumevernieres reviewed May 12, 2023

View reviewed changes

prototypes/configs/cp0.sh Show resolved Hide resolved

guillaumevernieres reviewed May 12, 2023

View reviewed changes

prototypes/gen_prototype.sh Show resolved Hide resolved

CoryMartin-NOAA added 2 commits May 12, 2023 16:48

Run rocotorun and rocotostat

d6f8eb4

Merge branch 'feature/gen_proto' of https://github.com/NOAA-EMC/gdasapp…

3292266

… into feature/gen_proto

CoryMartin-NOAA marked this pull request as ready for review May 12, 2023 17:11

CoryMartin-NOAA added 2 commits May 12, 2023 13:50

Merge branch 'develop' into feature/gen_proto

fc5a92e

comment out bad test

7535781

guillaumevernieres previously approved these changes May 12, 2023

View reviewed changes

CoryMartin-NOAA self-assigned this May 12, 2023

CoryMartin-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion labels May 12, 2023

emcbot removed the hera-GW-RT Queue for automated testing with global-workflow on Hera label May 12, 2023

emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels May 12, 2023

replace NMEM_ENKF with NMEM_ENS in jjobs_ens_init.sh (#467)

6c72f5a

CoryMartin-NOAA added 2 commits May 15, 2023 19:05

modify run_jjobs

b06683a

Merge branch 'feature/gen_proto' of https://github.com/NOAA-EMC/GDASApp…

0331671

… into feature/gen_proto

CoryMartin-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels May 15, 2023

emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels May 15, 2023

emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels May 15, 2023

CoryMartin-NOAA requested review from guillaumevernieres, AndrewEichmann-NOAA and kevindougherty-noaa May 15, 2023 20:33

RussTreadon-NOAA added orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels May 15, 2023

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels May 16, 2023

guillaumevernieres approved these changes May 16, 2023

View reviewed changes

CoryMartin-NOAA merged commit e527786 into develop May 16, 2023
15 checks passed

CoryMartin-NOAA deleted the feature/gen_proto branch May 16, 2023 01:07

emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels May 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Script to clone, build workflow and generate experiment for prototypes #470

Script to clone, build workflow and generate experiment for prototypes #470

CoryMartin-NOAA commented May 12, 2023 •

edited

Loading

CoryMartin-NOAA commented May 12, 2023

guillaumevernieres commented May 12, 2023

CoryMartin-NOAA commented May 12, 2023

guillaumevernieres left a comment

emcbot commented May 12, 2023

RussTreadon-NOAA commented May 13, 2023

RussTreadon-NOAA commented May 13, 2023

CoryMartin-NOAA commented May 15, 2023

emcbot commented May 15, 2023

RussTreadon-NOAA commented May 15, 2023

guillaumevernieres left a comment

emcbot commented May 16, 2023

Script to clone, build workflow and generate experiment for prototypes #470

Script to clone, build workflow and generate experiment for prototypes #470

Conversation

CoryMartin-NOAA commented May 12, 2023 • edited Loading

CoryMartin-NOAA commented May 12, 2023

guillaumevernieres commented May 12, 2023

CoryMartin-NOAA commented May 12, 2023

guillaumevernieres left a comment

Choose a reason for hiding this comment

emcbot commented May 12, 2023

RussTreadon-NOAA commented May 13, 2023

RussTreadon-NOAA commented May 13, 2023

CoryMartin-NOAA commented May 15, 2023

emcbot commented May 15, 2023

RussTreadon-NOAA commented May 15, 2023

guillaumevernieres left a comment

Choose a reason for hiding this comment

emcbot commented May 16, 2023

CoryMartin-NOAA commented May 12, 2023 •

edited

Loading