replace model_data with model

[RussTreadon-NOAA]

NCO requires model_data be replaced with model in the path to model data. This PR satisfies NCO's requirement.

Resolves #1198

[RussTreadon-NOAA]

Completion of this PR requires model_data be replaced with model in lowres data paths in GDASAPP_TESTDATA on WCOSS2, Hera, Hercules, and Orion. Specifically the following needs to be done

• cd $GDAAPP_TESTDATA/lowres
• cd enkfgdas.20210323/12
• For mem001, mem002, and mem003,
  • mv model_data model
  • ln -fs model_data model - this is done so that g-w dependent GDASApp ctests still work before the model_data to model change is implemented in g-w.
• cd $GDAAPP_TESTDATA/lowres
  • cd gdas.20210323/12
  • mv model_data model
  • ln -fs model model_data

The above has been done on

• WCOSS2: GDASAPP_TESTDATA=/lfs/h2/emc/da/noscrub/emc.da/GDASApp/data
• Hera: GDASAPP_TESTDATA=/scratch1/NCEPDEV/da/role.jedipara/CI/GDASApp/data

Currently modulefiles/GDAS/hera.intel.lua sets

setenv("GDASAPP_TESTDATA","/scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/data")

A modified copy of hera.intel.lua was committed to feature/model at adb1b6d. The modified hera modulefile in feature/model points GDASAPP_TESTDATA at role.jedipara

@CoryMartin-NOAA and @guillaumevernieres , I see role-da on Orion/Hercules. I can access this account. However, is this the role account we want to use on Orion/Hercules for GDASAPP data? Do we want a new EMC-DA role account on Orion/Hercules?

[CoryMartin-NOAA]

@RussTreadon-NOAA I believe that is the role account we have access to for this purpose

[RussTreadon-NOAA]

Hera ctests
Install feature/model inside working copy of g-w PR #2700. Run test_gdasapp. All tests pass

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/pr2700/sorc/gdas.cd/build
      Start 1488: test_gdasapp_util_coding_norms
 1/48 Test #1488: test_gdasapp_util_coding_norms ........................   Passed    2.72 sec
      Start 1489: test_gdasapp_util_ioda_example
 2/48 Test #1489: test_gdasapp_util_ioda_example ........................   Passed    2.39 sec
...
      Start 1869: test_gdasapp_aero_gen_3dvar_yaml
48/48 Test #1869: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    1.25 sec

100% tests passed, 0 tests failed out of 48

Label Time Summary:
gdas-utils    =  17.73 sec*proc (11 tests)
script        =  17.73 sec*proc (11 tests)

Total Test time (real) = 3529.84 sec

Orion ctests
Install feature/model inside working copy of g-w PR #2700. Run test_gdasapp. All tests pass

Test project /work2/noaa/da/rtreadon/git/global-workflow/rename_atm/sorc/gdas.cd/build
      Start 1489: test_gdasapp_util_coding_norms
 1/48 Test #1489: test_gdasapp_util_coding_norms ........................   Passed    6.68 sec
      Start 1490: test_gdasapp_util_ioda_example
 2/48 Test #1490: test_gdasapp_util_ioda_example ........................   Passed    2.51 sec
...
      Start 1870: test_gdasapp_aero_gen_3dvar_yaml
48/48 Test #1870: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    0.54 sec

100% tests passed, 0 tests failed out of 48

Label Time Summary:
gdas-utils    =  15.32 sec*proc (11 tests)
script        =  15.32 sec*proc (11 tests)

Total Test time (real) = 1884.31 sec

WCOSS2 (Dogwood) ctests
Install feature/model inside working copy of g-w PR #2700. Run test_gdasapp. 20 our of 46 tests fail. This is a known problem. Several GDASApp ctests assume SLURM is available. WCOSS2 uses PBS. The tests which failed using feature/model are the same tests which failed in the past.

57% tests passed, 20 tests failed out of 46

Label Time Summary:
gdas-utils    =  10.47 sec*proc (9 tests)
script        =  10.47 sec*proc (9 tests)

Total Test time (real) = 358.00 sec

The following tests FAILED:
        1751 - test_gdasapp_fv3jedi_fv3inc (Not Run)
        1756 - test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS (Failed)
        1757 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP (Failed)
        1758 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT (Failed)
        1759 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
        1760 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN (Failed)
        1761 - test_gdasapp_soca_copy_scratch (Failed)
        1762 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
        1763 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
        1764 - test_gdasapp_soca_socahybridweights (Failed)
        1765 - test_gdasapp_soca_incr_handler (Failed)
        1766 - test_gdasapp_soca_ens_handler (Failed)
        1769 - test_gdasapp_snow_apply_jediincr (Failed)
        1770 - test_gdasapp_snow_letkfoi_snowda (Failed)
        1776 - test_gdasapp_atm_jjob_var_run (Failed)
        1777 - test_gdasapp_atm_jjob_var_inc (Failed)
        1778 - test_gdasapp_atm_jjob_var_final (Failed)
        1780 - test_gdasapp_atm_jjob_ens_run (Failed)
        1781 - test_gdasapp_atm_jjob_ens_inc (Failed)
        1782 - test_gdasapp_atm_jjob_ens_final (Failed)

[RussTreadon-NOAA]

@CoryMartin-NOAA , I agree that the problem Yongzou reports is related to emcrzdm issues. However, doesn't the build first check if the tarfile is available from disk? If true, my renaming of the test directories on WCOSS2 (Dogwood) will cause problems for developers. The develop wcoss2.intel.lua has

setenv("CRTM_FIX","/lfs/h2/emc/da/noscrub/emc.da/GDASApp/fix/crtm/2.4.0")
setenv("GDASAPP_TESTDATA","/lfs/h2/emc/da/noscrub/emc.da/GDASApp/data")
setenv("GDASAPP_UNIT_TEST_DATA_PATH", "/lfs/h2/emc/da/noscrub/emc.da/GDASApp/data/test")

These paths have been changed to

setenv("CRTM_FIX","/lfs/h2/emc/da/noscrub/emc.da/GDASApp/fix/crtm/2.4.0")
setenv("GDASAPP_TESTDATA","/lfs/h2/emc/da/noscrub/emc.da/GDASApp/testdata")
setenv("GDASAPP_UNIT_TEST_DATA_PATH", "/lfs/h2/emc/da/noscrub/emc.da/GDASApp/unittestdata")

The Hera, Hercules, and Orion modulefiles in develop still point at your directories.

[CoryMartin-NOAA]

Good point @RussTreadon-NOAA

[RussTreadon-NOAA]

@CoryMartin-NOAA : If we are OK with this PR in it's current form, we should merge since some developers are building GDASApp on WCOSS2.