Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/public v2 gaea update #352

Conversation

natalie-perlin
Copy link
Collaborator

DESCRIPTION OF CHANGES:

Changes to this SRW release/public-v2 branch are specific to Gaea, and reflect the recent updates to the Lmod/8.7.12, miniconda3/4.12.0, and the hpc-stack modules built with the updated Lmod and miniconda. All these packages and the module manager have been installed under the role.epic account, maintained by EPIC, and in a following common space on Gaea:
/lustre/f2/dev/wpo/role.epic/contrib/
(also linked as /lustre/f2/dev/role.epic/contrib/)

These updates were found to be needed to address missing python packages in existing regional_workflow conda environments, when python plotting routines were used to generate basic plots of the model results.

The changes are related to Gaea only, and their sole purpose is to initialize Lmod properly (which is not installed system-wide on Gaea), and update the paths of the modules.

The files in the SRW repo that were updated, under the ./ufs-srweather-app/ main directory:

./etc/lmod-setup.sh, lmod_setup.csh
./modulefiles/build_gaea_intel
./modulefiles/wflow_gaea

Running the SRW also requred corresponding updates to the regional_workflow repository (yet unmerged, release/public-v2 branch), for which the PR-380 has been created in that repository:
PR-380

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

TESTS CONDUCTED:

The test has been done on Gaea, to build the SRW and to run a workflow using rocoto and crontab, or the following test case from the comprehensive tests suite:
grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.

Because of the dependency of the PR-380 in the regional_workflow repo, temporary modifications to the Externals.cfs in the tested code have been made to use the following regional_workflow repository and branch:

[regional_workflow]
protocol = git
#repo_url = https://github.com/ufs-community/regional_workflow
repo_url = https://github.com/natalie-perlin/regional_workflow
# Specify either a branch name or a hash but not both.
branch = release/public-v2-gaea-update
local_path = regional_workflow
required = True
...

The test intended to provide the concept of loading the new Lmod, modules and packages properly, building the SRW code, and a functional workflow that launches and successfully completes the test.

Location of the test code built and launched on Gaea, under EPIC account:
/lustre/f2/dev/wpo/role.epic/sandbox/SRW/ufs-srw-pub-v2-update/

A corresponding test directory on Gaea is the following:
/lustre/f2/dev/wpo/role.epic/sandbox/SRW/expt_dirs/grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16

  • hera.intel
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • NOAA Cloud (indicate which platform)
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)
    expt_dirs/grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16

DEPENDENCIES:

ISSUE:

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes - only tested the workflow functionality, a single test
  • [ not yet, a dependent PR-380 is as described above] Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • Needs Cheyenne test
  • Needs Jet test
  • Needs Hera test
  • Needs Orion test
  • help wanted

CONTRIBUTORS (optional):

…epic account

Add initialization of the Lmod installed on Gaea under the role.epic account, and maintained by EPIC, in /lustre/f2/dev/role.epic/contrib/ space
…pic account

Added initialization of the Lmod installed on Gaea under the role.epic account, and maintained by EPIC, in /lustre/f2/dev/role.epic/contrib/ space
Updated paths for the modules needed to build the SRW, 
build following the update of the Lmod/8.7.12. All of the packages were installed under role.epic account in /lustre/f2/dev/role.epic/contrib/ space.
Updated paths for the modules needed to run the workflow for the SRW, 
build following the update of the Lmod/8.7.12. All of the packages were installed under role.epic account in /lustre/f2/dev/role.epic/contrib/ space.
Copy link
Collaborator

@panll panll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@danielabdi-noaa
Copy link
Collaborator

@natalie-perlin I am able to checkout regional workflow with manage_externals and build your PR on gaea applying it to release/public-v2 branch so will approve! Originally I encountered the issue you mentioned with nemsio library when I applied the PR on top of develop by mistake.

@natalie-perlin natalie-perlin deleted the release/public-v2-gaea-update branch January 24, 2023 11:20
@natalie-perlin natalie-perlin restored the release/public-v2-gaea-update branch January 24, 2023 11:21
@natalie-perlin natalie-perlin deleted the release/public-v2-gaea-update branch January 24, 2023 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants