Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update noaacloud module files #433

Closed

Conversation

ulmononian
Copy link
Collaborator

@ulmononian ulmononian commented Oct 26, 2022

DESCRIPTION OF CHANGES:

Recently, the SRW spack-stack environment was updated on the noaacloud platforms (AWS, Azure, GCP) to match the package versions of SRW develop's srw_common module file. Also, as with PR #431, new miniconda (w/ regional_workflow environment) installations were made on each cloud platform.

To address the aforementioned changes, this PR updates:

  • the noaacloud module files (inlcuding the addition of an srw_common_spack.lua file, similar to that in srw-release-v2)
  • MET/METplus paths & PRE_TASK_CMDS (to solve a 'time' package issue on AWS) in the noaacloud machine YAML file
  • the ush/load_modules_run_task.sh script to fix cloud-based regional_workflow activation issues (may conflict w/ fix for conda on gaea #428)

Type of change

  • [ x] Bug fix (non-breaking change which fixes an issue)
  • [x ] New feature (non-breaking change which adds functionality)

TESTS CONDUCTED:

  • hera.intel
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • [ x] NOAA Cloud (aws, azure, gcp; srw-develop builds complete on each; workflow checks underway w/ select WE2E tests: all tasks passing on azure)
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • [x ] My changes may need updates to the documentation.
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • [ x] My changes generate no new warnings. - Some new warnings in run-time log files appeared, such as BASH_SOURCE[1]: unbound variable . This was observed following recent PR merges.

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • Needs Cheyenne test
  • Needs Jet test
  • Needs Hera test
  • Needs Orion test
  • help wanted

CONTRIBUTORS (optional):

@EdwardSnyder-NOAA

ulmononian and others added 9 commits October 24, 2022 08:58
update spack-stack location & esmf version
update miniconda paths
update met/metplus locations
…o fix python package loading issues when loading regional_workflow environment on the cloud.
Add time package installation to noaacloud.yaml file to enable access to time package at runtime (via compute node) within workflow.
@ulmononian ulmononian closed this Oct 26, 2022
@ulmononian ulmononian reopened this Oct 26, 2022
RUN_CMD_FCST: mpiexec -np ${PE_MEMBER01}
RUN_CMD_POST: mpiexec -np $nprocs
RUN_CMD_SERIAL: time
RUN_CMD_UTILS: mpiexec -np $nprocs
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; }'
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; sudo yum -y install time; }'
BUILD_MOD_FN: wflow_noaacloud
Copy link
Collaborator

@danielabdi-noaa danielabdi-noaa Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ulmononian Do you know why this line is needed? I wanted to remove it in the past because it doesn't make sense to pass the workflow module as the build module file. Not specifying it in the machine file is equivalent to using BUILD_MOD_FN: build_noaacloud_intel, so it would be nice if you can test removing this line to see if it works.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielabdi-noaa thanks for taking a look at this! let me look into this and i will get back to you.

@MichaelLueken
Copy link
Collaborator

@ulmononian With the inclusion of @mark-a-potts PR #444, can this PR be closed, or are there additional changes that still need to be merged?

@MichaelLueken
Copy link
Collaborator

The majority of these changes have been merged via PR #444. Closing this PR now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants