Skip to content

Refactor mache.deploy for E3SM-Unified deployment support#374

Merged
xylar merged 8 commits intoE3SM-Project:mainfrom
xylar:updates-for-unified
Apr 2, 2026
Merged

Refactor mache.deploy for E3SM-Unified deployment support#374
xylar merged 8 commits intoE3SM-Project:mainfrom
xylar:updates-for-unified

Conversation

@xylar
Copy link
Copy Markdown
Collaborator

@xylar xylar commented Mar 29, 2026

Add the deploy changes needed to support the E3SM-Unified migration
without baking E3SM-specific behavior into generic downstream workflows.

  • add platform default machine configs in mache.machines
  • support separate login and compute pixi environments in deploy run/load
  • allow hooks to override pixi prefix, channels, MPI, and software version
  • pass spack deployment results and generated load scripts through runtime
  • treat the hpc pixi variant correctly when rendering package build strings
  • preserve generic mache behavior by keeping e3sm-unified-specific fallback
    logic out of mache core
  • update deploy documentation and templates to match the new workflow

Checklist

  • User's Guide has been updated if needed
  • Tests pass and new features are covered by tests
  • Testing comment, if appropriate, in the PR documents testing used to verify the changes

xylar added 5 commits March 28, 2026 14:06
Add the deploy changes needed to support the E3SM-Unified migration
without baking E3SM-specific behavior into generic downstream workflows.

- add platform default machine configs in mache.machines
- support separate login and compute pixi environments in deploy run/load
- allow hooks to override pixi prefix, channels, MPI, and software version
- pass spack deployment results and generated load scripts through runtime
- treat the hpc pixi variant correctly when rendering package build strings
- preserve generic mache behavior by keeping e3sm_unified-specific fallback
  logic out of mache core
- update deploy documentation and templates to match the new workflow
Cover the deploy runtime changes added for the E3SM-Unified workflow.

- test runtime pixi prefix and channel overrides
- test login/compute environment load-script generation
- test platform default machine config loading
- test known-machine handling for new default cfg files
- test that machine metadata remains scoped correctly after the deploy changes
text
Make `--pixi-path` the canonical deploy CLI flag for the pixi
environment location, while keeping `--prefix` as a deprecated alias
for backward compatibility.

Update the generated CLI spec, bootstrap parser, deploy-time argument
resolution, and focused tests for both the new and legacy spellings.
@xylar xylar requested a review from andrewdnolan March 29, 2026 14:47
@xylar xylar self-assigned this Mar 29, 2026
@xylar xylar added enhancement New feature or request clean-up Cleanup or maintenance of code that does not alter behavior or functionality labels Mar 29, 2026
@xylar
Copy link
Copy Markdown
Collaborator Author

xylar commented Mar 29, 2026

Testing

I did a test deployment on Chrysalis with this branch and E3SM-Project/e3sm-unified#145. Pixi (login and compute) and spack environments were successfully deployed. Permissions appear to have been updated as desired. The test_*.sh load script ended up in the expected place. I could source it and see that ncks as in the pixi environment by default, but in the spack environment if I faked a Slurm job ID.

I tested both with local-build and conda-forge as the package source, and both are working as expected.

xylar added 3 commits March 30, 2026 16:36
When we are installing a mache release candidate into a bootstrap
environment, we need to get it from the `label/mache_dev` label
on conda-forge.
@xylar xylar merged commit b8833b0 into E3SM-Project:main Apr 2, 2026
9 checks passed
@xylar xylar deleted the updates-for-unified branch April 2, 2026 10:08
@xylar xylar removed the request for review from andrewdnolan April 2, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clean-up Cleanup or maintenance of code that does not alter behavior or functionality enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant