Skip to content

Conversation

@infotroph
Copy link
Member

Description

Modifies the Sipnet job.sh template to make it compute the absolute paths of OUTDIR, RUNDIR, SITE_MET, and BINARY at runtime, thus allowing paths in pecan.xml to be relative to the workflow's run directory and thus potentially host-agnostic.

Paths specified as absolute will continue to work as before.

I also went ahead and implemented path expansion for $BINARY while I was at it -- even if most people keep specifying this as an absolute path, it may be useful for side-by-side testing of different Sipnet builds on the same system before deployment.

Motivation and Context

The existing PEcAn standard practice is to populate machine-specific absolute paths in the XML file, either directly in text or by specifying database identifiers that are then looked up in BETYdb and expanded to absolute paths during the settings check process. This is all well and good when all files can be assumed tracked in BETYdb, but becomes annoying in db-free runs, and it means one XML file cannot be reliably reused across hosts.

A popular* alternate design is to use portable project directories: If all files needed for a run live in the same directory and all paths are expressed relative to the root of that directory, then paths stay valid when that directory is copied as a unit, either within or between machines. This PR attempts to support portable directories without breaking workflows that specify absolute paths -- if you see a case I missed that will change absolute paths, please flag it!

A particular motivating use case for me: I often create a project directory on my machine and then test-run it on multiple systems by mounting the same folder into different Docker images, meaning the same file might live in quick succession at /Users/chrisb/projects/foo/met/bar.clim, /work/met/bar.clim, and /projectnb/dietzelab/chrisb/foo/met/bar.clim. Sure, I could write another script to edit the XML each time... or I could write it as ./met/bar.clim and always run the workflow from the directory that contains the XML.

*(Yes, by "a popular" I mean "my preferred." Guilty as charged.)

Review Time Estimate

  • Immediately
  • Within one week
  • When possible

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • My name is in the list of CITATION.cff
  • I agree that PEcAn Project may distribute my contribution under any or all of
    • the same license as the existing code,
    • and/or the BSD 3-clause license.
  • I have updated the CHANGELOG.md.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@infotroph infotroph merged commit 791df40 into PecanProject:develop Jan 23, 2025
17 of 23 checks passed
@infotroph infotroph deleted the sipnet-relative-paths branch January 23, 2025 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants