-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
copy over changes from before de-submodule transition
- Loading branch information
1 parent
7464ca2
commit 96526b4
Showing
5 changed files
with
567 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Overrides the shell detected from the shebang. | ||
# This is useful for files meant to be included (and thus lacking a shebang), | ||
# or possibly as a more targeted alternative to 'disable=SC2039'. | ||
shell=sh | ||
# You can override this configuration on the command line with --shell | ||
|
||
# Always allow ShellCheck to open arbitrary files from 'source' statements. | ||
external-sources=true | ||
check-sourced=true | ||
|
||
# Enable all optional checks | ||
enable=all | ||
|
||
# We want the most picky shellcheck so we have it even flag style errors | ||
severity=style | ||
|
||
# This function is invoked in an 'if' condition so set -e will be disabled. | ||
# Invoke separately if failures should cause the script to exit. | ||
# - We don't want to exit if errors happen inside a check, that's why we have a check... | ||
disable=SC2310 | ||
|
||
# Bash implicitly disabled set -e for this function invocation because | ||
# it's inside a command substitution. Add set -e; before it or enable inherit_errexit. | ||
# - Don't care if we inherit errexit inside substitutions, we do checks for that. | ||
disable=SC2311 | ||
|
||
# Consider invoking this command separately to avoid masking its return value | ||
# (or use '|| true' to ignore). | ||
# - We already check errors and adding "|| true" everywhere hinders readability. | ||
disable=SC2312 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
# LDMX Environment Initialization | ||
|
||
The LDMX development and running environment is defined as a container image. | ||
Running this container image properly can be complicated on many systems, | ||
so we've isolated the running of this container into shell scripts so that | ||
the average user/developer does not need to worry about the intricacies | ||
of container running. | ||
|
||
## Legacy Solution | ||
The first development of this solution defined a series of functions in | ||
`bash` that wrapped `docker` and `singularity` and focused on running the | ||
image as built by [LDMX-Software/docker](https://github.com/LDMX-Software/docker). | ||
This solution, while functional, has a few edge-case issues and is difficult | ||
to test independently of ldmx-sw. | ||
These issues motivated the development of a separate project which we can | ||
now use for development of ldmx-sw. | ||
|
||
## denv | ||
[denv](https://tomeichlersmith.github.io/denv/) | ||
is a project to generalize the original bash functions to support | ||
more shells, more systems, and more container runners. With a ground-up | ||
redesign, it has benefited from lessons learned while using the original | ||
bash functions and, as a independent project, can be thoroughly tested. | ||
|
||
Utilizing `denv` to develop and run ldmx-sw is a simpler task than | ||
trying to use the container runners directly; however, it is a bit | ||
more complicated than the original bash functions due to its generality. | ||
This makes defining wrapping environment scripts a helpful tool for | ||
easing the burden placed on any users. Below, the LDMX denv configuration | ||
is documented to help users understand what is happening as well as | ||
help anyone interested in developing a wrapping script to be used | ||
within the shell of their choice. | ||
|
||
### Initialization | ||
``` | ||
denv \ | ||
init \ # use denv special sub-command for initialization | ||
--clean-env \ # avoid passing environment variables | ||
--name "ldmx" \ # name this denv after our experiment :) | ||
"ldmx/dev:latest" \ # use the dev image | ||
<path-to-ldmx-base> # path to directory containing ldmx-sw | ||
``` | ||
For example, if you are in the ldmx-sw directory, | ||
``` | ||
denv init --clean-env --name ldmx ldmx/dev:latest .. | ||
``` | ||
The default path is the current directory, so you could also init immediately | ||
after cloning ldmx-sw. | ||
``` | ||
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git | ||
denv init --clean-env --name ldmx ldmx/dev:latest | ||
``` | ||
The default `--name` of a denv is the directory where `init` was run, | ||
so the final example (and the one I use on most of my machines) is | ||
``` | ||
mkdir ldmx | ||
cd ldmx | ||
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git | ||
denv init --clean-env ldmx/dev:latest | ||
``` | ||
|
||
### Wrapping | ||
Wrapping `denv` can have the benefit of reducing the amount of characters | ||
needed for the average user to type when doing common tasks. Environment | ||
scripts wrapping denv for a specific shell should do the following. | ||
|
||
- Deduce `LDMX_BASE` and define it as an environment variable for the user | ||
- Make sure the LDMX denv is initialized for the user | ||
- Define the `ldmx` function with CLI defined below | ||
|
||
### `ldmx` CLI Specification | ||
v0.0.0: just an idea | ||
|
||
- `ldmx` | ||
- `help` : print help message describing the CLI | ||
- `config` : print shell version being used and run `denv config print` | ||
- `use` : first argument is repo (`dev` or `pro`) and then second argument is tag (e.g. `latest`) | ||
- deduce full docker image tag name from repo and tag and then if expand the tag to include | ||
the path on `/cvmfs` (if available) or just the DockerHub tag (if not) | ||
- `pull` : call `ldmx use` and then `denv config image pull` | ||
- `mount` : pass arguments to `denv config mounts` | ||
- `setenv` : pass arguments to `denv config env copy` | ||
- `<cmd>` : pass directly to `denv` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
############################################################################### | ||
# ldmx-denv-init.sh | ||
# Make sure that the LDMX denv is initialized properly before proceeding. | ||
# | ||
# This script is desigend to be idempotent: i.e. if the denv is already | ||
# properly configured, then this script should not have any effect. | ||
############################################################################### | ||
|
||
# print each argument on its own line with the first line | ||
# prefixed with "ERROR [ldmx-denv-init.sh]: ". | ||
_error() { | ||
printf >&2 "\033[1;31mERROR [ldmx-denv-init.sh]: \033[0m\033[31m%s\n" "$1" | ||
shift | ||
while [ "$#" -gt "0" ]; do | ||
printf >&2 " %s\n" "$1" | ||
shift | ||
done | ||
printf >&2 "\033[0m" | ||
} | ||
|
||
# print the question passed to us and process user input | ||
# continues in infinite loop if user is not explicitly answering yes or no | ||
# Arguments | ||
# 1 - question to ask the user (yes or no) | ||
# Output | ||
# returns 0 if user answers yes and 1 if user answers no | ||
_user_confirm() { | ||
question="$*" | ||
while true; do | ||
printf "\033[1m%s\033[0m [Y/n] " "${question}" | ||
read -r ans | ||
case "${ans}" in | ||
Y|y|yes) | ||
return 0 | ||
;; | ||
N|n|no) | ||
return 1 | ||
;; | ||
*) | ||
_error "${ans} is not one of Y, y, yes, N, n, or no." | ||
;; | ||
esac | ||
done | ||
} | ||
|
||
if ! command -v denv > /dev/null 2>&1; then | ||
_error "'denv' is not installed." | ||
if _user_confirm "Do you wish to try to install 'denv' now?"; then | ||
curl -s https://raw.githubusercontent.com/tomeichlersmith/denv/main/install | sh | ||
else | ||
printf "Not attempting to install 'denv'. Please follow the instructions at\n %s\n" \ | ||
"https://tomeichlersmith.github.io/denv/getting_started.html#installation" | ||
unset -f _error _user_confirm | ||
return 1 | ||
fi | ||
fi | ||
|
||
if ! denv check --quiet; then | ||
# maybe try loading apptainer via the 'module' command? | ||
# https://github.com/LDMX-Software/ldmx-sw/issues/1248#issuecomment-1896618339 | ||
denv check | ||
_error "'denv' unable to find a supported container runner." \ | ||
"Install one of the container runners 'denv' checked for above." | ||
unset -f _error _user_confirm | ||
return 2 | ||
fi | ||
|
||
############################################################################### | ||
# Attempt to deduce LDMX_BASE | ||
# Unfortunately, deducing the path to a script being sourced is not possible | ||
# using POSIX-compliant methods[1], so we need to fallback to separating | ||
# different running scenarios. | ||
# | ||
# [1]: https://stackoverflow.com/a/29835459 | ||
# | ||
# Users can avoid all of this complication by pre-defining the LDMX_BASE | ||
# environment variable. | ||
############################################################################### | ||
if [ -z "${LDMX_BASE+x}" ]; then | ||
_full_path() ( | ||
CDPATH= | ||
cd -- "${1}" && pwd -P | ||
) | ||
_base_from_script() { | ||
_full_path "$(dirname -- "${1}")/../../" | ||
} | ||
if [ -n "${ZSH_VERSION}" ]; then | ||
# zsh defines ZSH_EVAL_CONTEXT holding the details on how | ||
# this code is being executed | ||
# disable warnings about undefined variable and string indexing | ||
# shellcheck disable=SC2154,SC3057 | ||
LDMX_BASE="$(_base_from_script "${ZSH_EVAL_CONTEXT:file}")" | ||
elif [ -n "${KSH_VERSION}" ]; then | ||
# Korn defines the _ variable to the the file being sourced | ||
# disable warning about undefined '_' | ||
# shellcheck disable=SC3028 | ||
LDMX_BASE="$(_base_from_script "${_}")" | ||
elif [ -n "${BASH_VERSION}" ]; then | ||
# running from bash, bash defines the BASH_SOURCE array to help us | ||
# find the location of this file | ||
# disable warning about undefined variable and array references | ||
# shellcheck disable=SC3028,SC3054 | ||
LDMX_BASE="$(_base_from_script "${BASH_SOURCE[0]}")" | ||
else | ||
# unable to deduce shell, resort to pwd | ||
if expr "${PWD}" : ".*ldmx-sw$"; then | ||
LDMX_BASE="$(_full_path ..)" | ||
elif expr "${PWD}" : ".*ldmx-sw/scripts$"; then | ||
LDMX_BASE="$(_full_path ../..)" | ||
fi | ||
fi | ||
unset -f _full_path _base_from_script | ||
fi | ||
# re-export LDMX_BASE in case of user does inline variable definition like | ||
# LDMX_BASE=/path/to/ldmx/base source /full/path/to/ldmx-denv-init.sh | ||
export LDMX_BASE | ||
|
||
############################################################################### | ||
# Check if the LDMX denv is initialized. If not, do a default initialization. | ||
# TODO: update to `denv check --workspace` introduced in denv v0.7.0 to avoid | ||
# reliance on internal implementation on how denv stores its config | ||
############################################################################### | ||
if [ ! -f "${LDMX_BASE}/.denv/config" ]; then | ||
denv init --clean-env --name "ldmx" "ldmx/dev:latest" "${LDMX_BASE}" | ||
fi | ||
unset -f _error _user_confirm |
Oops, something went wrong.