Skip to content

Commit

Permalink
copy over changes from before de-submodule transition
Browse files Browse the repository at this point in the history
  • Loading branch information
tomeichlersmith committed May 1, 2024
1 parent 7464ca2 commit 96526b4
Show file tree
Hide file tree
Showing 5 changed files with 567 additions and 1 deletion.
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ endif()

# If a user is building outside of a Docker or Singularity environment,
# warn them.
if (NOT EXISTS /.dockerenv AND NOT EXISTS /singularity)
if (NOT EXISTS /.dockerenv AND NOT EXISTS /singularity AND NOT EXISTS /run/.containerenv)
message(WARNING "You are not inside a container; you may be working in an untested environment.")
endif()

Expand Down
30 changes: 30 additions & 0 deletions scripts/.shellcheckrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Overrides the shell detected from the shebang.
# This is useful for files meant to be included (and thus lacking a shebang),
# or possibly as a more targeted alternative to 'disable=SC2039'.
shell=sh
# You can override this configuration on the command line with --shell

# Always allow ShellCheck to open arbitrary files from 'source' statements.
external-sources=true
check-sourced=true

# Enable all optional checks
enable=all

# We want the most picky shellcheck so we have it even flag style errors
severity=style

# This function is invoked in an 'if' condition so set -e will be disabled.
# Invoke separately if failures should cause the script to exit.
# - We don't want to exit if errors happen inside a check, that's why we have a check...
disable=SC2310

# Bash implicitly disabled set -e for this function invocation because
# it's inside a command substitution. Add set -e; before it or enable inherit_errexit.
# - Don't care if we inherit errexit inside substitutions, we do checks for that.
disable=SC2311

# Consider invoking this command separately to avoid masking its return value
# (or use '|| true' to ignore).
# - We already check errors and adding "|| true" everywhere hinders readability.
disable=SC2312
84 changes: 84 additions & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# LDMX Environment Initialization

The LDMX development and running environment is defined as a container image.
Running this container image properly can be complicated on many systems,
so we've isolated the running of this container into shell scripts so that
the average user/developer does not need to worry about the intricacies
of container running.

## Legacy Solution
The first development of this solution defined a series of functions in
`bash` that wrapped `docker` and `singularity` and focused on running the
image as built by [LDMX-Software/docker](https://github.com/LDMX-Software/docker).
This solution, while functional, has a few edge-case issues and is difficult
to test independently of ldmx-sw.
These issues motivated the development of a separate project which we can
now use for development of ldmx-sw.

## denv
[denv](https://tomeichlersmith.github.io/denv/)
is a project to generalize the original bash functions to support
more shells, more systems, and more container runners. With a ground-up
redesign, it has benefited from lessons learned while using the original
bash functions and, as a independent project, can be thoroughly tested.

Utilizing `denv` to develop and run ldmx-sw is a simpler task than
trying to use the container runners directly; however, it is a bit
more complicated than the original bash functions due to its generality.
This makes defining wrapping environment scripts a helpful tool for
easing the burden placed on any users. Below, the LDMX denv configuration
is documented to help users understand what is happening as well as
help anyone interested in developing a wrapping script to be used
within the shell of their choice.

### Initialization
```
denv \
init \ # use denv special sub-command for initialization
--clean-env \ # avoid passing environment variables
--name "ldmx" \ # name this denv after our experiment :)
"ldmx/dev:latest" \ # use the dev image
<path-to-ldmx-base> # path to directory containing ldmx-sw
```
For example, if you are in the ldmx-sw directory,
```
denv init --clean-env --name ldmx ldmx/dev:latest ..
```
The default path is the current directory, so you could also init immediately
after cloning ldmx-sw.
```
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git
denv init --clean-env --name ldmx ldmx/dev:latest
```
The default `--name` of a denv is the directory where `init` was run,
so the final example (and the one I use on most of my machines) is
```
mkdir ldmx
cd ldmx
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git
denv init --clean-env ldmx/dev:latest
```

### Wrapping
Wrapping `denv` can have the benefit of reducing the amount of characters
needed for the average user to type when doing common tasks. Environment
scripts wrapping denv for a specific shell should do the following.

- Deduce `LDMX_BASE` and define it as an environment variable for the user
- Make sure the LDMX denv is initialized for the user
- Define the `ldmx` function with CLI defined below

### `ldmx` CLI Specification
v0.0.0: just an idea

- `ldmx`
- `help` : print help message describing the CLI
- `config` : print shell version being used and run `denv config print`
- `use` : first argument is repo (`dev` or `pro`) and then second argument is tag (e.g. `latest`)
- deduce full docker image tag name from repo and tag and then if expand the tag to include
the path on `/cvmfs` (if available) or just the DockerHub tag (if not)
- `pull` : call `ldmx use` and then `denv config image pull`
- `mount` : pass arguments to `denv config mounts`
- `setenv` : pass arguments to `denv config env copy`
- `<cmd>` : pass directly to `denv`

126 changes: 126 additions & 0 deletions scripts/ldmx-denv-init.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
###############################################################################
# ldmx-denv-init.sh
# Make sure that the LDMX denv is initialized properly before proceeding.
#
# This script is desigend to be idempotent: i.e. if the denv is already
# properly configured, then this script should not have any effect.
###############################################################################

# print each argument on its own line with the first line
# prefixed with "ERROR [ldmx-denv-init.sh]: ".
_error() {
printf >&2 "\033[1;31mERROR [ldmx-denv-init.sh]: \033[0m\033[31m%s\n" "$1"
shift
while [ "$#" -gt "0" ]; do
printf >&2 " %s\n" "$1"
shift
done
printf >&2 "\033[0m"
}

# print the question passed to us and process user input
# continues in infinite loop if user is not explicitly answering yes or no
# Arguments
# 1 - question to ask the user (yes or no)
# Output
# returns 0 if user answers yes and 1 if user answers no
_user_confirm() {
question="$*"
while true; do
printf "\033[1m%s\033[0m [Y/n] " "${question}"
read -r ans
case "${ans}" in
Y|y|yes)
return 0
;;
N|n|no)
return 1
;;
*)
_error "${ans} is not one of Y, y, yes, N, n, or no."
;;
esac
done
}

if ! command -v denv > /dev/null 2>&1; then
_error "'denv' is not installed."
if _user_confirm "Do you wish to try to install 'denv' now?"; then
curl -s https://raw.githubusercontent.com/tomeichlersmith/denv/main/install | sh
else
printf "Not attempting to install 'denv'. Please follow the instructions at\n %s\n" \
"https://tomeichlersmith.github.io/denv/getting_started.html#installation"
unset -f _error _user_confirm
return 1
fi
fi

if ! denv check --quiet; then
# maybe try loading apptainer via the 'module' command?
# https://github.com/LDMX-Software/ldmx-sw/issues/1248#issuecomment-1896618339
denv check
_error "'denv' unable to find a supported container runner." \
"Install one of the container runners 'denv' checked for above."
unset -f _error _user_confirm
return 2
fi

###############################################################################
# Attempt to deduce LDMX_BASE
# Unfortunately, deducing the path to a script being sourced is not possible
# using POSIX-compliant methods[1], so we need to fallback to separating
# different running scenarios.
#
# [1]: https://stackoverflow.com/a/29835459
#
# Users can avoid all of this complication by pre-defining the LDMX_BASE
# environment variable.
###############################################################################
if [ -z "${LDMX_BASE+x}" ]; then
_full_path() (
CDPATH=
cd -- "${1}" && pwd -P
)
_base_from_script() {
_full_path "$(dirname -- "${1}")/../../"
}
if [ -n "${ZSH_VERSION}" ]; then
# zsh defines ZSH_EVAL_CONTEXT holding the details on how
# this code is being executed
# disable warnings about undefined variable and string indexing
# shellcheck disable=SC2154,SC3057
LDMX_BASE="$(_base_from_script "${ZSH_EVAL_CONTEXT:file}")"
elif [ -n "${KSH_VERSION}" ]; then
# Korn defines the _ variable to the the file being sourced
# disable warning about undefined '_'
# shellcheck disable=SC3028
LDMX_BASE="$(_base_from_script "${_}")"
elif [ -n "${BASH_VERSION}" ]; then
# running from bash, bash defines the BASH_SOURCE array to help us
# find the location of this file
# disable warning about undefined variable and array references
# shellcheck disable=SC3028,SC3054
LDMX_BASE="$(_base_from_script "${BASH_SOURCE[0]}")"
else
# unable to deduce shell, resort to pwd
if expr "${PWD}" : ".*ldmx-sw$"; then
LDMX_BASE="$(_full_path ..)"
elif expr "${PWD}" : ".*ldmx-sw/scripts$"; then
LDMX_BASE="$(_full_path ../..)"
fi
fi
unset -f _full_path _base_from_script
fi
# re-export LDMX_BASE in case of user does inline variable definition like
# LDMX_BASE=/path/to/ldmx/base source /full/path/to/ldmx-denv-init.sh
export LDMX_BASE

###############################################################################
# Check if the LDMX denv is initialized. If not, do a default initialization.
# TODO: update to `denv check --workspace` introduced in denv v0.7.0 to avoid
# reliance on internal implementation on how denv stores its config
###############################################################################
if [ ! -f "${LDMX_BASE}/.denv/config" ]; then
denv init --clean-env --name "ldmx" "ldmx/dev:latest" "${LDMX_BASE}"
fi
unset -f _error _user_confirm
Loading

0 comments on commit 96526b4

Please sign in to comment.