Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use (but don't require) denv in ldmx env #1269

Draft
wants to merge 9 commits into
base: trunk
Choose a base branch
from
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ endif()

# If a user is building outside of a Docker or Singularity environment,
# warn them.
if (NOT EXISTS /.dockerenv AND NOT EXISTS /singularity)
if (NOT EXISTS /.dockerenv AND NOT EXISTS /singularity AND NOT EXISTS /run/.containerenv)
message(WARNING "You are not inside a container; you may be working in an untested environment.")
endif()

Expand Down
30 changes: 30 additions & 0 deletions scripts/.shellcheckrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Overrides the shell detected from the shebang.
# This is useful for files meant to be included (and thus lacking a shebang),
# or possibly as a more targeted alternative to 'disable=SC2039'.
shell=sh
# You can override this configuration on the command line with --shell

# Always allow ShellCheck to open arbitrary files from 'source' statements.
external-sources=true
check-sourced=true

# Enable all optional checks
enable=all

# We want the most picky shellcheck so we have it even flag style errors
severity=style

# This function is invoked in an 'if' condition so set -e will be disabled.
# Invoke separately if failures should cause the script to exit.
# - We don't want to exit if errors happen inside a check, that's why we have a check...
disable=SC2310

# Bash implicitly disabled set -e for this function invocation because
# it's inside a command substitution. Add set -e; before it or enable inherit_errexit.
# - Don't care if we inherit errexit inside substitutions, we do checks for that.
disable=SC2311

# Consider invoking this command separately to avoid masking its return value
# (or use '|| true' to ignore).
# - We already check errors and adding "|| true" everywhere hinders readability.
disable=SC2312
88 changes: 88 additions & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# LDMX Environment Initialization

The LDMX development and running environment is defined as a container image.
Running this container image properly can be complicated on many systems,
so we've isolated the running of this container into shell scripts so that
the average user/developer does not need to worry about the intricacies
of container running.

## Legacy Solution
The first development of this solution defined a series of functions in
`bash` that wrapped `docker` and `singularity` and focused on running the
image as built by [LDMX-Software/docker](https://github.com/LDMX-Software/docker).
This solution, while functional, has a few edge-case issues and is difficult
to test independently of ldmx-sw.
These issues motivated the development of a separate project which we can
now use for development of ldmx-sw.

## denv
[denv](https://tomeichlersmith.github.io/denv/)
is a project to generalize the original bash functions to support
more shells, more systems, and more container runners. With a ground-up
redesign, it has benefited from lessons learned while using the original
bash functions and, as a independent project, can be thoroughly tested.

Utilizing `denv` to develop and run ldmx-sw is a simpler task than
trying to use the container runners directly; however, it is a bit
more complicated than the original bash functions due to its generality.
This makes defining wrapping environment scripts a helpful tool for
easing the burden placed on any users. Below, the LDMX denv configuration
is documented to help users understand what is happening as well as
help anyone interested in developing a wrapping script to be used
within the shell of their choice.

### Initialization
In order to properly mimic the environment as defined by the `ldmx`
suite of bash functions, we need to initialize a denv with a few
options.
```
denv \
init \ # use denv special sub-command for initialization
--clean-env \ # avoid passing environment variables
--name "ldmx" \ # name this denv after our experiment :)
"ldmx/dev:latest" \ # use the dev image
<path-to-ldmx-base> # path to directory containing ldmx-sw
```
For example, if you are in the ldmx-sw directory,
```
denv init --clean-env --name ldmx ldmx/dev:latest ..
```
The default path is the current directory, so you could also init immediately
after cloning ldmx-sw.
```
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git
denv init --clean-env --name ldmx ldmx/dev:latest
```
The default `--name` of a denv is the directory where `init` was run,
so the final example (and the one I use on most of my machines) is
```
mkdir ldmx
cd ldmx
git clone --recursive git@github.com:ldmx-software/ldmx-sw.git
denv init --clean-env ldmx/dev:latest
```

### Wrapping
Wrapping `denv` can have the benefit of reducing the amount of characters
needed for the average user to type when doing common tasks. Environment
scripts wrapping denv for a specific shell should do the following.

- Deduce `LDMX_BASE` and define it as an environment variable for the user
- Make sure the LDMX denv is initialized for the user
- Define the `ldmx` function with CLI defined below

### `ldmx` CLI Specification
v0.0.0: just an idea

- `ldmx`
- `help` : print help message describing the CLI
- `config` : print shell version being used and run `denv config print`
- `use` : first argument is repo (`dev` or `pro`) and then second argument is tag (e.g. `latest`)
- deduce full docker image tag name from repo and tag and then if expand the tag to include
the path on `/cvmfs` (if available) or just the DockerHub tag (if not)
- `pull` : call `ldmx use` and then `denv config image pull`
- `mount` : pass arguments to `denv config mounts`
- `setenv` : pass arguments to `denv config env copy`
- `compile` : pass arguments to `denv ${LDMX_BASE}/ldmx-sw/ldmx-compile.sh`
- `recompFire` : pass arguments to `denv ${LDMX_BASE}/ldmx-sw/ldmx-recompFire.sh`
- `<cmd>` : pass directly to `denv`
17 changes: 12 additions & 5 deletions scripts/ldmx-compile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,23 @@
# or
# ldmx setenv LDMX_COMPILE_BUILD=<build location>

if [ -z "$LDMX_COMPILE_CORES" ]; then
if [[ -z "${LDMX_COMPILE_CORES}" ]]; then
LDMX_COMPILE_CORES=$(nproc)
fi

if [ -z "$LDMX_COMPILE_BUILD" ]; then
LDMX_COMPILE_BUILD=${LDMX_BASE}/ldmx-sw/
if [[ -z "${LDMX_COMPILE_BUILD}" ]]; then
if [[ -z "${LDMX_BASE}" ]]; then
echo "ERROR: LDMX_BASE is undefined." \
" This script should be run within the containerized environment."
exit 1
fi
LDMX_COMPILE_BUILD="${LDMX_BASE}/ldmx-sw"
fi

echo "-- Compiling ldmx-sw in ${LDMX_COMPILE_BUILD} with ${LDMX_COMPILE_CORES} cores"

# Compile ldmx-sw
cmake -B ${LDMX_COMPILE_BUILD}/build -S ${LDMX_COMPILE_BUILD} $@
cmake --build ${LDMX_COMPILE_BUILD}/build --target install -j=${LDMX_COMPILE_CORES}
# any arguments are passed to cmake to configure the build
# shellcheck disable=SC2068
cmake -B "${LDMX_COMPILE_BUILD}/build" -S "${LDMX_COMPILE_BUILD}" $@
cmake --build "${LDMX_COMPILE_BUILD}/build" --target install -j="${LDMX_COMPILE_CORES}"
Loading