Skip to content

Commit

Permalink
Add spack-based workflow to add all supported Geant4 versions to CI (#…
Browse files Browse the repository at this point in the history
…1149)

* Add note about pull requests
* Show ccache stats even on failure
* Add spack-based CI build
* Add spack build to push/pr
* Mark failing builds as experimental
* Fix missing compiler/version from matrix
* Use patched geant4 in upstream spack
* Use only celer buildcache
* Shorten padded length
* Don't push the buildcache...
* Load version information to fix examples
  • Loading branch information
sethrj committed Mar 18, 2024
1 parent d600808 commit 63821fb
Show file tree
Hide file tree
Showing 8 changed files with 194 additions and 6 deletions.
1 change: 1 addition & 0 deletions .github/workflows/build-fast.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ jobs:
run: |
./bin/celer-sim --version
- name: Show ccache stats
if: ${{!cancelled()}}
run: |
ccache -s
Expand Down
143 changes: 143 additions & 0 deletions .github/workflows/build-spack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# Build directly on the GitHub runner with caching
name: build-spack
on:
workflow_dispatch:
workflow_call:

concurrency:
group: build-spack-${{github.ref}}-${{github.event.pull_request.number || github.run_number}}-${{github.workflow}}

env:
SPACK_REF: 968ad02473b12f6305cc1fe19f2a0d706f171154

jobs:
linux:
name: jammy-g4-${{matrix.geant}}
strategy:
matrix:
compiler: [clang]
version: [15]
geant: ["10.5", "10.6", "10.7", "11.1", "11.2"]
experimental: [true]
include:
- geant: "11.0"
experimental: false
compiler: clang
version: 15
continue-on-error: ${{matrix.experimental}}
runs-on: ubuntu-22.04
permissions:
packages: write
env:
CCACHE_DIR: "${{github.workspace}}/.ccache"
CCACHE_MAXSIZE: "1G"
CELERITAS_VERSION: "0.5.0"
SPACK_VIEW: "/opt/spack-view"
SPACK_BUILDCACHE: "celer-buildcache" # see spack.yaml
CC: ${{matrix.compiler}}-${{matrix.version}}
CXX: ${{matrix.compiler == 'gcc' && 'g++' || 'clang++'}}-${{matrix.version}}
steps:
- name: Check out Celeritas
uses: actions/checkout@v4
with:
fetch-depth: 255
fetch-tags: true # to get version information
- name: Setup Spack
uses: spack/setup-spack@5ab3c91bdefffffad9a7e45d1d156146afebb3a7
with:
ref: ${{env.SPACK_REF}}
buildcache: true
color: true
path: spack-src
- name: Initialize spack environment
run: |
cp scripts/ci/spack.yaml .
spack -e . add geant4@${{matrix.geant}}
spack -e . config add packages:all:require:"'%${{matrix.compiler}}@${{matrix.version}} target=x86_64_v3'"
spack -e . compiler find
# Add the spack ref so that updating spack will reconcretize
echo "# Concretized with ${{env.SPACK_REF}}" >> spack.yaml
- name: Cache concretization
uses: actions/cache@v4
with:
path: spack.lock
key: lock-${{hashFiles('spack.yaml')}}-${{github.run_id}}
restore-keys: |
lock-${{hashFiles('spack.yaml')}}
- name: Concretize
run: |
spack -e . -v concretize
- name: Install dependencies with Spack
run: |
spack -e . env depfile -o Makefile
make -Orecurse -j $(($(nproc) + 1)) SPACK_INSTALL_FLAGS=--no-check-signature
- name: Save Spack installation to build cache
run: |
spack -e . mirror set --push --oci-username ${{github.actor}} --oci-password "${{secrets.GITHUB_TOKEN}}" ${SPACK_BUILDCACHE}
spack -e . buildcache push -j $(($(nproc) + 1)) --base-image ubuntu:22.04 --unsigned --update-index ${SPACK_BUILDCACHE}
if: ${{github.event_name == 'push' && !cancelled()}}
- name: Activate environment
run: |
test -d "${SPACK_VIEW}"
echo "${SPACK_VIEW}/bin" >> $GITHUB_PATH
echo "CMAKE_PREFIX_PATH=${SPACK_VIEW}:${CMAKE_PREFIX_PATH}" >> $GITHUB_ENV
spack env activate . --sh > "${SPACK_VIEW}/rc"
- name: Cache ccache
uses: actions/cache@v4
with:
path: ${{env.CCACHE_DIR}}
key: ccache-spack-${{matrix.compiler}}-${{matrix.version}}-${{matrix.geant}}-${{github.run_id}}
restore-keys: |
ccache-spack-${{matrix.compiler}}-${{matrix.version}}-${{matrix.geant}}-
ccache-spack-${{matrix.compiler}}-${{matrix.version}}-
- name: Zero ccache stats
run: |
ccache -z
- name: Configure Celeritas
run: |
mkdir build && cd build
cmake -GNinja --log-level=VERBOSE \
-DCELERITAS_BUILD_DEMOS:BOOL=ON \
-DCELERITAS_BUILD_TESTS:BOOL=ON \
-DCELERITAS_USE_SWIG=OFF \
-DCELERITAS_USE_Geant4=ON \
-DCELERITAS_DEBUG:BOOL=ON \
-DCMAKE_BUILD_TYPE="Release" \
-DCMAKE_INSTALL_PREFIX="${{github.workspace}}/install" \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_FLAGS="-Werror -Wall -Wextra -pedantic -Wno-error=deprecated-declarations" \
..
- name: Build all
working-directory: build
run: |
ninja -v -k0
- name: Run tests
working-directory: build
continue-on-error: true # TODO: disable or fix failing tests
run: |
ctest --parallel $(nproc) --timeout 15 --output-on-failure \
--test-output-size-passed=32768 --test-output-size-failed=1048576
- name: Install
working-directory: build
run: |
ninja install
- name: Check installation
working-directory: install
run: |
for exe in orange-update celer-export-geant celer-dump-data \
celer-sim celer-g4; do
test -x "bin/${exe}"
done
./bin/celer-sim --version
- name: Build examples
run: |
. ${SPACK_VIEW}/rc
CMAKE_PRESET=spack-vecgeom ./scripts/ci/test-examples.sh
- name: Show ccache stats
if: ${{!cancelled()}}
run: |
ccache -s
# vim: set nowrap tw=100:
12 changes: 8 additions & 4 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,24 @@ jobs:
steps:
- name: Success
run: "true"
build:
build-full:
needs: [all-prechecks]
uses: ./.github/workflows/build-full.yml
build-spack:
needs: [all-prechecks]
uses: ./.github/workflows/build-spack.yml

# Specifying a dependent job allows us to select a single "requires" check in the project GitHub settings
all:
if: ${{ always() }}
if: ${{always()}}
needs:
- build
- build-full
- build-spack
runs-on: ubuntu-latest
steps:
- name: Decide whether the needed jobs succeeded or failed
uses: re-actors/alls-green@release/v1
with:
jobs: ${{ toJSON(needs) }}
jobs: ${{toJSON(needs)}}

# vim: set nowrap tw=100:
4 changes: 3 additions & 1 deletion .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@ jobs:
uses: ./.github/workflows/build-fast.yml
build-full:
uses: ./.github/workflows/build-full.yml
build-spack:
uses: ./.github/workflows/build-spack.yml
doc:
uses: ./.github/workflows/doc.yml
all:
needs: [build-fast, build-full, doc]
needs: [build-fast, build-full, build-spack, doc]
runs-on: ubuntu-latest
steps:
- name: Success
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ compile_commands.json
cmake_install.cmake
/CMakeCache.txt
/CMakeFiles
/spack.yaml
8 changes: 7 additions & 1 deletion CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,13 @@ Submitting code changes

When you believe that you've made a substantive [#]_ and self-contained
improvement to the code, it's time to create a `pull request`_ (PR) to get
feedback on your changes before they're merged into the code base.
feedback on your changes before they're merged into the code base. The pull
request should be as close to a "single change" as possible (i.e., the short
pull request title can essentially describe the entire change set), typically
a few hundred lines (not including tests and test data). A pull request could
be as small as a single line for a bug fix. If your changes involve both
substantial refactoring and new features, try to split the refactoring into a
separate commit.

Before opening the pull request, check that the :ref:`code <code_guidelines>`
and :ref:`style <style_guidelines>` guidelines have been followed for all new
Expand Down
30 changes: 30 additions & 0 deletions scripts/ci/spack.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Note that the Geant4 spec is added by the CI script.
# Most of these specs should correspond to
# https://github.com/celeritas-project/spack-gha-buildcache/blob/main/spack.yaml
spack:
view: /opt/spack-view
specs:
- ccache
- cmake
- googletest
- hepmc3
- libtree
- ninja
- nlohmann-json
- root
- vecgeom
config:
install_tree:
root: /opt/spack
padded_length: 116
projections:
all: "{name}/{version}/{hash:7}"
mirrors:
celer-buildcache:
url: oci://ghcr.io/celeritas-project/spack-gha-buildcache
signed: false
packages:
root:
variants: ~aqua ~davix ~examples ~opengl ~x ~tbb cxxstd=17
all:
variants: cxxstd=17
1 change: 1 addition & 0 deletions scripts/ci/test-examples.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ test -n "${CMAKE_PRESET}" || (
build_local() {
git clean -fxd .
EXAMPLE_INSTALL=${PWD}/install
printf "\e[1;32]mTesting in ${PWD}\e[m\n"
mkdir build
cd build
cmake -G Ninja \
Expand Down

0 comments on commit 63821fb

Please sign in to comment.