Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add HEP cloud_pipeline stack #40932

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0fe9023
dd4hep: depends_on root +root7 in some cases
wdconinc Apr 15, 2024
fd84014
dd4hep: avoid self-referential whens -> conflicts
wdconinc Apr 15, 2024
af9e682
feat: add HEP cloud_pipeline stack
wdconinc Nov 7, 2023
fea0c97
[hep] add hep jobs to .gitlab-ci.yml
wdconinc Nov 7, 2023
f92d5dc
[hep] rm hep-generate image override
wdconinc Nov 7, 2023
6a11f15
[hep] fix root variants; add some more packages
wdconinc Nov 7, 2023
db3b678
[hep] disable hep-build for now
wdconinc Nov 9, 2023
a01bf56
[hep] hepmc3 +interfaces
wdconinc Nov 10, 2023
4ce3379
[hep] hep-generate on gcc-11.4 runner image
wdconinc Nov 10, 2023
80c942f
[hep] sherpa -blackhat
wdconinc Nov 10, 2023
843ce64
[hep] align package requirements to e4s for more cache hits
wdconinc Nov 10, 2023
2896207
[hep] lcio -examples due to conflicts
wdconinc Nov 19, 2023
b6c26ff
[hep] marks geant4-data as external
wdconinc Nov 20, 2023
c05b0ee
[hep] enable build
wdconinc Nov 20, 2023
569cfac
[hep] elfutils now always has bzip2 xz dependency
wdconinc Apr 14, 2024
32e981e
[hep] avoid gaudi ^gdb@8.1 with explicit version gdb@14.1
wdconinc Apr 15, 2024
b4e0cea
[hep] geant4 -vtk for now
wdconinc Apr 15, 2024
9e89c7b
[hep] root ~oracle
wdconinc Apr 15, 2024
b018b0d
[hep] external geant4-data@11.2.0
wdconinc Apr 15, 2024
eea9209
[hep] sherap ~python
wdconinc Apr 15, 2024
f3512bc
[hep] gaudi ^gdb +python to avoid failing gdb@8.1
wdconinc Apr 15, 2024
d9d970a
[hep] pythia8 ~madgraph5amc
wdconinc Apr 15, 2024
203f549
[hep] remove garfieldpp which fails to download
wdconinc Apr 16, 2024
9d13d1e
[hep] sherpa ~recola because recola needs pic variant
wdconinc Apr 16, 2024
140df2d
[hep] sherpa ~rivet since conflicts("cxxstd=11", when="~rivet")
wdconinc Apr 17, 2024
9235771
[hep] unify: when_possible
wdconinc Apr 24, 2024
09a6545
[hep] thepeg ~rivet
wdconinc Apr 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 changes: 23 additions & 0 deletions share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml
Expand Up @@ -807,6 +807,29 @@ ml-darwin-aarch64-mps-build:
- artifacts: True
job: ml-darwin-aarch64-mps-generate

########################################
# High Energy Physics (HEP) - Linux x86_64 (CPU)
########################################
.hep:
extends: [ ".linux_x86_64_v3" ]
variables:
SPACK_CI_STACK_NAME: hep

hep-generate:
extends: [ ".hep", ".generate-x86_64"]
image: ghcr.io/spack/ubuntu20.04-runner-amd64-gcc-11.4:2023.08.01
wdconinc marked this conversation as resolved.
Show resolved Hide resolved

hep-build:
extends: [ ".hep", ".build" ]
trigger:
include:
- artifact: jobs_scratch_dir/cloud-ci-pipeline.yml
job: hep-generate
strategy: depend
needs:
- artifacts: True
job: hep-generate

########################################
# Deprecated CI testing
########################################
Expand Down
95 changes: 95 additions & 0 deletions share/spack/gitlab/cloud_pipelines/stacks/hep/spack.yaml
@@ -0,0 +1,95 @@
spack:
view: false

concretizer:
reuse: false
unify: when_possible

packages:
all:
require: '%gcc target=x86_64_v3'
providers:
blas: [openblas]
mpi: [mpich]
variants: +mpi
binutils:
variants: +ld +gold +headers +libiberty ~nls
elfutils:
variants: ~nls
gdb:
version: [14.1]
hdf5:
variants: +fortran +hl +shared
libfabric:
variants: fabrics=sockets,tcp,udp,rxm
libunwind:
variants: +pic +xz
openblas:
variants: threads=openmp
xz:
variants: +pic
mesa:
version: [21.3.8]
mpi:
require: mpich
mpich:
require: '~wrapperrpath ~hwloc'
ncurses:
require: '@6.3 +termlib'
tbb:
require: intel-tbb
boost:
version: [1.79.0]
variants: +atomic +chrono +container +date_time +exception +filesystem +graph
+iostreams +locale +log +math +mpi +multithreaded +program_options +random
+regex +serialization +shared +signals +stacktrace +system +test +thread +timer
cxxstd=17 visibility=global
libffi:
require: "@3.4.4"
cuda:
version: [11.8.0]

# Mark geant4 data as external to prevent wasting bandwidth on GB-scale files
geant4-data:
externals:
- spec: geant4-data@11.2.0
prefix: /usr
- spec: geant4-data@11.1.0
prefix: /usr
- spec: geant4-data@11.0.0
prefix: /usr

specs:
# CPU
- acts +analysis +binaries +dd4hep +edm4hep +examples +fatras +geant4 +hepmc3 +identification +podio +pythia8 +python +tgeo
- dd4hep +ddalign +ddcad +ddcond +dddetectors +dddigi +ddeve +ddg4 +ddrec +edm4hep +hepmc3 +lcio +utilityapps +xercesc
- delphes +pythia8
- edm4hep
- fastjet
- fjcontrib
- gaudi +aida +examples +heppdt +xercesc ^gdb +python
- geant4 +opengl +qt +threads ~vtk
- hepmc
- hepmc3 +interfaces +protobuf +python +rootio
- herwig3 +njet +vbfnlo
- lcio -examples +jar +rootdict # Note: lcio +examples ^ncurses -termlib, which leads to conflicts
- lhapdf +python
- madgraph5amc
- opendatadetector
- podio +rntuple +sio
- py-uproot +lz4 +xrootd +zstd
- pythia8 +evtgen +fastjet +hdf5 +hepmc +hepmc3 +lhapdf ~madgraph5amc +python +rivet -root # pythia8 and root circularly depend
- rivet hepmc=3
- root +davix +dcache +examples +fftw +fits +fortran +gdml +graphviz +gsl +http +math +minuit +mlp +mysql +opengl ~oracle +postgres +pythia6 +pythia8 +python +r +roofit +root7 +shadow +spectrum +sqlite +ssl +tbb +threads +tmva +unuran +vc +vdt +veccore +webgui +x +xml +xrootd
- sherpa +analysis -blackhat +fastjet +gzip +hepmc2 +hepmc3 +hepmc3root +hztool +lhapdf +lhole +openloops +pythia ~python ~recola ~rivet +root +ufo
- thepeg ~rivet
- vecgeom +gdml +geant4 +root
- xrootd +davix +http +krb5 +python +readline +scitokens-cpp

ci:
pipeline-gen:
- build-job:
image: "ghcr.io/spack/ubuntu20.04-runner-amd64-gcc-11.4:2023.08.01"

cdash:
build-group: HEP
4 changes: 4 additions & 0 deletions var/spack/repos/builtin/packages/dd4hep/package.py
Expand Up @@ -161,10 +161,14 @@ class Dd4hep(CMakePackage):
depends_on("boost +iostreams", when="+ddg4")
depends_on("boost +system +filesystem", when="%gcc@:7")
depends_on("root @6.08: +gdml +math +python")
depends_on("root @6.12.2: +root7", when="@1.26:") # DDCoreGraphics needs ROOT::ROOTHistDraw
with when("+ddeve"):
depends_on("root @6.08: +x +opengl")
depends_on("root @:6.27", when="@:1.23")
conflicts("^root ~webgui", when="^root@6.28:")
# For DD4hep >= 1.24, DDEve_Interface needs ROOT::ROOTGeomViewer if ROOT >= 6.27
conflicts("^root ~root7", when="@1.24: ^root@6.27:")
conflicts("^root ~webgui", when="@1.24: ^root@6.27:")
depends_on("root @6.08: +gdml +math +python +x +opengl", when="+utilityapps")

extends("python")
Expand Down