build-eic: support comma-separated --build-type (default,nightly)#252
Merged
wdconinc merged 2 commits intocopilot/extract-build-scriptsfrom Apr 23, 2026
Merged
Conversation
Build both default and nightly images in a single sequential invocation
so that the four shared Docker stages (builder/runtime concretization and
installation of the default spack environment) are reused from BuildKit's
layer cache. Cuts redundant Spack compilation for nightly builds that
follow a default build on the same runner.
Key changes:
- build-eic.sh: change BUILD_TYPE default to "default,nightly"; split on
comma; validate each token; loop over types – resolving EIC package SHAs
and constructing the docker-buildx command per iteration. Shared setup
(benchmark SHAs, mirrors.yaml, SPACK_DUPLICATE_ALLOWLIST, ARCH) runs once
before the loop. Metadata files are written as
${METADATA_FILE%.json}-<build_type>.json; logs as build-<build_type>.log.
- .gitlab-ci.yml: restructure eic parallel matrix from 16 to 10 jobs by
using BUILD_TYPE="default,nightly" as a plain string (not a list dimension)
for environments that previously had separate default and nightly rows.
Drop ${BUILD_TYPE} from METADATA_FILE (script now appends it). Update
.build artifacts glob to build-*.log. Fix .nightly rules so that combined
jobs override BUILD_TYPE to "default" on stable branches (instead of
matching only the now-absent BUILD_TYPE=="nightly" string), while
nightly-only jobs (ci_without_acts) retain when:never semantics.
- .github/workflows/build-push.yml: set BUILD_TYPE="default,nightly" for
ci and xl matrix entries. Export one digest file per build type and
upload as separate artifacts. Add BUILD_TYPE dimension to eic-manifest
matrix; update artifact download pattern; add build-type-aware tags.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the container build tooling to support building multiple EIC image variants (default + nightly) in a single build-eic.sh invocation, enabling better BuildKit cache reuse and reducing CI job fan-out across GitLab CI and GitHub Actions.
Changes:
- Extend
build-eic.shto accept a comma-separated--build-typelist and build each type sequentially with per-type logs and metadata files. - Restructure
.gitlab-ci.ymlEIC build matrix to use combinedBUILD_TYPE: "default,nightly"where appropriate, reducing parallel jobs and updating artifact collection. - Update
.github/workflows/build-push.ymlto build combined types, export/upload per-type digest artifacts, and push per-type manifests/tags.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
build-eic.sh |
Adds comma-separated build-type parsing and loops builds per type, generating per-type logs/metadata. |
.gitlab-ci.yml |
Reduces matrix size via combined build types, adjusts nightly rules, and changes artifacts globbing. |
.github/workflows/build-push.yml |
Exports/upload per-type digest artifacts and pushes manifests per build type. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extend
build-eic.shto accept a comma-separated--build-typeargument (default:"default,nightly") so both image variants can be built in a single invocation.Motivation
When default and nightly builds run sequentially on the same runner, BuildKit's layer cache already contains the four shared stages after the default build:
builder_concretization_defaultbuilder_installation_default(most expensive: full Spack compile)runtime_concretization_defaultruntime_installation_defaultThe nightly build reuses these without re-executing Spack compilation or pulling large layers from the registry.
Additional savings: 7 benchmark/campaign SHA resolutions and
mirrors.yamlgeneration happen once instead of twice.Changes
build-eic.shBUILD_TYPEchanged from"default"to"default,nightly"mirrors.yaml,SPACK_DUPLICATE_ALLOWLIST,ARCH) runs once before the loopdocker buildxcommand; writesbuild-${build_type}.logand${METADATA_FILE%.json}-${build_type}.json.gitlab-ci.ymleicparallel matrix from 16 → 10 jobs by usingBUILD_TYPE: "default,nightly"as a plain string (not a list dimension)${BUILD_TYPE}fromMETADATA_FILEtemplate (script appends it).buildartifacts glob tobuild-*.log.nightlyrules: combined jobs overrideBUILD_TYPEto"default"on stable branches; nightly-only jobs (ci_without_acts) keepwhen: never.github/workflows/build-push.ymlBUILD_TYPE: "default,nightly"forciandxlmatrix entries (GitHub Actions now also builds nightly)BUILD_TYPE: [default, nightly]dimension toeic-manifestmatrix; updates artifact download pattern and tags