fix: concretizer: max_dupes: llvm: 1 - #376
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Debian base-image Spack configuration to treat LLVM as an external-only dependency by marking it not buildable during external discovery, helping prevent unintended extra LLVM builds that can increase container size.
Changes:
- Update
spack external findinvocation forllvmto use--not-buildable, aligning LLVM handling with other externals likegccandcuda.
wdconinc
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
spack-environment/tf/spack.yaml:1
- This file now starts with
external:instead of the required top-levelspack:key used by all other Spack environments in this repo. As written, this is not a valid Spack environment configuration and will breakspack env activate/ concretization for the tf environment.
spack:
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (3)
containers/debian/Dockerfile:296
- The LLVM external spec is being patched with yq before
spack external find --scope spack llvmruns later in the Dockerfile. Sincespack external findcan rewrite/refreshpackages.yaml, this ordering risks losing the+python targets=alltweak needed by downstream packages (e.g. py-tensorflow libclang bindings). Apply the yq patch immediately after the external is discovered (and guard it when no externals are present) so the final config is deterministic.
# Ensure LLVM externals satisfy iwyu variant requirements (targets=all)
yq -iy '.packages.llvm.externals[].spec += " +python targets=all"' ${SPACK_ROOT}/etc/spack/packages.yaml
spack config blame compilers
spack config blame packages
EOF
spack-environment/tf/spack.yaml:6
- This change removes the TensorFlow environment’s local
llvm: 2override so it inherits the new defaultllvm: 1. However,spack-environment/tf/epic/spack.yamlstill setsconcretizer:duplicates:max_dupes: llvm: 2, which contradicts the PR description’s claim that only the Julia environment overrides the default and may reintroduce duplicate LLVM in the TF epic environment.
include:
- ../concretizer.yaml
- ../config.yaml
- ../packages.yaml
- ../view.yaml
spack-environment/concretizer.yaml:19
- Now that
concretizer.yamldefines a default duplicate policy (llvm: 1), the header comment above is slightly misleading when it implies duplicate allowances live only in per-environmentspack.yamlfiles. Updating that comment will help future maintainers understand where defaults vs overrides belong.
duplicates:
max_dupes:
llvm: 1
Removed the concretizer section and updated the structure.
Add concretizer settings for LLVM duplicates in spack.yaml
wdconinc
force-pushed
the
llvm-buildable-false
branch
from
August 2, 2026 19:18
5b7e074 to
c210ea0
Compare
veprbl
approved these changes
Aug 3, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Briefly, what does this PR introduce? Please link to any relevant presentations or discussions.
This PR defines the default maximum instances of llvm in spack environments to be one, requiring the use of the external (system) llvm. This is only overridden in the Julia environment which requires a special llvm. In general, this avoids installing additional surprise llvm versions that bloat the container size in e.g. cuda, tf.
In order to make tensorflow possible with the external llvm-20 (from the point of view of spack), we ensure that the right system packages are installed and that spack indeed allows the clang python bindings to come from llvm installs.
What is the urgency of this PR?
What kind of change does this PR introduce?
Please check if any of the following apply