Merge from upstream 2024-04-24 #332
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fixes spack#43454 by exposing a threadsafe variant * [@spackbot] updating style on behalf of Chrismarsh * fix style --------- Co-authored-by: Chrismarsh <Chrismarsh@users.noreply.github.com>
* Add MPI and arch bugfixes to SCOTCH * Update scotch/package.py
If ONEAPI_ROOT is not set as an environment variable, the current approach will raise an error. Instead we can compute the OneAPI_ROOT from the compiler paths like we do with vcvarsall.
The installer runs `get_dependent_ids`, which follows edges outside the subdag that's being installed, so it returns a superset of the actual dependents. That's generally fine, except that it calls `s.package` on every dependent, which triggers a package class to be instantiated, which is a lot of work. Instead, compute the package id from the spec, since that's all that's used anyways and does not trigger *lots* of slow and redundant instantiations of package objects.
* Relax compiler and target mismatches The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority. * Optimize version badness for runtimes at very low priority We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime. * Optimize runtime attributes at very low priority This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers * Exclude specs without runtimes from reuse This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
* updating package.py for py-celery, py-kombu, py-amq
* added more py-kombu package versions
* fix copyrights and stype on py-kombu/package.py
* removed extra spaces
* added py-billiard 4.2.0 and added back the license('BSD-3-Clause')
* removed extra spaces in py-celery/package.py
* fixed py-amqp 2.4.0 sha; fixed py-celery's dependency of py-click (when version restrictions)
* more clean up on specifying version bounds
* Adding comgr as dependency * adding more smoke test deps
* CI: Add debug to the log aggregation script
…m-opencl (spack#43444) * For avx build, the start address of values_ buffer in KernelParameters is not a correct as it is computed based on 16-byte alignment. * Style check error fix
* Add new versions of tinyxml2 Added 7.0.0 to 10.0.0 * Add the variant "shared"
* e4s ci: add nekbone, nek5000 * remove nek5000
* py-deephyper: add latest version: v0.6.0 * e4s: add py-deephyper * v0.6.0: depend on python@3.7:3.11 * add py-packaging constraint so arm64 builds work * [@spackbot] updating style on behalf of eugeneswalker
* ParaView: Update version 5.12.0 Add 5.12.0 release Update default to 5.12.0 * Add patch for building ParaView 5.12 with kits * Drop VTKm from neoverse
* upgrade new versions * style fix * update jaxlib deps (not cuda and bazel yet) * update jaxlib cuda versions * update jaxlib cuda versions * update jaxlib cuda versions * chore: style fix * Update package.py * Update package.py * fix: typo * docs: add source for cuda version * py-jaxlib 0.4.14 also doesn't build on ppc64le * Add 0.4.26 --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
The reverse provider lookup may have stale entries for deleted packages, which used to cause errors. It's hard to invalidate those cache entries, so this commit simply drops entries w/o invalidating the cache. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Tags: add more build tools * py-pythran: add maintainer
* Adding UCX 1.16.0 * Fixed hash
When looking at where we spend our time in solver setup, I noticed a fair bit of time is spent
in `Spec.format()`, and `Spec.format()` is a pretty old, slow, convoluted method.
This PR does a number of things:
- [x] Consolidate most of what was being done manually with a character loop and several
regexes into a single regex.
- [x] Precompile regexes where we keep them
- [x] Remove the `transform=` argument to `Spec.format()` which was only used in one
place in the code (modules) to uppercase env var names, but added a lot of complexity
- [x] Avoid escaping and colorizing specs unless necessary
- [x] Refactor a lot of the colorization logic to avoid unnecessary object construction
- [x] Add type hints and remove some spots in the code where we were using nonexistent
arguments to `format()`.
- [x] Add trivial cases to `__str__` in `VariantMap` and `VersionList` to avoid sorting
- [x] Avoid calling `isinstance()` in the main loop of `Spec.format()`
- [x] Don't bother constructing a `string` representation for the result of `_prev_version`
as it is only used for comparisons.
In my timings (on all the specs formatted in a solve of `hdf5`), this is over 2.67x faster than the
original `format()`, and it seems to reduce setup time by around a second (for `hdf5`).
The default runner changed on GitHub for macOS, and that revealed a bug in a package when running audits
If there's no compiler we currently don't have any external libc for the solver. This commit adds a fallback on libc from the current Python process, which works if it is dynamically linked. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Add `%oneapi`, remove compilers that have been discontinued upstream
test dependencies constrains build / link type deps, so avoid that
…k#43459)" (spack#43819) This reverts commit 9e2558b.
Some packages can't be redistributed in source or binary form. We need an explicit way to say that in a package.
This adds a `redistribute()` directive so that package authors can write, e.g.:
```python
redistribute(source=False, binary=False)
```
You can also do this conditionally with `when=`, as with other directives, e.g.:
```python
# 12.0 and higher are proprietary
redistribute(source=False, binary=False, when="@12.0:")
# can't redistribute when we depend on some proprietary dependency
redistribute(source=False, binary=False, when="^proprietary-dependency")
```
To prevent Spack from adding either their sources or binaries to public mirrors and build caches. You can still unconditionally add things *if* you run either:
* `spack mirror create --private`
* `spack buildcache push --private`
But the default behavior for build caches is not to include non-redistributable packages in either mirrors or build caches. We have previously done this manually for our public buildcache, but with this we can start maintaining redistributability directly in packages.
Caveats: currently the default for `redistribute()` is `True` for both `source` and `binary`, and you can only set either of them to `False` via this directive.
- [x] add `redistribute()` directive
- [x] add `redistribute_source` and `redistribute_binary` class methods to `PackageBase`
- [x] add `--private` option to `spack mirror`
- [x] add `--private` option to `spack buildcache push`
- [x] test exclusion of packages from source mirror (both as a root and as a dependency)
- [x] test exclusion of packages from binary mirror (both as a root and as a dependency)
* fix so trilinos@master uses correct kokkos (@4.3.00) * Update var/spack/repos/builtin/packages/trilinos/package.py
If a flang-new exists, which is rather unlikely, it probably means the user wants it as a fortran compiler.
knoepfel
approved these changes
Apr 24, 2024
greenc-FNAL
force-pushed
the
maintenance/merge-from-upstream-2024-04-24
branch
from
d247544
to
df87319
Compare
…e-from-upstream-2024-04-24
greenc-FNAL
force-pushed
the
maintenance/merge-from-upstream-2024-04-24
branch
from
df87319
to
742e8c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
when_possible: add failure detection and explicit message (Concretizewhen_possible: add failure detection and explicit message spack/spack#43202)plugins(fastjet: new multi-valued variantpluginsspack/spack#43523)spack findoutput in environments (Improvespack findoutput in environments spack/spack#42334)platform=on externals spack/spack#43758)import distrooutside of internals spack/spack#43772)reuse:default to True (concretizer: updatereuse:default to True spack/spack#41302)spec formatspeed (Refactor to improvespec formatspeed spack/spack#43712)redistribute()directive (Add newredistribute()directive spack/spack#20185)