Improve hit-rate on buildcaches #43272
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spackbot-app
bot
added
core
Smart. On second thought: does it work with externals? |
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
06c0c69
to
db6f8f9
Compare
haampie
reviewed
Mar 20, 2024
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
db6f8f9
to
2379050
Compare
|
Example of the current state (the occasional |
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
630a5fe
to
5c5de4d
Compare
|
The changes here give a much better hit rate on store / buildcaches. I tried to benchmark on a baseline without any mirror and it's mostly the same. There's one consistent spike I need to investigate further: radiuss.pr.nomirror.csv
|
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
5c5de4d
to
ed006df
Compare
alalazo
commented
Mar 25, 2024
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
ed006df
to
f550c55
Compare
The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned to the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority.
We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime.
This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers
This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
This is to avoid differences on m1 with respect to x86_64 machines
This is to avoid differences on m1 with respect to x86_64
alalazo
force-pushed
the
solver/relax-os-and-target-matching
branch
from
5376bef
to
523c704
Compare
|
I just tried to re-measure the benchmark above. There must be something in the
radiuss.pr.nomirror.csv
Footnotes
|
alalazo
changed the title
haampie
reviewed
Apr 5, 2024
haampie
reviewed
Apr 5, 2024
|
I think this is fine. As per discussion in dm, users can always But we have to document it (in high impact changes for instance). |
haampie
approved these changes
Apr 5, 2024
tldahlgren
pushed a commit
to tldahlgren/spack
that referenced
this pull request
Apr 23, 2024
* Relax compiler and target mismatches The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority. * Optimize version badness for runtimes at very low priority We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime. * Optimize runtime attributes at very low priority This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers * Exclude specs without runtimes from reuse This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
teaguesterling
pushed a commit
to teaguesterling/spack
that referenced
this pull request
Jun 15, 2024
* Relax compiler and target mismatches The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority. * Optimize version badness for runtimes at very low priority We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime. * Optimize runtime attributes at very low priority This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers * Exclude specs without runtimes from reuse This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR tries to modify the optimization criteria to have more hits on buildcaches by default.
Modifications:
gcc-runtimeto be reused (they'll be mostly broken at runtime)