gitlab ci: Print better information about broken specs #33124
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a pipeline generation job is automatically failed because it generated jobs for specs known to be broken on develop, print better information about the broken specs that were encountered. Include at a minimum the hash and the url of the job whose failure caused it to be put on the broken specs list in the first place.
spackbot-app
bot
added
commands
core
|
Same issue with the macos python 3.8 tests on this PR as I have been encountering on #32596 where I could never figure out what the actual failure was. On that PR, the test passed after the third or fourth retry, will try the same thing here. |
kwryankrattiger
approved these changes
Oct 13, 2022
haampie
reviewed
Oct 14, 2022
| tty.die(error_msg) | ||
| tty.error("This pipeline generated hashes known to be broken on develop:") | ||
| display_broken_spec_messages(broken_specs_url, known_broken_specs_encountered) | ||
| tty.die() |
There was a problem hiding this comment.
Yeah, sorry. My test didn't notice it because it has to pass allow_failure=True to the SpackCommand, since the command is expected to fail in this case and the expected text was in the output. I added a fix here since I was ready to merge that, but have had to retry the py27 and py38 tests more times on that PR. If that requires any more retries, I'll make a PR just for this.
luke-dt
added a commit
to dantaslab/spack
that referenced
this pull request
Oct 17, 2022
* py-execnet: 1.9.0 (spack#33282) * py-execnet: 1.9.0 * bounds * add version 1.12.3 of parallel-netcdf (spack#33286) * Add checksum for py-psutil 5.9.2 (spack#33139) * gitlab ci: Print better information about broken specs (spack#33124) When a pipeline generation job is automatically failed because it generated jobs for specs known to be broken on develop, print better information about the broken specs that were encountered. Include at a minimum the hash and the url of the job whose failure caused it to be put on the broken specs list in the first place. * meson: remove slash in path (spack#33292) * Add checksum for py-gitpython 3.1.27 (spack#33285) * Add checksum for py-gitpython 3.1.27 * Update package.py * Update var/spack/repos/builtin/packages/py-gitpython/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * UPC++/GASNet-EX 2022.9.0 update (spack#33277) * gasnet: Add new release hash * upcxx: Add new release hash * gasnet: misc updates * upcxx: misc updates * Fix pika@0.9.0 sha (spack#33307) * hip@5.2.0 onwards: set prefix properly (spack#33257) * hip-set-prefix-rocm5.2.0-onwards * Update var/spack/repos/builtin/packages/hip/package.py Update description Co-authored-by: Satish Balay <balay@mcs.anl.gov> * petsc,py-petsc4py,slepc,py-slepc4py: add version 3.18.0 (spack#32938) * petsc,py-petsc4py,slepc,py-slepc4py: add version 3.18.0 * workaround for dealii build failure [with petsc version check] * pism: add compatibility fix to for petsc@3.18 * add in hipsolver dependency * py-libensemble: updating package for v0.9.3 (spack#33298) * commit updating py-libensemble package for 0.9.3 * removed commented-out lines * Add checksum for py-oauthlib 3.2.1 (spack#33201) * Add checksum for py-oauthlib 3.2.1 * Update package.py * [@spackbot] updating style on behalf of iarspider * Update package.py * Update package.py Co-authored-by: iarspider <iarspider@users.noreply.github.com> * ninja: New version 1.11.1 (spack#33215) * seacas: update to latest release (spack#33330) Add checksum for latest tag/release * gptl: new version 8.1.1; use the correct mpi fortran compiler (spack#33235) * gptl: new version 8.1.1; use the correct `mpifc` * add `F90` and `$F77` * glib: add 2.74.0 and 2.72.4 (spack#33332) * depfile: update docs (spack#33279) * rocksdb: add 7.7.3 (spack#33341) * Add checksum for py-seaborn 0.12.0 (spack#33145) * Add checksum for py-seaborn 0.12.0 * Update package.py * Update package.py * [@spackbot] updating style on behalf of iarspider * Update package.py Co-authored-by: iarspider <iarspider@users.noreply.github.com> * CI: allow multiple matches to combine tags (spack#32290) Currently "spack ci generate" chooses the first matching entry in gitlab-ci:mappings to fill attributes for a generated build-job, requiring that the entire configuration matrix is listed out explicitly. This unfortunately causes significant problems in environments with large configuration spaces, for example the environment in spack#31598 (spack.yaml) supports 5 operating systems, 3 architectures and 130 packages with explicit size requirements, resulting in 1300 lines of configuration YAML. This patch adds a configuraiton option to the gitlab-ci schema called "match_behavior"; when it is set to "merge", all matching entries are applied in order to the final build-job, allowing a few entries to cover an entire matrix of configurations. The default for "match_behavior" is "first", which behaves as before this commit (only the runner attributes of the first match are used). In addition, match entries may now include a "remove-attributes" configuration, which allows matches to remove tags that have been aggregated by prior matches. This only makes sense to use with "match_behavior:merge". You can combine "runner-attributes" with "remove-attributes" to effectively override prior tags. * meson: update OneAPI compiler support patch (spack#33293) * py-tensorflow: fix zlib (spack#33349) * py-tensorflow: fix zlib * [@spackbot] updating style on behalf of haampie Co-authored-by: haampie <haampie@users.noreply.github.com> * py-meson-python: add new versions (spack#33294) * tasmanian: disable openmp by default (spack#33345) * octopus: upgrade to 12.1 (spack#33343) * py-sphinx: add v5.3 and v5.2 (spack#33356) * py-setuptools: add v65.5.0 (spack#33353) * libblastrampoline: Add versions 5.1.1, 5.2.0 (spack#33352) * nextflow: add v20.10.0 (spack#33354) * sdl2: add v2.0.22 and v2.24.1 (spack#33351) * mariadb-c-client: add 3.3.2, 3.2.7, 3.1.18, 3.0.10 (spack#33335) * py-tensorflow-hub: zlib, again. (spack#33359) * Add checksum for py-sniffio 1.3.0 (spack#32975) * py-numpy: add v1.23.4 (spack#33260) * py-jupyterlab-pygments: install from wheel to avoid cyclic dependency (spack#33278) * py-jupyterlab-pygments: avoid cyclic dependency * Fix style * Update package.py * Update package.py * [@spackbot] updating style on behalf of iarspider * Update package.py * Flake-8 * fix Co-authored-by: iarspider <iarspider@users.noreply.github.com> * installer.py: show timers for binary install (spack#33305) Print a message of the form ``` Fetch mm:ss. Build: mm:ss. Total: mm:ss ``` when installing from buildcache. Previously this only happened for source builds. * Add checksum for py-astroid 2.12.7, py-astroid 2.12.10, py-setuptools 62.6.0, py-wrapt 1.14.1, py-pylint 2.15.0 (spack#32976) * Add checksum for py-astroid 2.12.7, py-setuptools 62.6.0 * Also add checksum for py-wrapt * Update package.py * Update package.py (spack#57) * Update package.py * Update package.py * Update package.py * gitlab ci: Do not force protected build jobs to run on aws runners (spack#33314) * installer.py: fix/test get_deptypes (spack#33363) Fixing an oversight in spack#32537 `get_deptypes` should depend on new `package/dependencies_cache_only` props. * Add checksum for py-grpcio-tools 1.48.1 (spack#33358) * Add checksum for py-prompt-toolkit 3.0.31 (spack#33362) Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Jim Edwards <jedwards@ucar.edu> Co-authored-by: iarspider <iarspider@gmail.com> Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Dan Bonachea <dobonachea@lbl.gov> Co-authored-by: Auriane R <48684432+aurianer@users.noreply.github.com> Co-authored-by: eugeneswalker <38933153+eugeneswalker@users.noreply.github.com> Co-authored-by: Satish Balay <balay@mcs.anl.gov> Co-authored-by: John-Luke Navarro <33131245+jlnav@users.noreply.github.com> Co-authored-by: iarspider <iarspider@users.noreply.github.com> Co-authored-by: Erik Schnetter <schnetter@gmail.com> Co-authored-by: Greg Sjaardema <gsjaardema@gmail.com> Co-authored-by: WuK <i@wu-kan.cn> Co-authored-by: Michael Kuhn <michael.kuhn@ovgu.de> Co-authored-by: Jonathon Anderson <17242663+blue42u@users.noreply.github.com> Co-authored-by: haampie <haampie@users.noreply.github.com> Co-authored-by: Miroslav Stoyanov <30537612+mkstoyanov@users.noreply.github.com> Co-authored-by: Hans Fangohr <fangohr@users.noreply.github.com> Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com> Co-authored-by: Diego Alvarez <dialvarezs@gmail.com> Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a pipeline generation job is automatically failed because it generated jobs for specs known to be broken on develop, print better information about the broken specs that were encountered. Include at a minimum the hash and the url of the job whose failure caused it to be put on the broken specs list in the first place.