Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix rebuild check in bot/build.sh script, and always resume from the correct temporary directory #518

Merged
merged 8 commits into from
Mar 27, 2024

Conversation

bedroge
Copy link
Collaborator

@bedroge bedroge commented Mar 27, 2024

Due to the set -e, the grep command that checks for rebuild easystacks would cause the entire script to fail if there is nothing to be rebuilt. Fixed that by adding a true command to the end.

Also included some other fixes, e.g. one to make sure that the tarball step resumes from the right temp dir (either from the rebuild step or build step, depending on whether a rebuild was done or not).

Copy link

eessi-bot-aws bot commented Mar 27, 2024

Instance eessi-bot-mc-aws is configured to build:

  • arch x86_64/generic for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/generic for repo eessi-hpc.org-2023.06-software
  • arch x86_64/generic for repo eessi.io-2023.06-compat
  • arch x86_64/generic for repo eessi.io-2023.06-software
  • arch x86_64/intel/haswell for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/intel/haswell for repo eessi-hpc.org-2023.06-software
  • arch x86_64/intel/haswell for repo eessi.io-2023.06-compat
  • arch x86_64/intel/haswell for repo eessi.io-2023.06-software
  • arch x86_64/intel/skylake_avx512 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/intel/skylake_avx512 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/intel/skylake_avx512 for repo eessi.io-2023.06-compat
  • arch x86_64/intel/skylake_avx512 for repo eessi.io-2023.06-software
  • arch x86_64/amd/zen2 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/amd/zen2 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/amd/zen2 for repo eessi.io-2023.06-compat
  • arch x86_64/amd/zen2 for repo eessi.io-2023.06-software
  • arch x86_64/amd/zen3 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/amd/zen3 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/amd/zen3 for repo eessi.io-2023.06-compat
  • arch x86_64/amd/zen3 for repo eessi.io-2023.06-software
  • arch aarch64/generic for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/generic for repo eessi-hpc.org-2023.06-software
  • arch aarch64/generic for repo eessi.io-2023.06-compat
  • arch aarch64/generic for repo eessi.io-2023.06-software
  • arch aarch64/neoverse_n1 for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/neoverse_n1 for repo eessi-hpc.org-2023.06-software
  • arch aarch64/neoverse_n1 for repo eessi.io-2023.06-compat
  • arch aarch64/neoverse_n1 for repo eessi.io-2023.06-software
  • arch aarch64/neoverse_v1 for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/neoverse_v1 for repo eessi-hpc.org-2023.06-software
  • arch aarch64/neoverse_v1 for repo eessi.io-2023.06-compat
  • arch aarch64/neoverse_v1 for repo eessi.io-2023.06-software

bot/build.sh Outdated Show resolved Hide resolved
Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>
bot/build.sh Show resolved Hide resolved
@trz42
Copy link
Collaborator

trz42 commented Mar 27, 2024

PR has been extensively tested in #519

The only scenario that is currently not supported is when a single PR both rebuilds and adds some package. This can be fixed by the following (tested in #519 (comment))

diff --git a/EESSI-install-software.sh b/EESSI-install-software.sh
index 470663e..63b68a5 100755
--- a/EESSI-install-software.sh
+++ b/EESSI-install-software.sh
@@ -204,11 +204,14 @@ ${EESSI_PREFIX}/scripts/gpu_support/nvidia/install_cuda_host_injections.sh -c 12

 # use PR patch file to determine in which easystack files stuff was added
 changed_easystacks=$(cat ${pr_diff} | grep '^+++' | cut -f2 -d' ' | sed 's@^[a-z]/@@g' | grep '^easystacks/.*yml$' | egrep -v 'known-issues|missing')
-if [ -z ${changed_easystacks} ]; then
+if [ -z "${changed_easystacks}" ]; then
     echo "No missing installations, party time!"  # Ensure the bot report success, as there was nothing to be build here
 else

-    for easystack_file in ${changed_easystacks}; do
+    # first process rebuilds if any, then easystack files for new installations
+    rebuild_easystacks=$(echo "${changed_easystacks}" | grep /rebuilds/ || true)
+    new_easystacks=$(echo "${changed_easystacks}" | grep -v /rebuilds/ || true)
+    for easystack_file in ${rebuild_easystacks} ${new_easystacks}; do

         echo -e "Processing easystack file ${easystack_file}...\n\n"

or a similar change in #507

Copy link
Collaborator

@trz42 trz42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Tested extensively in #519. Issues discovered therein have been addressed here too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants