Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310

Merged

Conversation

trz42
Copy link
Collaborator

@trz42 trz42 commented Apr 7, 2024

This PR adds parallel v20230722. It also serves as a test for #309

SPDX license identifier: GPL-3.0-or-later

Missing installations:

1 out of 3 required modules missing:

* parallel/20230722-GCCcore-12.3.0 (parallel-20230722-GCCcore-12.3.0.eb)

We process this PR as follows:

  1. We build parallel and verify if the tarballs do no longer include the Lmod cache updates.
  2. If so, we merge remove Lmod cache update #309 first, and update this PR here.
  3. Finally, we deploy the built tarballs and verify if the Lmod caches are updated on the Stratum 0 during the ingestion.

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Instance AWS-MC-NESSI is configured to build:

  • arch x86_64/generic for repo nessi-2022.11-swl-deb10
  • arch x86_64/generic for repo nessi-2023.06-cl
  • arch x86_64/generic for repo nessi-2023.06-swl-deb10
  • arch x86_64/generic for repo nessi-2023.06-swl-deb11
  • arch x86_64/intel/skylake_avx512 for repo nessi-2022.11-swl-deb10
  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-cl
  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb10
  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb11
  • arch x86_64/amd/zen2 for repo nessi-2022.11-swl-deb10
  • arch x86_64/amd/zen2 for repo nessi-2023.06-cl
  • arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb10
  • arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb11
  • arch aarch64/generic for repo nessi-2022.11-swl-deb10
  • arch aarch64/generic for repo nessi-2023.06-cl
  • arch aarch64/generic for repo nessi-2023.06-swl-deb10
  • arch aarch64/generic for repo nessi-2023.06-swl-deb11

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Instance eX3-NESSI is configured to build:

  • arch x86_64/amd/zen2 for repo nessi-2023.06-cl
  • arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb10
  • arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb11
  • arch aarch64/generic for repo nessi-2023.06-cl
  • arch aarch64/generic for repo nessi-2023.06-swl-deb10
  • arch aarch64/generic for repo nessi-2023.06-swl-deb11

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Instance Fram-NESSI is configured to build:

  • arch x86_64/generic for repo nessi-2023.06-cl
  • arch x86_64/generic for repo nessi-2023.06-swl-deb10
  • arch x86_64/generic for repo nessi-2023.06-swl-deb11
  • arch x86_64/intel/broadwell for repo nessi-2023.06-cl
  • arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb10
  • arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb11

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Instance Saga-NESSI is configured to build:

  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-cl
  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb10
  • arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb11
  • arch x86_64/intel/broadwell for repo nessi-2023.06-cl
  • arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb10
  • arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb11
  • arch x86_64/generic for repo nessi-2023.06-cl
  • arch x86_64/generic for repo nessi-2023.06-swl-deb10
  • arch x86_64/generic for repo nessi-2023.06-swl-deb11

@trz42 trz42 mentioned this pull request Apr 7, 2024
@trz42
Copy link
Collaborator Author

trz42 commented Apr 7, 2024

bot: build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell
bot: build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake
bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Updates by the bot instance AWS-MC-NESSI (click for details)
  • received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42

    • expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
  • received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42

    • expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
  • received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42

    • expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
  • handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:

    • no jobs were submitted
  • handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:

    • no jobs were submitted
  • handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:

  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Updates by the bot instance Saga-NESSI (click for details)
  • received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42

    • expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
  • received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42

    • expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
  • received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42

    • expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
  • handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:

    • no jobs were submitted
  • handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:

  • handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:

    • no jobs were submitted

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Updates by the bot instance eX3-NESSI (click for details)
  • received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42

    • expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
  • received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42

    • expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
  • received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42

    • expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
  • handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:

    • no jobs were submitted
  • handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:

    • no jobs were submitted
  • handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:

  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:

    • no jobs were submitted

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

Updates by the bot instance Fram-NESSI (click for details)
  • received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42

    • expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
  • received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42

    • expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
  • received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42

    • expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
  • received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42

    • expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
  • handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:

  • handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:

    • no jobs were submitted
  • handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:

    • no jobs were submitted
  • handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:

    • no jobs were submitted

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

New job on instance Fram-NESSI for architecture x86_64-intel-broadwell for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.04/pr_310/5762232

date job status comment
Apr 07 15:42:52 UTC 2024 submitted job id 5762232 awaits release by job manager
Apr 07 15:43:35 UTC 2024 released job awaits launch by Slurm scheduler
Apr 07 15:44:37 UTC 2024 running job 5762232 is running
Apr 07 15:47:40 UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-5762232.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gzsize: 2 MiB (2344154 bytes)
entries: 114
modules under 2023.06/software/linux/x86_64/intel/broadwell/modules/all
parallel/20230722-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/intel/broadwell/software
parallel/20230722-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/intel/broadwell
no other files in tarball
Apr 07 15:47:40 UTC 2024 test result (no tests yet)
Apr 07 17:47:00 UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz to S3 bucket succeeded
Apr 07 05:57:17 PM UTC 2024 staged tarball eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz downloaded to Stratum-0
Apr 07 05:58:30 PM UTC 2024 pr_opened merge PR https://github.com/NorESSI/2024-staging/pull/112 to approve ingest
Apr 07 07:04:02 PM UTC 2024 approved 👍 tarball eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz approved, see PR https://github.com/NorESSI/2024-staging/pull/112
Apr 07 07:06:23 PM UTC 2024 ingested 🎉 tarball eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz successfully ingested at 2023.06/software/linux/x86_64/intel/broadwell/

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

New job on instance eX3-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb11 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.04/pr_310/163957

date job status comment
Apr 07 03:42:52 PM UTC 2024 submitted job id 163957 awaits release by job manager
Apr 07 03:43:25 PM UTC 2024 released job awaits launch by Slurm scheduler
Apr 07 03:44:27 PM UTC 2024 running job 163957 is running
Apr 07 03:51:35 PM UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-163957.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gzsize: 2 MiB (2344787 bytes)
entries: 114
modules under 2023.06/software/linux/aarch64/generic/modules/all
parallel/20230722-GCCcore-12.3.0.lua
software under 2023.06/software/linux/aarch64/generic/software
parallel/20230722-GCCcore-12.3.0
other under 2023.06/software/linux/aarch64/generic
no other files in tarball
Apr 07 03:51:35 PM UTC 2024 test result (no tests yet)
Apr 07 05:48:26 PM UTC 2024 uploaded transfer of eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz to S3 bucket succeeded
Apr 07 05:57:11 PM UTC 2024 staged tarball eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz downloaded to Stratum-0
Apr 07 05:58:21 PM UTC 2024 pr_opened merge PR https://github.com/NorESSI/2024-staging/pull/109 to approve ingest
Apr 07 06:03:32 PM UTC 2024 approved 👍 tarball eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz approved, see PR https://github.com/NorESSI/2024-staging/pull/109
Apr 07 06:13:10 PM UTC 2024 ingested 🎉 tarball eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz successfully ingested at 2023.06/software/linux/aarch64/generic/

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

New job on instance AWS-MC-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.04/pr_310/9177

date job status comment
Apr 07 15:42:53 UTC 2024 submitted job id 9177 awaits release by job manager
Apr 07 15:43:23 UTC 2024 released job awaits launch by Slurm scheduler
Apr 07 15:48:27 UTC 2024 running job 9177 is running
Apr 07 15:56:42 UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-9177.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gzsize: 2 MiB (2344415 bytes)
entries: 114
modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all
parallel/20230722-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/amd/zen2/software
parallel/20230722-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/amd/zen2
no other files in tarball
Apr 07 15:56:42 UTC 2024 test result (no tests yet)
Apr 07 17:46:58 UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz to S3 bucket succeeded
Apr 07 05:57:13 PM UTC 2024 staged tarball eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz downloaded to Stratum-0
Apr 07 05:58:24 PM UTC 2024 pr_opened merge PR https://github.com/NorESSI/2024-staging/pull/110 to approve ingest
Apr 07 07:03:59 PM UTC 2024 approved 👍 tarball eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz approved, see PR https://github.com/NorESSI/2024-staging/pull/110
Apr 07 07:05:27 PM UTC 2024 ingested 🎉 tarball eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz successfully ingested at 2023.06/software/linux/x86_64/amd/zen2/

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

New job on instance AWS-MC-NESSI for architecture x86_64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.04/pr_310/9178

date job status comment
Apr 07 15:42:56 UTC 2024 submitted job id 9178 awaits release by job manager
Apr 07 15:43:25 UTC 2024 released job awaits launch by Slurm scheduler
Apr 07 15:48:29 UTC 2024 running job 9178 is running
Apr 07 15:56:44 UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-9178.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gzsize: 2 MiB (2345688 bytes)
entries: 114
modules under 2023.06/software/linux/x86_64/generic/modules/all
parallel/20230722-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/generic/software
parallel/20230722-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/generic
no other files in tarball
Apr 07 15:56:44 UTC 2024 test result (no tests yet)
Apr 07 17:47:18 UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz to S3 bucket succeeded
Apr 07 05:57:15 PM UTC 2024 staged tarball eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz downloaded to Stratum-0
Apr 07 05:58:27 PM UTC 2024 pr_opened merge PR https://github.com/NorESSI/2024-staging/pull/111 to approve ingest
Apr 07 06:46:28 PM UTC 2024 approved 👍 tarball eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz approved, see PR https://github.com/NorESSI/2024-staging/pull/111
Apr 07 06:47:55 PM UTC 2024 ingested 🎉 tarball eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz successfully ingested at 2023.06/software/linux/x86_64/generic/

@nessi-bot
Copy link

nessi-bot bot commented Apr 7, 2024

New job on instance Saga-NESSI for architecture x86_64-intel-skylake_avx512 for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.04/pr_310/11097101

date job status comment
Apr 07 03:43:02 PM UTC 2024 submitted job id 11097101 awaits release by job manager
Apr 07 03:43:44 PM UTC 2024 released job awaits launch by Slurm scheduler
Apr 07 03:44:46 PM UTC 2024 running job 11097101 is running
Apr 07 03:51:54 PM UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-11097101.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gzsize: 2 MiB (2346064 bytes)
entries: 114
modules under 2023.06/software/linux/x86_64/intel/skylake_avx512/modules/all
parallel/20230722-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/intel/skylake_avx512/software
parallel/20230722-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/intel/skylake_avx512
no other files in tarball
Apr 07 03:51:54 PM UTC 2024 test result (no tests yet)
Apr 07 05:47:04 PM UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz to S3 bucket succeeded
Apr 07 05:57:19 PM UTC 2024 staged tarball eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz downloaded to Stratum-0
Apr 07 05:58:33 PM UTC 2024 pr_opened merge PR https://github.com/NorESSI/2024-staging/pull/113 to approve ingest
Apr 07 07:04:04 PM UTC 2024 approved 👍 tarball eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz approved, see PR https://github.com/NorESSI/2024-staging/pull/113
Apr 07 07:06:54 PM UTC 2024 ingested 🎉 tarball eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz successfully ingested at 2023.06/software/linux/x86_64/intel/skylake_avx512/

@trz42
Copy link
Collaborator Author

trz42 commented Apr 7, 2024

Checklist before starting deployment (setting bot:deploy label):

  • Check if the SPDX license identifier is provided
  • Check whether builds for all required architectures succeed (SUCCESS message + reasonably sized tarball)
  • Check if the PR is up-to-date with the target branch nessi.no-2023.06 in the repository (if not what are the differences)
  • Assess if all requested changes are sound (checking files changed on GitHub.com)
  • Verify that all easyconfig/s being built are included with the EB version used (if not why not)
  • Review changes (if any) needed to get the build(s) succeed (common changes for all architectures, changes for a single architecture, changes because of build environment specifics, etc.)

@trz42
Copy link
Collaborator Author

trz42 commented Apr 7, 2024

Target architectures

  • x86_64-intel-broadwell
  • x86_64-intel-skylake_avx512
  • x86_64-amd-zen2
  • x86_64-generic
  • aarch64-generic

Note! We want to carefully deploy this step-by-step (tarball-by-tarball) to verify if the changed procedure on the Stratum-0 works as designed (ingesting correctly and creating an updated Lmod cache).

Checklist for deployment/ingestion

  • at least one tarball for each architecture has been uploaded
  • at least one tarball for each architecture has been staged
  • for each staged tarball was a PR for approval/rejection opened
  • for each architecture one and only one tarball has been approved
  • for each architecture one and only one tarball was ingested
  • all tests in the CI have succeeded (rerun failed tests ~ 10 mins after last ingest)
  • has the package become available via CernVM-FS for all architectures ingested (for lmod cache updates pay attention to timestamp of files)
command & log

command

cd /cvmfs/pilot.nessi.no/versions/2023.06/software/linux ; \
ls -l \
      x86_64/{generic,amd/zen2,intel/[bsc]*}/{software,modules/all}/parallel/* \
      aarch64/generic/{software,modules/all}/parallel/* \
      x86_64/{generic,amd/zen2,intel/[bsc]*}/.lmod/cache/* \
      aarch64/generic/.lmod/cache/*

log after ingest of aarch64/generic

ls: cannot access 'x86_64/generic/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/generic/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/modules/all/parallel/*': No such file or directory
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1382346 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  916127 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46  aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1443712 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1430887 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1441339 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  x86_64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1460323 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  694693 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  490787 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1472188 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/timestamp

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share 

log after ingest of x86_64/generic

ls: cannot access 'x86_64/amd/zen2/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/modules/all/parallel/*': No such file or directory
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46  aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1430887 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/generic/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1139 Apr  7 17:53  x86_64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/timestamp

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share

x86_64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

log after ingest of x86_64/{amd/zen2,intel/broadwell,intel/skylake_avx512}

-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46 aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1434685 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  954361 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1434685 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  954361 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1140 Apr  7 17:53 x86_64/amd/zen2/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/generic/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1139 Apr  7 17:53 x86_64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1451317 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  965862 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1451317 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  965862 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1147 Apr  7 17:45 x86_64/intel/broadwell/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1463197 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  974077 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1152 Apr  7 17:47 x86_64/intel/skylake_avx512/modules/all/parallel/20230722-GCCcore-12.3.0.lua

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share

x86_64/amd/zen2/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

x86_64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

x86_64/intel/broadwell/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:45 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:45 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:45 share

x86_64/intel/skylake_avx512/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:47 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:47 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:47 share

Copy link

@poksumdo poksumdo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All works smoothly -- no Lmod cache updates when building software, but during ingestion at Stratum 0. 🎉

@poksumdo poksumdo merged commit deed9bb into NorESSI:nessi.no-2023.06 Apr 7, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants