Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround to fix broken curl installation #623

Merged
merged 4 commits into from
Jul 16, 2024

Conversation

hvelab
Copy link
Contributor

@hvelab hvelab commented Jun 26, 2024

As reported in the EESSI support portal issue #25, curl fails in RHEL 8 and above systems as CA files location differs. Works for all rhel based systems as I could reproduce the issue also in them.

$ cat /etc/os-release 
NAME="Rocky Linux"
VERSION="9.3 (Blue Onyx)"
ID="rocky"
ID_LIKE="rhel centos fedora"

$ source software-layer/init/bash 
Found EESSI repo @ /cvmfs/software.eessi.io/versions/2023.06!
archdetect says x86_64/intel/skylake_avx512
Using x86_64/intel/skylake_avx512 as software subdirectory.
[...]
Environment set up to use EESSI (2023.06), have fun!

{EESSI 2023.06} $ ml CMake
{EESSI 2023.06} $ curl https://www.example.com
<!doctype html>
<html>
<head>
    <title>Example Domain</title>
[...]

I am not a fan of hardcording in the init script but investigating the issue I haven't found a more suitable way to solve it for our use case. Maybe another solution would be to add a modlua footer exporting this variable in the affected modules?

Copy link

eessi-bot-aws bot commented Jun 26, 2024

Instance eessi-bot-mc-aws is configured to build for:

  • architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
  • repositories: eessi.io-2023.06-compat, eessi-hpc.org-2023.06-software, eessi-hpc.org-2023.06-compat, eessi.io-2023.06-software

Instance boegel-bot-deucalion is configured to build for:

  • architectures: aarch64/a64fx
  • repositories: eessi.io-2023.06-software

Copy link

eessi-bot-aws bot commented Jun 26, 2024

Instance eessi-bot-mc-azure is configured to build for:

  • architectures: x86_64/amd/zen4
  • repositories: eessi.io-2023.06-software, eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat

@ocaisa
Copy link
Member

ocaisa commented Jun 27, 2024

Currently we ship curl in the compat layer, so there's no way to avoid setting this in the initialisation script. If it ever came to be that we stopped doing that, then it would be enough to set this via an Lmod hook (this may be worth a comment in the init script where this is done)

Copy link
Member

@ocaisa ocaisa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but can you add a comment to mention that if curl we're removed from the compat layer we could do this via the cURL module file (or an Lmod hook) instead

…sed systems

Co-authored-by: ocaisa <alan.ocais@cecam.org>
init/eessi_environment_variables Outdated Show resolved Hide resolved
Add bigger comment

Co-authored-by: ocaisa <alan.ocais@cecam.org>
@xinan1911
Copy link
Collaborator

@ocaisa Could you take a final check before merging this PR?

@ocaisa
Copy link
Member

ocaisa commented Jul 15, 2024

I can't merge this because I don't know how to deploy it

@bedroge
Copy link
Collaborator

bedroge commented Jul 16, 2024

I'm not completely sure anymore if this can now be done with the bot (we did make some changes related to including the init scripts). Let's try it, and otherwise I can do it manually.

@bedroge
Copy link
Collaborator

bedroge commented Jul 16, 2024

bot: build repo:eessi.io-2023.06-software arch:x86_64/generic

Copy link

eessi-bot-aws bot commented Jul 16, 2024

Updates by the bot instance eessi-bot-mc-aws (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/generic from bedroge

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/generic
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/generic resulted in:

Copy link

eessi-bot-aws bot commented Jul 16, 2024

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/generic from bedroge

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/generic
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/generic resulted in:

    • no jobs were submitted

Copy link

eessi-bot-aws bot commented Jul 16, 2024

New job on instance eessi-bot-mc-aws for architecture x86_64-generic for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.07/pr_623/14519

date job status comment
Jul 16 07:50:03 UTC 2024 submitted job id 14519 awaits release by job manager
Jul 16 07:50:26 UTC 2024 released job awaits launch by Slurm scheduler
Jul 16 11:44:04 UTC 2024 running job 14519 is running
Jul 16 11:56:25 UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-14519.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-generic-1721130246.tar.gzsize: 0 MiB (2121 bytes)
entries: 1
modules under 2023.06/software/linux/x86_64/generic/modules/all
no module files in tarball
software under 2023.06/software/linux/x86_64/generic/software
no software packages in tarball
other under 2023.06/software/linux/x86_64/generic
2023.06/init/eessi_environment_variables
Jul 16 11:56:25 UTC 2024 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ PASSED ] Ran 14/14 test case(s) from 14 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-14519.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Jul 16 12:01:55 UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-generic-1721130246.tar.gz to S3 bucket succeeded

@bedroge
Copy link
Collaborator

bedroge commented Jul 16, 2024

Looks like it actually worked 🎉.

I also tested the fix on our Rocky 8 cluster: I got an error without the fix, but after applying the fix the curl command worked fine.

Copy link
Collaborator

@bedroge bedroge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@bedroge bedroge added bug Something isn't working 2023.06-software.eessi.io 2023.06 version of software.eessi.io bot:deploy Ask bot to deploy missing software installations to EESSI labels Jul 16, 2024
@ocaisa ocaisa merged commit c2cbd39 into EESSI:2023.06-software.eessi.io Jul 16, 2024
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2023.06-software.eessi.io 2023.06 version of software.eessi.io bot:deploy Ask bot to deploy missing software installations to EESSI bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants