Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace Z3 dependency for PyTorch-2.1.2-foss-2023a.eb #501

Conversation

trz42
Copy link
Collaborator

@trz42 trz42 commented Mar 13, 2024

A recent change to the easyconfig PyTorch-2.1.2-foss-2023a.eb (see easybuilders/easybuild-easyconfigs#20050) lets the check for missing installations fail, for example, see
https://github.com/EESSI/software-layer/actions/runs/8268611041/job/22621916438?pr=500#step:4:2130

This PR adds a hook that attempts to replace the dependency for Z3 with the version that was used before (with suffix -Python-3.11.3).

CI should not fail anymore. However, running eb with --missing prints messages that the dependency has been replaced, but the renamed dependency is then not listed anymore. Running with --dry-run does show the renamed dependency. Not sure if this behaviour is expected or not.

@trz42 trz42 added bug Something isn't working 2023.06-software.eessi.io 2023.06 version of software.eessi.io labels Mar 13, 2024
Copy link

eessi-bot-aws bot commented Mar 13, 2024

Instance eessi-bot-mc-aws is configured to build:

  • arch x86_64/generic for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/generic for repo eessi-hpc.org-2023.06-software
  • arch x86_64/generic for repo eessi.io-2023.06-compat
  • arch x86_64/generic for repo eessi.io-2023.06-software
  • arch x86_64/intel/haswell for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/intel/haswell for repo eessi-hpc.org-2023.06-software
  • arch x86_64/intel/haswell for repo eessi.io-2023.06-compat
  • arch x86_64/intel/haswell for repo eessi.io-2023.06-software
  • arch x86_64/intel/skylake_avx512 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/intel/skylake_avx512 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/intel/skylake_avx512 for repo eessi.io-2023.06-compat
  • arch x86_64/intel/skylake_avx512 for repo eessi.io-2023.06-software
  • arch x86_64/amd/zen2 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/amd/zen2 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/amd/zen2 for repo eessi.io-2023.06-compat
  • arch x86_64/amd/zen2 for repo eessi.io-2023.06-software
  • arch x86_64/amd/zen3 for repo eessi-hpc.org-2023.06-compat
  • arch x86_64/amd/zen3 for repo eessi-hpc.org-2023.06-software
  • arch x86_64/amd/zen3 for repo eessi.io-2023.06-compat
  • arch x86_64/amd/zen3 for repo eessi.io-2023.06-software
  • arch aarch64/generic for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/generic for repo eessi-hpc.org-2023.06-software
  • arch aarch64/generic for repo eessi.io-2023.06-compat
  • arch aarch64/generic for repo eessi.io-2023.06-software
  • arch aarch64/neoverse_n1 for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/neoverse_n1 for repo eessi-hpc.org-2023.06-software
  • arch aarch64/neoverse_n1 for repo eessi.io-2023.06-compat
  • arch aarch64/neoverse_n1 for repo eessi.io-2023.06-software
  • arch aarch64/neoverse_v1 for repo eessi-hpc.org-2023.06-compat
  • arch aarch64/neoverse_v1 for repo eessi-hpc.org-2023.06-software
  • arch aarch64/neoverse_v1 for repo eessi.io-2023.06-compat
  • arch aarch64/neoverse_v1 for repo eessi.io-2023.06-software

@casparvl
Copy link
Collaborator

casparvl commented Mar 14, 2024

Hm, this is really due to the difference between the set of EasyConfigs used to build and the set of EasyConfig used when running --missing.

For the build, because PyTorch is in https://github.com/EESSI/software-layer/blob/2023.06-software.eessi.io/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml , we run EasyBuild version 4.9.0. For the --missing, we run the EasyConfigs from the develop branch, since @ocaisa 's #486 .

Make me wonder if this actually makes sense. There was a good reason for it, as it allowed us to strip a lot of --from-pr's, and enforce that EasyConfigs should be merged. However, it is fundamentally inconsistent to use different versions of the EasyConfig in the --missing check.

I need to think about what the best way is to proceed. This PR could work as a workaround. We could also explicitly install Z3 without the Python prefix (i.e. new naming), to make the --missing check pass (it doesn't really hurt to install that). Not sure what's preferable...

@boegel
Copy link
Contributor

boegel commented Mar 14, 2024

I would do the latter (installing Z3 without Python versionsuffix) in short term.

The idea is to switch to using --from-commit when that's ready anyway, and then many of the issues shouldn't present themselves anymore?

@ocaisa
Copy link
Member

ocaisa commented Mar 14, 2024

I'd be with @boegel here, the easy (and harmless) fix is to just install the new Z3

@trz42
Copy link
Collaborator Author

trz42 commented Mar 17, 2024

Obsolete after installing Z3 (#502)

@trz42 trz42 closed this Mar 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2023.06-software.eessi.io 2023.06 version of software.eessi.io bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants