Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tweak CMakeMake easyblock apply CMake policy to select python based on $PATH order, instead of highest Python version #3088

Merged
merged 1 commit into from Jan 26, 2024
Merged

tweak CMakeMake easyblock apply CMake policy to select python based on $PATH order, instead of highest Python version #3088

merged 1 commit into from Jan 26, 2024

Conversation

casparvl
Copy link
Contributor

Set policy so that the python is selected based on location, i.e. the order in PATH, rather than the newest version (which might be an OS python).

This fixes issues like seen in e.g. EESSI/software-layer#370 (comment)

Set policy so that the python is selected based on location, i.e. the…
5a46079 
… order in PATH, rather than the newest version (which might be an OS python).
@casparvl casparvl mentioned this pull request Jan 23, 2024
Closed
@casparvl
Copy link
Contributor Author

With this PR, testing on casacore (where I undid the previous fix of specifying the Python exectuable as a configopt), I got:

-- PYTHON3_EXECUTABLE ......... = /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Python/3.11.3-GCCcore-12.3.0/bin/python3.11
-- PYTHON3_LIBRARIES .......... = /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Python/3.11.3-GCCcore-12.3.0/lib/libpython3
.11.so
-- PYTHON3_NUMPY_INCLUDE_DIRS . = /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/SciPy-bundle/2023.07-gfbf-2023a/lib/python3
.11/site-packages/numpy/core/include

and the build succesfully completes. So, looks like this does what it should do.

@casparvl
Copy link
Contributor Author

Test report by @casparvl

Overview of tested easyconfigs (in order)

  • SUCCESS pypmt-1.1.0-gfbf-2023a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 545.23.08, Python 3.6.8
See https://gist.github.com/casparvl/d444c10fe10d9840a71827843facc53f for a full test report.

@casparvl
Copy link
Contributor Author

To check if the correct Python was selected, I checked the build log for pypmt:

-- Found Python3: /sw/arch/RHEL8/EB_production/2023/software/Python/3.11.3-GCCcore-12.3.0/bin/python3.11 (found version "3.11.3") found components: Interpreter Development Development.Module Development.Embed

As expected.

@casparvl
Copy link
Contributor Author

Test report by @casparvl

Overview of tested easyconfigs (in order)

  • SUCCESS libcint-5.4.0-gfbf-2023a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 545.23.08, Python 3.6.8
See https://gist.github.com/casparvl/d0b2adea1fdae0696bd0e4086938ce85 for a full test report.

@casparvl
Copy link
Contributor Author

Likewise, correctly located for libcint:

-- Found PythonInterp: /scratch-nvme/1/casparl/generic/software/Python/3.11.3-GCCcore-12.3.0/bin/python (found version "3.11.3")
-- Found python  /scratch-nvme/1/casparl/generic/software/Python/3.11.3-GCCcore-12.3.0/bin/python

@casparvl
Copy link
Contributor Author

casparvl commented Jan 26, 2024 •
edited

Just to try a number of builds of a package that uses CMakeMake and Python...

@boegelbot please test @ generoso
EB_ARGS="Arrow-6.0.0-foss-2021a.eb Arrow-6.0.0-foss-2021b.eb Arrow-8.0.0-foss-2022a.eb Arrow-11.0.0-gfbf-2022b.eb Arrow-14.0.1-gfbf-2023a.eb --include-easyblocks-from-pr=3088"

@casparvl
Copy link
Contributor Author

Test report by @casparvl

Overview of tested easyconfigs (in order)

  • SUCCESS OpenMM-8.0.0-foss-2023a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 545.23.08, Python 3.6.8
See https://gist.github.com/casparvl/5bf1e3d1b6def87f88abfeee804c2203 for a full test report.

@casparvl
Copy link
Contributor Author

OpenMM configure also looks as it should be:

-- Found PythonInterp: /scratch-nvme/1/casparl/generic/software/Python/3.11.3-GCCcore-12.3.0/bin/python (found version "3.11.3")
CMake Warning (dev) at CMakeLists.txt:21 (PROJECT):
  cmake_minimum_required() should be called prior to this top-level project()
  call.  Please see the cmake-commands(7) manual for usage documentation of
  both commands.

@boegelbot
Copy link

@casparvl: Request for testing this PR well received on login1

PR test command 'EB_PR=3088 EB_ARGS="Arrow-6.0.0-foss-2021a.eb Arrow-6.0.0-foss-2021b.eb Arrow-8.0.0-foss-2022a.eb Arrow-11.0.0-gfbf-2022b.eb Arrow-14.0.1-gfbf-2023a.eb --include-easyblocks-from-pr=3088" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_3088 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 12747

Test results coming soon (I hope)...

- notification for comment with ID 1912269105 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@casparvl
Copy link
Contributor Author

Test report by @casparvl

Overview of tested easyconfigs (in order)

  • SUCCESS RapidJSON-1.1.0-20230928-GCCcore-12.3.0.eb
  • SUCCESS utf8proc-2.8.0-GCCcore-12.3.0.eb
  • SUCCESS Arrow-14.0.1-gfbf-2023a.eb

Build succeeded for 3 out of 3 (1 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 545.23.08, Python 3.6.8
See https://gist.github.com/casparvl/a0fe189f50a54286409e9ba0c13934fc for a full test report.

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS Arrow-6.0.0-foss-2021a.eb
  • SUCCESS Arrow-6.0.0-foss-2021b.eb
  • SUCCESS Arrow-8.0.0-foss-2022a.eb
  • SUCCESS Arrow-11.0.0-gfbf-2022b.eb
  • SUCCESS Arrow-14.0.1-gfbf-2023a.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/b9a6da0657aad13150b7f31c4a46b8c4 for a full test report.

@boegel boegel added the bug fix label Jan 26, 2024
@boegel boegel added this to the release after 4.9.0 milestone Jan 26, 2024
boegel
boegel approved these changes Jan 26, 2024
View reviewed changes
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit 941c1eb into easybuilders:develop Jan 26, 2024
47 checks passed
@boegel boegel changed the title Apply CMake policy to select python based on PATH order, instead of highest version Apply CMake policy to select python based on $PATH order, instead of highest Python version Jan 26, 2024
@casparvl
Copy link
Contributor Author

Test report by @casparvl

Overview of tested easyconfigs (in order)

  • SUCCESS GDAL-3.7.1-foss-2023a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 545.23.08, Python 3.6.8
See https://gist.github.com/casparvl/b95ac33020ded0449240b60624e6ff33 for a full test report.

@boegel boegel changed the title Apply CMake policy to select python based on $PATH order, instead of highest Python version apply CMake policy to select python based on $PATH order, instead of highest Python version Jan 30, 2024
This was referenced Feb 14, 2024
Closed
Merged
@bedroge bedroge mentioned this pull request Feb 20, 2024
Merged
@boegel boegel mentioned this pull request Mar 13, 2024
Merged
@boegel boegel changed the title apply CMake policy to select python based on $PATH order, instead of highest Python version tweak CMakeMake easyblock apply CMake policy to select python based on $PATH order, instead of highest Python version Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@boegel boegel boegel approved these changes

Assignees
No one assigned
Labels
bug fix
Projects
None yet
Milestone
4.9.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@casparvl @boegelbot @boegel