From 8b569c82e2ed296ebca3aa5eae7528e202851ac0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Sun, 5 Oct 2025 23:56:56 +0200 Subject: [PATCH 01/19] Attempt to write python-search-path docs --- docs/python-search-path.md | 57 ++++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 58 insertions(+) create mode 100644 docs/python-search-path.md diff --git a/docs/python-search-path.md b/docs/python-search-path.md new file mode 100644 index 000000000..17d2f47e0 --- /dev/null +++ b/docs/python-search-path.md @@ -0,0 +1,57 @@ +# Python search path + +In order to locate python packages in modules, easybuild has conventionally used the `PYTHONPATH` environment variable. However, this has several issues: + +1. `PYTHONPATH` has highest priority; it prevents users from makinga custom virtual environment on top of modules and shadowing packages. +2. Packages are picked up even for incompatible python versions, e.g. an OS installed old python 3.6 will break if packages from 3.12 exists in `PYTHONPATH`. +3. Modules can't have optional dependencies on different Python versions, as PYTHONPATH points directly to the `site-packages` subdirectory. +4. Packages with `pth` files can't work with PYTHONPATH and requires being added to the site dir. + +Unfortuanately, Python offers to environment variables to do the correct thing here. +To solve this (initially for multi-deps) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. +It is included in standard Python installations made with easybuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. + +This does the correct thing by only considering the correct Python version, and puts the module provided python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. + +## Using `--prefer-python-search-path` + +Since v5.0.0 the new global configuration option `--prefer-python-search-path` can be used to EasyBuild prefer the use of either `PYTHONPATH` or `EBPYTHONPREFIXES`. +For backwards compatibility with existing modules, EasyBuild was unfortunately required to keep `PYTHONPATH` as the default. +Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then PYTHONPATH must be used. If ulti-deps is used, then `EBPYTHONPREFIXES` is required. + +If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing python bundles you have installed and make sure to rebuild the modules. +Failure to do so might leave you with a few easyconfigs that requiring shadowing of older package versions not working correctly due to the import priority changing. + +If you are swithing to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. + +## Fixing existing modules + +You can convert existing modules by rebuilding them with `eb --module-only --rebuild ...`. +You can find the modules that use `PYTHONPATH` by e.g. grepping through your modules: + +```bash +grep --include '*.lua' -Rl 'PYTHONPATH.*site-packages' /path/to/modules/all +``` + +If you use the default module naming scheme + +```bash +cd $MODULEPATH +grep -Rl PYTHONPATH */*.lua | grep -v EasyBuild | sed -e 's+/+-+' | sed -e 's+lua$+eb+' | xargs eb --rebuild --module-only +``` + +If done correctly, your old module files will have gone from + +```lua +prepend_path("PYTHONPATH", pathJoin(root, "lib", "python3.12", "site-packages")) +# or +prepend_path("PYTHONPATH", pathJoin(root, "lib/python3.12/site-packages")) +``` + +to + +```lua +prepend_path("EBPYTHONPREFIXES", root) +``` + +Remember to back up all your module files first before rebuilding. diff --git a/mkdocs.yml b/mkdocs.yml index 6cb670066..e30a2a5ea 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -73,6 +73,7 @@ nav: - Cray support: cray-support.md - Customizing EasyBuild via hooks: hooks.md - Including Python modules: including-additional-python-modules.md + - Customize Python search path: python-search-path.md - Packaging support: packaging-support.md - RPATH support: rpath-support.md - Using external modules: using-external-modules.md From bbab73ec542acc4206aacc797d627fb72b2ace0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:06:41 +0200 Subject: [PATCH 02/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 17d2f47e0..4a62067f6 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -22,7 +22,7 @@ Note that the option is just the preferred option, if the package path doesn't f If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing python bundles you have installed and make sure to rebuild the modules. Failure to do so might leave you with a few easyconfigs that requiring shadowing of older package versions not working correctly due to the import priority changing. -If you are swithing to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. +If you are switching to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. ## Fixing existing modules From c24bc109bded9cf1dc424aa246baaa7232a9f06d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:06:54 +0200 Subject: [PATCH 03/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 4a62067f6..a5756cc2d 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -17,7 +17,7 @@ This does the correct thing by only considering the correct Python version, and Since v5.0.0 the new global configuration option `--prefer-python-search-path` can be used to EasyBuild prefer the use of either `PYTHONPATH` or `EBPYTHONPREFIXES`. For backwards compatibility with existing modules, EasyBuild was unfortunately required to keep `PYTHONPATH` as the default. -Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then PYTHONPATH must be used. If ulti-deps is used, then `EBPYTHONPREFIXES` is required. +Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then `PYTHONPATH` must be used. If multi-deps is used, then `EBPYTHONPREFIXES` is required. If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing python bundles you have installed and make sure to rebuild the modules. Failure to do so might leave you with a few easyconfigs that requiring shadowing of older package versions not working correctly due to the import priority changing. From 156ede79565eff1167fdcc87d9a0448f161f03a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:07:04 +0200 Subject: [PATCH 04/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index a5756cc2d..e539328b7 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -7,7 +7,7 @@ In order to locate python packages in modules, easybuild has conventionally used 3. Modules can't have optional dependencies on different Python versions, as PYTHONPATH points directly to the `site-packages` subdirectory. 4. Packages with `pth` files can't work with PYTHONPATH and requires being added to the site dir. -Unfortuanately, Python offers to environment variables to do the correct thing here. +Unfortunately, Python offers no environment variables to do the correct thing here. To solve this (initially for multi-deps) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. It is included in standard Python installations made with easybuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. From 71c01f7eef23e42a7695d6e90c8602f13b7a75af Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:07:17 +0200 Subject: [PATCH 05/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index e539328b7..e6e6c8622 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -9,7 +9,7 @@ In order to locate python packages in modules, easybuild has conventionally used Unfortunately, Python offers no environment variables to do the correct thing here. To solve this (initially for multi-deps) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. -It is included in standard Python installations made with easybuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. +It is included in standard Python installations made with EasyBuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. This does the correct thing by only considering the correct Python version, and puts the module provided python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. From 69032fe87026fe4d89c145fc2ece7aea0c8425aa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:07:30 +0200 Subject: [PATCH 06/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index e6e6c8622..eb627ead0 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -11,7 +11,7 @@ Unfortunately, Python offers no environment variables to do the correct thing he To solve this (initially for multi-deps) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. It is included in standard Python installations made with EasyBuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. -This does the correct thing by only considering the correct Python version, and puts the module provided python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. +This does the correct thing by only considering the correct Python version, and puts the module provided Python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. ## Using `--prefer-python-search-path` From 9e7d6c8709347a45efd991a0b9b6c4db87769d22 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 10:07:46 +0200 Subject: [PATCH 07/19] Typo fix from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index eb627ead0..5fc850c48 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -19,7 +19,7 @@ Since v5.0.0 the new global configuration option `--prefer-python-search-path` c For backwards compatibility with existing modules, EasyBuild was unfortunately required to keep `PYTHONPATH` as the default. Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then `PYTHONPATH` must be used. If multi-deps is used, then `EBPYTHONPREFIXES` is required. -If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing python bundles you have installed and make sure to rebuild the modules. +If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing Python bundles you have installed and make sure to rebuild the modules. Failure to do so might leave you with a few easyconfigs that requiring shadowing of older package versions not working correctly due to the import priority changing. If you are switching to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. From 8114ecd380fc1314c523f766650701f7fe70d30d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:56:46 +0200 Subject: [PATCH 08/19] Apply suggestion from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 5fc850c48..57e8e2053 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -20,7 +20,7 @@ For backwards compatibility with existing modules, EasyBuild was unfortunately r Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then `PYTHONPATH` must be used. If multi-deps is used, then `EBPYTHONPREFIXES` is required. If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing Python bundles you have installed and make sure to rebuild the modules. -Failure to do so might leave you with a few easyconfigs that requiring shadowing of older package versions not working correctly due to the import priority changing. +Failure to do so might leave you with a few easyconfigs that require shadowing of older package versions not working correctly due to the import priority changing. If you are switching to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. From 6b1ee82474e103fb5008892440d58686fe5319bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:56:52 +0200 Subject: [PATCH 09/19] Apply suggestion from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 57e8e2053..8ca28752d 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -15,7 +15,7 @@ This does the correct thing by only considering the correct Python version, and ## Using `--prefer-python-search-path` -Since v5.0.0 the new global configuration option `--prefer-python-search-path` can be used to EasyBuild prefer the use of either `PYTHONPATH` or `EBPYTHONPREFIXES`. +Since v5.0.0 the new global configuration option `--prefer-python-search-path` can be used for EasyBuild to prefer the use of either `PYTHONPATH` or `EBPYTHONPREFIXES`. For backwards compatibility with existing modules, EasyBuild was unfortunately required to keep `PYTHONPATH` as the default. Note that the option is just the preferred option, if the package path doesn't follow the standard `lib/pythonY.X/site-packages` format then `PYTHONPATH` must be used. If multi-deps is used, then `EBPYTHONPREFIXES` is required. From 831c7f19a3daf59db0a59f932dc6b481ffdb9ef4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:56:59 +0200 Subject: [PATCH 10/19] Apply suggestion from @smoors Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 8ca28752d..341d0d06d 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -1,6 +1,6 @@ # Python search path -In order to locate python packages in modules, easybuild has conventionally used the `PYTHONPATH` environment variable. However, this has several issues: +In order to locate python packages in modules, before v5.0.0, EasyBuild conventionally used the `PYTHONPATH` environment variable. However, this has several issues: 1. `PYTHONPATH` has highest priority; it prevents users from makinga custom virtual environment on top of modules and shadowing packages. 2. Packages are picked up even for incompatible python versions, e.g. an OS installed old python 3.6 will break if packages from 3.12 exists in `PYTHONPATH`. From 226392773f4b01d18ca9224cbb364614cbd4b8ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:57:05 +0200 Subject: [PATCH 11/19] Apply suggestion from @verdurin Co-authored-by: Adam Huffman --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 341d0d06d..cbde82ad2 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -2,7 +2,7 @@ In order to locate python packages in modules, before v5.0.0, EasyBuild conventionally used the `PYTHONPATH` environment variable. However, this has several issues: -1. `PYTHONPATH` has highest priority; it prevents users from makinga custom virtual environment on top of modules and shadowing packages. +1. `PYTHONPATH` has highest priority; it prevents users from making a custom virtual environment on top of modules and shadowing packages. 2. Packages are picked up even for incompatible python versions, e.g. an OS installed old python 3.6 will break if packages from 3.12 exists in `PYTHONPATH`. 3. Modules can't have optional dependencies on different Python versions, as PYTHONPATH points directly to the `site-packages` subdirectory. 4. Packages with `pth` files can't work with PYTHONPATH and requires being added to the site dir. From 9833430052023ffceb3aff1ee62aa92d1dbb97dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:57:12 +0200 Subject: [PATCH 12/19] Apply suggestion from @smoors Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index cbde82ad2..8413f39d6 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -8,7 +8,7 @@ In order to locate python packages in modules, before v5.0.0, EasyBuild conventi 4. Packages with `pth` files can't work with PYTHONPATH and requires being added to the site dir. Unfortunately, Python offers no environment variables to do the correct thing here. -To solve this (initially for multi-deps) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. +To solve this (initially for Python `multi_deps`) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. It is included in standard Python installations made with EasyBuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. This does the correct thing by only considering the correct Python version, and puts the module provided Python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. From ea7af4bb51ffeaa4757c072dd4a208f874ed8718 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:57:18 +0200 Subject: [PATCH 13/19] Apply suggestion from @smoors Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 8413f39d6..9aa56fda8 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -11,7 +11,7 @@ Unfortunately, Python offers no environment variables to do the correct thing he To solve this (initially for Python `multi_deps`) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. It is included in standard Python installations made with EasyBuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. -This does the correct thing by only considering the correct Python version, and puts the module provided Python packages at the lowest priority path, allowing a user venv to shadow the packages correctly. +This does the correct thing by only considering the correct Python version, and puts Python packages provided by the environment module at the lowest priority, allowing a user venv to override them. ## Using `--prefer-python-search-path` From 1fd3c60f2e19bd66b45bf6c39bbe856dc3e1e6d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:57:24 +0200 Subject: [PATCH 14/19] Apply suggestion from @smoors Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 9aa56fda8..84188025f 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -9,7 +9,7 @@ In order to locate python packages in modules, before v5.0.0, EasyBuild conventi Unfortunately, Python offers no environment variables to do the correct thing here. To solve this (initially for Python `multi_deps`) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. -It is included in standard Python installations made with EasyBuild. You can opt out of having this `sitecustomize.py` if you have strong technical reasons to avoid it. +It is included in standard Python installations made with EasyBuild. You can opt out of using `sitecustomize.py` if you have strong technical reasons to avoid it. This does the correct thing by only considering the correct Python version, and puts Python packages provided by the environment module at the lowest priority, allowing a user venv to override them. From e30e7e3eb7430f767dd3ffb48e1ef1e8ba27a2ab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Mon, 6 Oct 2025 12:57:30 +0200 Subject: [PATCH 15/19] Apply suggestion from @smoors Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 84188025f..800eec047 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -22,7 +22,7 @@ Note that the option is just the preferred option, if the package path doesn't f If you wish to switch to `EBPYTHONPREFIXES`, you should also traverse the existing Python bundles you have installed and make sure to rebuild the modules. Failure to do so might leave you with a few easyconfigs that require shadowing of older package versions not working correctly due to the import priority changing. -If you are switching to building a new tree, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. +If you are building a new software stack from scratch, you can safely switch this option to `EBPYTHONPREFIXES` and enjoy the benefits. ## Fixing existing modules From e12f63193f1959cb26c9d66b1622b7ae3da37ecc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Wed, 8 Oct 2025 11:22:51 +0200 Subject: [PATCH 16/19] Update docs/python-search-path.md Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 800eec047..4ee999a4d 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -1,4 +1,4 @@ -# Python search path +# Customizing Python search path In order to locate python packages in modules, before v5.0.0, EasyBuild conventionally used the `PYTHONPATH` environment variable. However, this has several issues: From bfa4584092ef274429bc26c1281441354b82ff84 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Wed, 8 Oct 2025 11:23:01 +0200 Subject: [PATCH 17/19] Update docs/python-search-path.md Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 4ee999a4d..093a44f3c 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -1,6 +1,6 @@ # Customizing Python search path -In order to locate python packages in modules, before v5.0.0, EasyBuild conventionally used the `PYTHONPATH` environment variable. However, this has several issues: +In order to locate Python packages in modules, before v5.0.0, EasyBuild conventionally used the `PYTHONPATH` environment variable. However, this has several issues: 1. `PYTHONPATH` has highest priority; it prevents users from making a custom virtual environment on top of modules and shadowing packages. 2. Packages are picked up even for incompatible python versions, e.g. an OS installed old python 3.6 will break if packages from 3.12 exists in `PYTHONPATH`. From 576b59603d223780dcf01270aee0acd391fd2288 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Wed, 8 Oct 2025 11:23:09 +0200 Subject: [PATCH 18/19] Update mkdocs.yml Co-authored-by: Sam Moors --- mkdocs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs.yml b/mkdocs.yml index e30a2a5ea..1b4aa1b02 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -73,7 +73,7 @@ nav: - Cray support: cray-support.md - Customizing EasyBuild via hooks: hooks.md - Including Python modules: including-additional-python-modules.md - - Customize Python search path: python-search-path.md + - Customizing Python search path: python-search-path.md - Packaging support: packaging-support.md - RPATH support: rpath-support.md - Using external modules: using-external-modules.md From d2182d60147edb4126624efea2499c70104f4e5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mikael=20=C3=96hman?= Date: Wed, 8 Oct 2025 11:23:18 +0200 Subject: [PATCH 19/19] Update docs/python-search-path.md Co-authored-by: Sam Moors --- docs/python-search-path.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/python-search-path.md b/docs/python-search-path.md index 093a44f3c..7a766d581 100644 --- a/docs/python-search-path.md +++ b/docs/python-search-path.md @@ -11,7 +11,7 @@ Unfortunately, Python offers no environment variables to do the correct thing he To solve this (initially for Python `multi_deps`) EasyBuild has for a long time supported the use of the custom `EBPYTHONPREFIXES` via a `sitecustomize.py` script for the Python modules we build. It is included in standard Python installations made with EasyBuild. You can opt out of using `sitecustomize.py` if you have strong technical reasons to avoid it. -This does the correct thing by only considering the correct Python version, and puts Python packages provided by the environment module at the lowest priority, allowing a user venv to override them. +The `sitecustomize.py` script does the correct thing by only considering the correct Python version, and puts Python packages provided by the environment module at the lowest priority, allowing a user venv to override them. ## Using `--prefer-python-search-path`