diff --git a/docs/easybuild-v5/deprecated-functionality.md b/docs/easybuild-v5/deprecated-functionality.md index c6d40ef35..08666455d 100644 --- a/docs/easybuild-v5/deprecated-functionality.md +++ b/docs/easybuild-v5/deprecated-functionality.md @@ -110,13 +110,18 @@ The following checksum types are deprecated and should no longer be used: `md5`, ## `EnvironmentModulesC` or `EnvironmentModulesTcl` modules tools {: #modules-tools } -*(more info soon)* +The use of the older [Environment Modules](https://modules.readthedocs.io/) versions is deprecated. +This corresponds to setting the module tool in EasyBuild to either `EnvironmentModulesC` or +`EnvironmentModulesTcl`. + +Those using Environment Modules should use at least version 4.3 or newer. --- ## GC3Pie as job backend {: #gc3pie-job-backend } -*(more info soon)* +The use of the GC3Pie job backend is deprecated. Those [generating jobs][submitting_jobs] from +EasyBuild should use one of the other available backends instead, such as Slurm or PBS Python. --- @@ -128,7 +133,8 @@ The following checksum types are deprecated and should no longer be used: `md5`, ## `COMPILER*_FLAGS` attributes in `Compiler` class {: #compiler-constants } -*(replaced by `Compiler.COMPILER*_OPTIONS`, more info soon)* +In the `Compiler` class, the `COMPILER*_FLAGS attributes` were renamed to change `FLAGS` to `OPTIONS`. +This change was made to make it clearer as to the function of these attributes. --- diff --git a/docs/easybuild-v5/enhancements.md b/docs/easybuild-v5/enhancements.md index 039e2cf5f..d9329d5cf 100644 --- a/docs/easybuild-v5/enhancements.md +++ b/docs/easybuild-v5/enhancements.md @@ -63,14 +63,17 @@ If you would like to see other types of easyconfigs added as templates, please ## Mark support for installing extensions in parallel as stable (no longer experimental) {: #parallel-extensions-install-stable } -*(more info soon)* +The support for [installing extensions in parallel][installing_extensions_in_parallel] +no longer requires the use of the `--experimental` option. Currently this supports +installing R extensions in parallel. --- ## Mark easystack support as stable (no longer experimental) {: #easystack-stable } -*(more info soon)* +[Easystack files][easystack] are now considered stable and no longer require the use +of the `--experimental` option. --- @@ -168,7 +171,12 @@ status codes are reported in the corresponding logs. ## Copy build directory and/or log file(s) if installation failed to path specified via `--failed-install-build-dirs-path` or `--failed-install-logs-path` {: #copy-build-log-failed-installs } -*(more info soon)* +Often local storage (such as `/tmp`) or `/dev/shm` is used for the build path. When these are used with `--job` +then the build files and log files will be removed when the job finishes. If a build fails this can make +determining the reason for failure difficult. The options `--failed-install-build-dirs-path` and +`--failed-install-logs-path` have been added to copy the build directory of log files to a specified path in +the event of a failed build. + --- @@ -186,9 +194,13 @@ See [separate dedicated section](../implementing-easyblocks.md#module_load_envir --- -## `keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option {: #keep-debug-symbols } +## `--keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option {: #keep-debug-symbols } -*(more info soon)* +A configuration option `--keep-debug-symbols` was added to set the default value for the `debug` toolchain option. +Enabling this will add `-g` to flags such as `$CXXFLAGS`. + +!!! Warning + For some software enabling debug symbols will result in significant increase in disk space usage. --- @@ -316,7 +328,7 @@ This option is also available as easyconfig parameter ## Support not using `$PYTHONPATH` to specify the location of installed Python packages (via `--prefer-python-search-path`) { : #PYTHONPATH-vs-EBPYTHONPREFIXES } -*(more info soon)* +See [dedicated page](../python-search-path.md). --- diff --git a/docs/easybuild-v5/index.md b/docs/easybuild-v5/index.md index 6fe7890ad..1a83e7165 100644 --- a/docs/easybuild-v5/index.md +++ b/docs/easybuild-v5/index.md @@ -4,12 +4,6 @@ A high-level overview of changes in EasyBuild version 5.0.0 is listed below. Click on a particular item for more information. -!!! warning - - (Tue 18 March 2025) - The sections linked to from this overview below are known to be incomplete. - Extra information will be gradually added over the next couple of days. - --- ## Breaking changes {: #breaking-changes } @@ -76,7 +70,7 @@ Various significant enhancements are included in EasyBuild v5.0, including: - [Copy build directory and/or log file(s) if installation failed to path specified via `--failed-install-build-dirs-path` or `--failed-install-logs-path`](enhancements.md#copy-build-log-failed-installs) - [Specify changes that should be made by generated module files via `module_load_environment`](../implementing-easyblocks.md#module_load_environment) - [Add support for alternate easyconfig parameters/templates/constants](enhancements.md#alternative-easyconfig-parameters-templates-constants) -- [`keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option](enhancements.md#keep-debug-symbols) +- [`--keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option](enhancements.md#keep-debug-symbols) - [Provide control over how generated modules update search path for header files (via `--module-search-path-headers`)](enhancements.md#module-search-path-headers) - [Provide control over how EasyBuild specifies path to header files during installation (via `--search-path-cpp-headers`)](enhancements.md#search-path-cpp-headers) - [Provide control over how EasyBuild specifies path to libraries during installation (via `--search-path-linker`)](enhancements.md#search-path-linker) @@ -94,12 +88,12 @@ Functionality that was deprecated a while ago in EasyBuild v4.x has been ***remo - [EasyBuild bootstrap script](removed-functionality.md#bootstrap_script) - [Experimental support for `.yeb` easyconfig](removed-functionality.md#yeb) -- [`accept-eula` configuration setting](removed-functionality.md#accept-eula) +- [`--accept-eula` configuration setting](removed-functionality.md#accept-eula) - [`--wait-on-lock` configuration setting](removed-functionality.md#wait-on-lock) - [`is_generic_easyblock` function from `easybuild.framework.easyconfig.easyconfig`](removed-functionality.md#easyconfig-is_generic_easyblock) - [`copytree`, `rmtree2` functions from `easybuild.filetools`](removed-functionality.md#filetools-copytree-rmtree2) - [`fetch_extension_sources` method in `EasyBlock` class](removed-functionality.md#EasyBlock-fetch_extension_sources) -- [`mod_exists_regex_template` options in `ModulesTool.exist` method](removed-functionality.md#ModulesTool-exist-mod_exists_regex_template) +- [`mod_exists_regex_template` option in `ModulesTool.exist` method](removed-functionality.md#ModulesTool-exist-mod_exists_regex_template) - [`Toolchain.add_dependencies` method](removed-functionality.md#Toolchain-add_dependencies) - [`disable_templating` + `default_fallback` options in `get_easyblock_class` function](removed-functionality.md#get_easyblock_class) - [`skip_lower` option from `template_constant_dict` function](removed-functionality.md#template_constant_dict) @@ -143,13 +137,6 @@ Some functionality is being deprecated in EasyBuild v5.0, and is scheduled to be - [Toolchain support policy](../policies/toolchains.md) ---- - -## FAQ - -*(coming soon)* - - --- ## Other links diff --git a/docs/easybuild-v5/removed-functionality.md b/docs/easybuild-v5/removed-functionality.md index 1425af227..c2772bed5 100644 --- a/docs/easybuild-v5/removed-functionality.md +++ b/docs/easybuild-v5/removed-functionality.md @@ -42,93 +42,108 @@ format and little suggestion that anyone was using it at all. --- -## `accept-eula` configuration setting] {: #accept-eula } +## `--accept-eula` configuration setting] {: #accept-eula } -*(more info soon)* +The `--accept-eula` option was changed to `--accept_eula_for` in EasyBuild 4.3.4 where it was also enhanced to +support reular expressions. Use of `--accept-eula` has now been removed. --- ## `--wait-on-lock` configuration setting {: #wait-on-lock } -*(more info soon)* +The `--wait-on-lock` option was replaced and split into `--wait-on-lock-limit` and `--wait-on-lock-interval` in +EasyBuild 4.2.1. Use of `--wait-on-lock` has now been removed. --- ## `is_generic_easyblock` function from `easybuild.framework.easyconfig.easyconfig` {: #easyconfig-is_generic_easyblock } -*(more info soon)* +`is_generic_easyblock` was moved to `easybuild.tools.filetools` in EasyBuild 4.2.0. Use of `is_generic_easyblock` +from `easybuild.framework.easyconfig.easyconfig` has been removed. --- ## `copytree`, `rmtree2` functions from `easybuild.filetools` {: #filetools-copytree-rmtree2 } -*(more info soon)* +The use of `rmtree2` was replaced by `remove_dir` and the use of `copytree` by `copy_dir` in EasyBuild 4.2.0. +Use of `rmtree2` and `copytree`, from `easybuild.filetools`, has now been removed. --- ## `fetch_extension_sources` method in `EasyBlock` class {: #EasyBlock-fetch_extension_sources } -*(more info soon)* +`fetch_extension_sources` was replaced by `collect_exts_file_info`. in EasyBuild 4.5.0. Use of +`fetch_extension_sources` has now been removed. --- -## `mod_exists_regex_template` options in `ModulesTool.exist` method {: #ModulesTool-exist-mod_exists_regex_template } +## `mod_exists_regex_template` option in `ModulesTool.exist` method {: #ModulesTool-exist-mod_exists_regex_template } -*(more info soon)* +The `mod_exists_regex_template` option in `ModulesTool.exist` was unused and has now been removed. --- ## `Toolchain.add_dependencies` method {: #Toolchain-add_dependencies } -*(more info soon)* +`Toolchain.add_dependencies` has been removed. The dependencies should be passed to the `prepare` +method instead, using the `deps` argument. --- ## `disable_templating` + `default_fallback` options in `get_easyblock_class` function {: #get_easyblock_class } -*(more info soon)* +`disable_templating` was changed to be a function on the EasyConfig class in EasyBuild 4.4.0. +`disable_templating` and the `default_fallback` option for `get_easyblock_class` have now been removed. --- ## `skip_lower` option from `template_constant_dict` function {: #template_constant_dict } -*(more info soon)* +The `skip_lower` option in `template_constant_dict` had no effect in EasyBuild 4 and has now been +removed. --- ## `use_git_am` option for `apply_patch` function {: #apply_patch-use_git_am } -*(more info soon)* +The `use_git_am` option for `apply_patch` was renamed to `use_git` in EasyBuild 4.3.0 . This option +has now been removed. --- ## `skip_symlinks` option for `adjust_permissions` function {: #adjust_permissions-skip_symlinks } -*(more info soon)* +The `skip_symlinks` option in the `adjust_permissions` function has been removed. Symlinks are not +followed in this function. --- ## `log_error` option in `which` function {: #which-log_error } -*(more info soon)* +The `log_error` option in the `which` function was changed to `on_error` in EasyBuild 4.4.0 to allow +enhanced options on what to do when an error is encountered in this function. This option has now +been removed. --- ## `descr` option for `simple_option` function {: #simple_option-descr } -*(more info soon)* +The `descr` option in the `simple_option` function was removed. This has been unused in EasyBuild for +a long time. --- ## `dummy` toolchain {: #dummy-toolchain } -*(more info soon)* +The `dummy` toolchain was replaced by the `SYSTEM` toolchain in EasyBuild 4. The `dummy` toolchain has +now been removed. As part of this the `--add-system-to-minimal-toolchains` option replaces +`--add-dummy-to-minimal-toolchains`. --- ## Support for 32-bit targets {: #32-bit-targets } -*(more info soon)* +EasyBuild has support for 32-bit targets for `imkl` versions before 11. These have now been removed. --- diff --git a/docs/easystack-files.md b/docs/easystack-files.md index 897bca33b..5d665c64b 100644 --- a/docs/easystack-files.md +++ b/docs/easystack-files.md @@ -4,7 +4,7 @@ This documentation covers aspects of specifying a software stack to install with Easybuild with *easystack files*. !!! note - Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0. + Some breaking changes were made to the easystack support in EasyBuild v4.7.0. Easystack files must now use the `easyconfigs` top-level key to list *easyconfig filenames*, as opposed to the `software` top-level key and separate subkeys like `version`, `versionsuffix` and @@ -21,7 +21,7 @@ specify to EasyBuild what to install. To build software with *Easystack*, type: ``` shell -eb --easystack example.yaml --experimental +eb --easystack example.yaml ``` where `example.yaml` is the file with specifications that you just @@ -136,7 +136,7 @@ easyconfigs: and you run with ``` shell -eb --experimental --easystack my_easystack.yaml --dry-run +eb --easystack my_easystack.yaml --dry-run ``` this will have the same effect as running @@ -163,7 +163,7 @@ they take priority over the the ones on the command line, if the same configurat For example, running: ``` shell -eb --experimental --easystack my_easystack.yaml --disable-debug +eb --easystack my_easystack.yaml --disable-debug ``` will effectively cause `PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb` to be installed with debug logging enabled, diff --git a/docs/installing-extensions-in-parallel.md b/docs/installing-extensions-in-parallel.md index 5e28f20ed..68f8726b1 100644 --- a/docs/installing-extensions-in-parallel.md +++ b/docs/installing-extensions-in-parallel.md @@ -11,10 +11,6 @@ in the background, to exploit the availability of multiple cores. To let EasyBuild install extensions in parallel, the `parallel-extensions-install` configuration option must be enabled. -In addition, since the support for installing extensions in parallel is -an experimental feature, the `experimental` configuration option must be -also be enabled (see also [Experimental features][experimental_features]). - How many extension installations should be started (at maximum) at the same time is controlled by the `parallel` configuration option. @@ -45,9 +41,6 @@ There are a couple of caveats and known issues with installing extensions in parallel, which we hope to resolve in future EasyBuild releases. -This is also why the support for installing extensions in parallel is -currently marked as an experimental feature. - ### Only works for R extensions {: #installing_extensions_in_parallel_caveats_known_issues_only_r } The support for installing extensions in parallel currently only works diff --git a/docs/submitting-jobs.md b/docs/submitting-jobs.md index 554659ee1..e1d1dce7b 100644 --- a/docs/submitting-jobs.md +++ b/docs/submitting-jobs.md @@ -24,17 +24,17 @@ by the software dependency graph(s). The job backend to be used can be specified using the `--job-backend` EasyBuild configuration option. -Since EasyBuild 3.8.0, three backends are supported: +Three backends are supported: -* `GC3Pie` *(default)* (supported since EasyBuild 2.2.0) - * `GC3Pie` version 2.5.0 (or more recent) required () - * works with different resource managers and job schedulers, including TORQUE/PBS, Slurm, etc. - * **note**: requires that a GC3Pie configuration file is provided, see [Configuring the job backend][submitting_jobs_cfg_job_backend_config] +* `Slurm` *(default)* + * requires Slurm version 17.0 (or more recent), () * `PbsPython` * `pbs_python` version 4.1.0 (or more recent) required () * **note**: requires TORQUE resource manager () -* `Slurm` (supported since EasyBuild 3.8.0) - * requires Slurm version 17.0 (or more recent), () +* `GC3Pie` *(deprecated)* + * `GC3Pie` version 2.5.0 (or more recent) required () + * works with different resource managers and job schedulers, including TORQUE/PBS, Slurm, etc. + * **note**: requires that a GC3Pie configuration file is provided, see [Configuring the job backend][submitting_jobs_cfg_job_backend_config] ### Configuring the job backend (`--job-backend-config`) {: #submitting_jobs_cfg_job_backend_config } @@ -59,8 +59,8 @@ which job backend is being used: * if the `PbsPython` job backend is used, the (most common) number of available cores per workernode in the target resource is determined; this usually results in jobs requesting full workernodes (at least in terms of cores) by default -* if the `GC3Pie` or `Slurm` job backend is used, the requested number of cores is left unspecified, - which results in falling back to the default mechanism used by GC3Pie/Slurm to pick a number of cores; +* if the `Slurm` or `GC3Pie` job backend is used, the requested number of cores is left unspecified, + which results in falling back to the default mechanism used by Slurm/GC3Pie to pick a number of cores; most likely, this results in single-core jobs to be submitted by default @@ -79,7 +79,7 @@ can be specified via the `--job-deps-type` configuration setting: The default value for `-job-deps-type` depends on the job backend being used (see [Configuring the job backend][submitting_jobs_cfg_job_backend_config): -* for the `GC3Pie` and `Slurm` backends, `--job-deps-type=abort_on_error` is the default; +* for the `Slurm` and `GC3Pie` backends, `--job-deps-type=abort_on_error` is the default; * for the `PbsPython` backend, `--job-deps-type=always_run` is the default (because of historical reasons, and for the sake of backward compatibility)