From 847bb9e77dc0177171e143375cd044c4d7317163 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 14:10:33 +0100 Subject: [PATCH 01/12] add R module set R libs ilocation docs --- config/templates/hpc.template | 1 + mkdocs/docs/HPC/rlibs.md | 40 +++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100644 mkdocs/docs/HPC/rlibs.md diff --git a/config/templates/hpc.template b/config/templates/hpc.template index 04da766e0524..622f14417b18 100644 --- a/config/templates/hpc.template +++ b/config/templates/hpc.template @@ -59,6 +59,7 @@ nav: - OpenFOAM: openFOAM.md - Python: python.md - Python virtual environments: setting_up_python_virtual_environments.md + - R Libraries: rlibs.md - Transcribe: transcribe.md - VS Code Tunnel: vscodetunnel.md - FAQ: diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md new file mode 100644 index 000000000000..800b13a3bc67 --- /dev/null +++ b/mkdocs/docs/HPC/rlibs.md @@ -0,0 +1,40 @@ +# R libraries + +Please keep in mind, that this is not a genral manual +abut installing `R` libraries, but only system specific information about self +installed R libraries at {{ hpcinfra }}. + +Our infrastructure contains different CPU arcitectures, that are +not neccesarily compatible with each other. Additionally, we have +multiple versions of `R` installed with different versions of toolchains, and by default, +self installed libraries are installed in separate directories based only +on R major and minor versions. + +In this way, you might use self installed R libraries on incopatible architectures and/or +compiled with incompatible toolchains, which might lead to hangs or other types of errors. + +When you load a centrally installed `R` module we make sure that self installed libraries +will be installed to an operational system, arhictecture, toolchain, and `R` version +dependent directory. The loaction of this directory by default is +(by setting `R_LIBS_USER` environmental variable to) +`$VSC_DATA/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. + +If you would like to have another location (for example you are using a central location +in your VO), you should set the environmental variable `R_LIBS_BASEDIR` to the desired location +*before* you load any centrally installed `R` module. In this case, the location of the +directory for self installed `R` libraries will be +`$R_LIBS_BASEDIR/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. +If this directory is not writtable, you can still be able to use already installed +`R` libararies from that location, but you will not be able to install `R` libarires +yourself. + +You can always check the default location(s) of `R` libraries by issuing `.libPaths()` command +in `R`. This command will show the library locations ordered by their priorites. + +Please be aware that if you have installed an `R` libaries yourself in the past, you might have to +reinstall those if you want to use them: +- on a different cluster +- with a different version of `R` +- with an `R` compiled with a different version of toolchain +- if the operational system is changed/updated. + From 3d33518c5eb9b84bcd065ad87acbfdb91d86d496 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 14:17:26 +0100 Subject: [PATCH 02/12] fix typos --- mkdocs/docs/HPC/rlibs.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md index 800b13a3bc67..1e1a206429c1 100644 --- a/mkdocs/docs/HPC/rlibs.md +++ b/mkdocs/docs/HPC/rlibs.md @@ -1,21 +1,21 @@ # R libraries -Please keep in mind, that this is not a genral manual +Please keep in mind, that this is not a general manual abut installing `R` libraries, but only system specific information about self installed R libraries at {{ hpcinfra }}. -Our infrastructure contains different CPU arcitectures, that are -not neccesarily compatible with each other. Additionally, we have +Our infrastructure contains different CPU architectures, that are +not necessarily compatible with each other. Additionally, we have multiple versions of `R` installed with different versions of toolchains, and by default, self installed libraries are installed in separate directories based only on R major and minor versions. -In this way, you might use self installed R libraries on incopatible architectures and/or +In this way, you might use self installed R libraries on incompatible architectures and/or compiled with incompatible toolchains, which might lead to hangs or other types of errors. When you load a centrally installed `R` module we make sure that self installed libraries -will be installed to an operational system, arhictecture, toolchain, and `R` version -dependent directory. The loaction of this directory by default is +will be installed to an operational system, architecture, toolchain, and `R` version +dependent directory. The location of this directory by default is (by setting `R_LIBS_USER` environmental variable to) `$VSC_DATA/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. @@ -24,14 +24,14 @@ in your VO), you should set the environmental variable `R_LIBS_BASEDIR` to the d *before* you load any centrally installed `R` module. In this case, the location of the directory for self installed `R` libraries will be `$R_LIBS_BASEDIR/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. -If this directory is not writtable, you can still be able to use already installed -`R` libararies from that location, but you will not be able to install `R` libarires +If this directory is not writable, you can still be able to use already installed +`R` libraries from that location, but you will not be able to install `R` libraries yourself. You can always check the default location(s) of `R` libraries by issuing `.libPaths()` command -in `R`. This command will show the library locations ordered by their priorites. +in `R`. This command will show the library locations ordered by their priorities. -Please be aware that if you have installed an `R` libaries yourself in the past, you might have to +Please be aware that if you have installed an `R` libraries yourself in the past, you might have to reinstall those if you want to use them: - on a different cluster - with a different version of `R` From 2a80b22915fb81c06a8f29c8d29966de52a3140f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 14:19:14 +0100 Subject: [PATCH 03/12] fix typos --- mkdocs/docs/HPC/rlibs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md index 1e1a206429c1..ac8d8d0b06b1 100644 --- a/mkdocs/docs/HPC/rlibs.md +++ b/mkdocs/docs/HPC/rlibs.md @@ -8,7 +8,7 @@ Our infrastructure contains different CPU architectures, that are not necessarily compatible with each other. Additionally, we have multiple versions of `R` installed with different versions of toolchains, and by default, self installed libraries are installed in separate directories based only -on R major and minor versions. +on `R` major and minor versions. In this way, you might use self installed R libraries on incompatible architectures and/or compiled with incompatible toolchains, which might lead to hangs or other types of errors. From 8e9b716f9365b7c49131b45e9cd7f4f2d0157439 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 14:20:52 +0100 Subject: [PATCH 04/12] fix typos --- mkdocs/docs/HPC/rlibs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md index ac8d8d0b06b1..d71ddf1764b2 100644 --- a/mkdocs/docs/HPC/rlibs.md +++ b/mkdocs/docs/HPC/rlibs.md @@ -10,7 +10,7 @@ multiple versions of `R` installed with different versions of toolchains, and by self installed libraries are installed in separate directories based only on `R` major and minor versions. -In this way, you might use self installed R libraries on incompatible architectures and/or +In this way, you might use self installed `R` libraries on incompatible architectures and/or compiled with incompatible toolchains, which might lead to hangs or other types of errors. When you load a centrally installed `R` module we make sure that self installed libraries From a7ae732ebfcc6bd4040241ebaf73eb9f0eba3dc4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 15:07:09 +0100 Subject: [PATCH 05/12] Apply suggestions from code review Co-authored-by: Kenneth Hoste --- config/templates/hpc.template | 2 +- mkdocs/docs/HPC/rlibs.md | 38 +++++++++++++++++++---------------- 2 files changed, 22 insertions(+), 18 deletions(-) diff --git a/config/templates/hpc.template b/config/templates/hpc.template index 622f14417b18..86c8b8a97433 100644 --- a/config/templates/hpc.template +++ b/config/templates/hpc.template @@ -59,7 +59,7 @@ nav: - OpenFOAM: openFOAM.md - Python: python.md - Python virtual environments: setting_up_python_virtual_environments.md - - R Libraries: rlibs.md + - R packages: r-packages.md - Transcribe: transcribe.md - VS Code Tunnel: vscodetunnel.md - FAQ: diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md index d71ddf1764b2..7d3223550f1d 100644 --- a/mkdocs/docs/HPC/rlibs.md +++ b/mkdocs/docs/HPC/rlibs.md @@ -1,37 +1,41 @@ -# R libraries +# R packages -Please keep in mind, that this is not a general manual -abut installing `R` libraries, but only system specific information about self -installed R libraries at {{ hpcinfra }}. +!!! note + Please keep in mind, that this is not general documentation + about installing `R` packages, but specific information about + self-installed R packages at the {{ hpcinfra }}. -Our infrastructure contains different CPU architectures, that are +The {{ hpcinfra }} contains different generations of CPUs, with different microarchitectures, that are not necessarily compatible with each other. Additionally, we have multiple versions of `R` installed with different versions of toolchains, and by default, self installed libraries are installed in separate directories based only -on `R` major and minor versions. +on `R` major and minor versions (only using `x.y` for R version `x.y.z`). -In this way, you might use self installed `R` libraries on incompatible architectures and/or -compiled with incompatible toolchains, which might lead to hangs or other types of errors. +In this way, you might use self-installed `R` packages on incompatible CPU microarchitectures and/or +compiled with incompatible toolchains, or for a different major version of the operating system, +which might lead to hangs or other types of errors. -When you load a centrally installed `R` module we make sure that self installed libraries -will be installed to an operational system, architecture, toolchain, and `R` version -dependent directory. The location of this directory by default is -(by setting `R_LIBS_USER` environmental variable to) +Since end of February 2025, we make sure that `R` packages that you install yourself will be installed +in a directory that is specific to the operating system version, CPU microarchitecture, toolchain, +and `R` version that is being used. The location of this directory by default is +(by setting `$R_LIBS_USER` environment variable to) `$VSC_DATA/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. +### Controlling the location for self-installed R packages + If you would like to have another location (for example you are using a central location -in your VO), you should set the environmental variable `R_LIBS_BASEDIR` to the desired location +in your Virtual Organisation), you should set the environment variable `$R_LIBS_BASEDIR` to the desired location *before* you load any centrally installed `R` module. In this case, the location of the -directory for self installed `R` libraries will be +directory for self-installed `R` packages will be `$R_LIBS_BASEDIR/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. If this directory is not writable, you can still be able to use already installed -`R` libraries from that location, but you will not be able to install `R` libraries +`R` packages from that location, but you will not be able to install `R` packages yourself. -You can always check the default location(s) of `R` libraries by issuing `.libPaths()` command +You can always check the default location(s) of `R` packages by issuing `.libPaths()` command in `R`. This command will show the library locations ordered by their priorities. -Please be aware that if you have installed an `R` libraries yourself in the past, you might have to +Please be aware that if you have installed an `R` packages yourself in the past (before end of February 2025), you might have to reinstall those if you want to use them: - on a different cluster - with a different version of `R` From fa797e98e9359c22b5f2434e4b3f3ef6bd0989e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 15:08:50 +0100 Subject: [PATCH 06/12] add section title --- mkdocs/docs/HPC/rlibs.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/rlibs.md index 7d3223550f1d..95faeaade5da 100644 --- a/mkdocs/docs/HPC/rlibs.md +++ b/mkdocs/docs/HPC/rlibs.md @@ -35,8 +35,10 @@ yourself. You can always check the default location(s) of `R` packages by issuing `.libPaths()` command in `R`. This command will show the library locations ordered by their priorities. -Please be aware that if you have installed an `R` packages yourself in the past (before end of February 2025), you might have to -reinstall those if you want to use them: +### Missing R packages intalled before + +Please be aware that if you have installed an `R` packages yourself in the past (before end of February 2025), +you might have to reinstall those if you want to use them: - on a different cluster - with a different version of `R` - with an `R` compiled with a different version of toolchain From ae9db83378bb0fcf858b130c258cc3d570acbcf8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Wed, 26 Feb 2025 16:07:04 +0100 Subject: [PATCH 07/12] rename rlibs.md to r-packages.md --- mkdocs/docs/HPC/{rlibs.md => r-packages.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename mkdocs/docs/HPC/{rlibs.md => r-packages.md} (100%) diff --git a/mkdocs/docs/HPC/rlibs.md b/mkdocs/docs/HPC/r-packages.md similarity index 100% rename from mkdocs/docs/HPC/rlibs.md rename to mkdocs/docs/HPC/r-packages.md From 1207481af61ce165e8c908412133d25d1ce0d1fc Mon Sep 17 00:00:00 2001 From: Kenneth Hoste Date: Wed, 26 Feb 2025 16:46:11 +0100 Subject: [PATCH 08/12] final minor tweaks to docs on R package installation location + fix formatting of bullet list at the end --- mkdocs/docs/HPC/r-packages.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mkdocs/docs/HPC/r-packages.md b/mkdocs/docs/HPC/r-packages.md index 95faeaade5da..6f26e40c40ad 100644 --- a/mkdocs/docs/HPC/r-packages.md +++ b/mkdocs/docs/HPC/r-packages.md @@ -1,14 +1,14 @@ # R packages !!! note - Please keep in mind, that this is not general documentation + Please keep in mind that this is not general documentation about installing `R` packages, but specific information about self-installed R packages at the {{ hpcinfra }}. The {{ hpcinfra }} contains different generations of CPUs, with different microarchitectures, that are not necessarily compatible with each other. Additionally, we have multiple versions of `R` installed with different versions of toolchains, and by default, -self installed libraries are installed in separate directories based only +self-installed packages are installed in separate directories based only on `R` major and minor versions (only using `x.y` for R version `x.y.z`). In this way, you might use self-installed `R` packages on incompatible CPU microarchitectures and/or @@ -28,17 +28,18 @@ in your Virtual Organisation), you should set the environment variable `$R_LIBS_ *before* you load any centrally installed `R` module. In this case, the location of the directory for self-installed `R` packages will be `$R_LIBS_BASEDIR/local_R_LIBS/$VSC_OS_LOCAL/$VSC_ARCH_LOCAL/-`. -If this directory is not writable, you can still be able to use already installed +If this directory is not writable, you will still be able to use already installed `R` packages from that location, but you will not be able to install `R` packages yourself. You can always check the default location(s) of `R` packages by issuing `.libPaths()` command -in `R`. This command will show the library locations ordered by their priorities. +in `R`. This command will show the package locations, ordered by their priorities. ### Missing R packages intalled before Please be aware that if you have installed an `R` packages yourself in the past (before end of February 2025), you might have to reinstall those if you want to use them: + - on a different cluster - with a different version of `R` - with an `R` compiled with a different version of toolchain From 5d4e8981b1c11b5e992522a78172031febe789a2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Thu, 27 Feb 2025 08:47:48 +0100 Subject: [PATCH 09/12] Update mkdocs/docs/HPC/r-packages.md Co-authored-by: Kenneth Hoste --- mkdocs/docs/HPC/r-packages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/HPC/r-packages.md b/mkdocs/docs/HPC/r-packages.md index 6f26e40c40ad..a7bb991d4230 100644 --- a/mkdocs/docs/HPC/r-packages.md +++ b/mkdocs/docs/HPC/r-packages.md @@ -43,5 +43,5 @@ you might have to reinstall those if you want to use them: - on a different cluster - with a different version of `R` - with an `R` compiled with a different version of toolchain -- if the operational system is changed/updated. +- if the operation system was changed/updated. From b0c3aa38eed492b35efcc7fe4ab805795b9c8fef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Thu, 27 Feb 2025 09:02:35 +0100 Subject: [PATCH 10/12] polishing the text --- mkdocs/docs/HPC/r-packages.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mkdocs/docs/HPC/r-packages.md b/mkdocs/docs/HPC/r-packages.md index a7bb991d4230..7d65cde9aa27 100644 --- a/mkdocs/docs/HPC/r-packages.md +++ b/mkdocs/docs/HPC/r-packages.md @@ -37,8 +37,9 @@ in `R`. This command will show the package locations, ordered by their prioritie ### Missing R packages intalled before -Please be aware that if you have installed an `R` packages yourself in the past (before end of February 2025), -you might have to reinstall those if you want to use them: +Please be aware that if you have installed `R` packages yourself before February 2025, +then you have to reinstall all of them. You might have to reinatsll self +installed `R` packages if you want to use them: - on a different cluster - with a different version of `R` From 6bed13245ee724077e07d9207f4f99557932cd62 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Thu, 27 Feb 2025 09:09:48 +0100 Subject: [PATCH 11/12] polishing the text --- mkdocs/docs/HPC/r-packages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/HPC/r-packages.md b/mkdocs/docs/HPC/r-packages.md index 7d65cde9aa27..5c95f33a5dd7 100644 --- a/mkdocs/docs/HPC/r-packages.md +++ b/mkdocs/docs/HPC/r-packages.md @@ -38,7 +38,7 @@ in `R`. This command will show the package locations, ordered by their prioritie ### Missing R packages intalled before Please be aware that if you have installed `R` packages yourself before February 2025, -then you have to reinstall all of them. You might have to reinatsll self +then you have to reinstall all of them. You might have to reinstall self installed `R` packages if you want to use them: - on a different cluster From 873cdcae8d30e5576bd0ec4613ee8fa7baeef9f4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bal=C3=A1zs=20Hajgat=C3=B3?= Date: Thu, 27 Feb 2025 10:22:28 +0100 Subject: [PATCH 12/12] Apply suggestions from code review Co-authored-by: Kenneth Hoste --- mkdocs/docs/HPC/r-packages.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mkdocs/docs/HPC/r-packages.md b/mkdocs/docs/HPC/r-packages.md index 5c95f33a5dd7..11610c502cc9 100644 --- a/mkdocs/docs/HPC/r-packages.md +++ b/mkdocs/docs/HPC/r-packages.md @@ -37,12 +37,12 @@ in `R`. This command will show the package locations, ordered by their prioritie ### Missing R packages intalled before -Please be aware that if you have installed `R` packages yourself before February 2025, +Please be aware that if you have installed `R` packages yourself before end of February 2025, then you have to reinstall all of them. You might have to reinstall self installed `R` packages if you want to use them: - on a different cluster - with a different version of `R` - with an `R` compiled with a different version of toolchain -- if the operation system was changed/updated. +- if the operating system was changed/updated.