Skip to content

Conference call notes 20240228

Kenneth Hoste edited this page Feb 28, 2024 · 2 revisions

(back to Conference calls)

Notes on the 241th EasyBuild conference call, Wednesday 28 Feb 2024 (09:00 UTC)

Attendees

List of attendees (19):

  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Davide Grassano (CECAM, Switzerland)
  • Jasper Grimm (University of York, UK)
  • Alexander Grund (TU Dresden, Germany)
  • Jorge Guerra (Universidad Politécnica de Madrid, Spain)
  • Sven Hansen (RWTH, Germany)
  • Thomas Hoffman (EMBL, Heidelberg, Germany)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Kurt Lust (UAntwerpen, Belgium + LUMI User Support Team)
  • Sam Moors (Vrije Universiteit Brussel, Belgium)
  • Sébastien Moretti (SIB, Switzerland)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Jure Pečar (EMBL, Heidelberg, Germany)
  • Jörg Saßmannshausen (Imperial College London, UK)
  • Åke Sandgren (Umeå University, Sweden)
  • Alexandre Strube (JSC, Germany)
  • Alain van Hoof (TU Eindhoven, Netherlands)
  • Lars Viklund (Umeå University, Sweden)

Agenda

  • overview of recent developments
  • Q&A

Recent developments

  • latest EasyBuild release: 4.9.0 (30 Dec 2023)
  • next EasyBuild release: hopefully EasyBuild v5.0.0
    • there will only be an EasyBuild v4.9.1 if there's a clear need for it, or if getting EasyBuild 5.0 is taking too long (way beyond 8 weeks after v4.9.0 release)
  • ETA for EasyBuild 5.0 release: hopefully first EasyBuild release of 2024
    • started doing short sprint meetings, each Monday at 15:00 CE(S)T to set next 5 goals to tackle that week
    • monthly sync meetings every 1st Wed of the month at 14:00 CE(S)T: next today Wed 7 Feb'24 at 14:00 CET
    • see also https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
    • to test development version of EasyBuild 5.0:
      # set up Python virtual environment, and jump into it
      python3 -m venv eb5
      source eb5/bin/activate
      
      # install EasyBuild 5.0 development version into it
      pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
      pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
      pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz
      
      # go!
      eb --version
  • recent changes
    • docs (merged PRs)
      • ...
    • framework (merged PRs)
      • bug fixes
        • use cp -dR instead of cp -a for shell script extraction (PR #4465)
      • enhancements
        • ...
      • changes
      • code cleanup
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • bug fixes
          • ...
        • changes
          • ...
        • enhancements
          • add support for running shell commands asynchronously with run_shell_cmd (PR #4444)
            • there's still a bug there somewhere, test_toy_exts_parallel sometimes fails in CI (more often on Python 3.6)
          • allow only alphanumeric characters in the output filename used in run_shell_cmd (PR #4454)
          • enhance download instructions by mentioning active source path (PR #4459)
          • add cuda compute capability integer format templates (PR #4463)
    • easyblocks (merged PRs)
      • bug fixes
        • ...
      • enhancements
        • honor pretestopts & testopts in PerlModule (PR #3231)
      • updates
        • ...
      • changes
        • ...
      • new easyblocks
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • bug fixes:
          • ...
        • changes:
          • update run_async methods in RPackage, Rmpi, and Rserve easyblocks to use submit run_shell_cmd call into thread pool (PR #3163)
    • easyconfigs (merged PRs)
      • over 100 easyconfig PRs were merged since last conf call
      • bug fixes/reports
        • remove numa configure option from hwloc 2+ (PR #19833)
        • add patch to fix MS_RDONLY error in OpenMPI 4.0.x (PR #19837, PR #19944)
        • Fix test failures of Perl-bundle-CPAN-5.38 in non-English locales (PR #19971)
        • add patch for adding a write memory barrier to all OpenMPI 4.1.x easyconfigs (PR #19940)
        • stop using non-existent --disable-libdeflate option for LibTIFF 4.1.0 (PR #19951)
        • add patch to remove git version check and fix test command for molmod 1.4.8 (PR #19952)
        • stop using non-existing --with-doc configure option for groff 1.23 (PR #19969)
        • add patch to fix tree-optimization bug in GCC 13.2.0 for ARM architectures with SVE support (PR #19974)
      • enhancements
        • ...
      • (noteworthy) new software
        • ...
      • noteworthy software updates
        • PyTorch v2.1.2 w/ foss/2023a + CUDA 12.1.1 (PR #19666)
          • max_failed_tests sets to 50
          • common causes of failures in PyTorch test suite
            • running with random seed
            • use of hypothesis by generating random inputs for tests
            • actual bugs in compiler, OpenMPI, OpenBLAS, ...
          • can we reach out to PyTorch team to give feedback on this?
          • ideally there would be a subset of tests that should definitely never fail, more reliable way of checking the installation
          • follow-up PRs that depends on PyTorch, should introduce PyTorch-bundle first (see easyconfigs PR #19987)
            • Alex: older ignite version required for MONAI, can still depend on PyTorch-bundle + add older ignite version as extension to MONAI easyconfig
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • add download_instructions to various easyconfigs (~40 PRs)
        • detect easyconfigs where missing information about downloading sources (PR #19881)
    • docs (open PRs + issues)
    • framework (open PRs + issues)
      • reported bugs / bug fixes
        • Run unit tests on an updated version of Modules 4 (PR #4415)
      • enhancements
        • add terse support to --missing-modules (PR #4407)
        • add new 'authors' field to detailed JSON software listing (PR #4460)
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • eb command fails outside of framework source folder (issue #4451)
        • active PRs
          • change tar command used in get_source_tarball_from_git to get reproducible tarballs (PR #4248)
            • Alex has some idea on how to ensure that tarball created for sources obtained via git_config are identical across Linux & macOS
          • rename unclear *run* methods to *install_extension* + rename install_extensions to install_all_extensions (PR #4400)
            • problems fixed by Alex, ready for review/merge
          • implement support for running interactive commands with run_shell_cmd (WIP PR #4453)
            • implementation is slowly progressing...
          • refactor easyconfig parameters and template constants to use a consistent naming style (issue #4464)
        • dormant issues/PRs (for now)
          • deprecate failure to resolve a template value (PR #3285)
          • drop load storm safe guard for Environment Modules v4.2.4+ (PR #4373)
            • we should also enable depends_on by default when using Lmod as modules tool in EasyBuild 5.0
          • replace parallel easyconfig parameter by maxparallel (PR #4398)
          • bump minimum required Tmod (4.x) to 4.3.0 (PR #4425)
            • requires that PR #4415 is merged first
          • deprecate run_cmd and run_cmd_qa & co, move them to easybuild._deprecated module (WIP) (PR #4433)
            • requires that implementation of run_shell_cmd is complete, incl. support for asynchronously running commands and running interactive shell commands (WIP)
          • support uninstalling software + module (issue #590)
          • clean up SLURM/Batchsystem environment before doing builds (issue #4434)
            • unset all $SLURM_* environment variables to avoid trouble with srun/MPI
            • should be done opt-in (--clean-slurm-env configuration option)
    • easyblocks (open PRs + issues)
      • bug reports/fixes
        • remove trailing slash in SLEPC_DIR (PR #3229)
        • fix OpenFOAM easyblock to correctly have is_dot_org as True when v >=11 and fix motorBike example for v >=11 (PR #3232)
      • enhancements
        • Support GROMACS builds with SYCL (AdaptiveCpp implementation) (PR #3077)
        • use proxy in sanity check of EB_OpenSSL_wrapper (PR #2922)
        • Score-P easyblock: Select the oneAPI compiler suite when enabled in the Intel compiler toolchain (PR #3228)
        • enhance CMakeMake easyblock to check whether correct Python installation was picked up by cmake (PR #3233)
        • Make OpenBLAS easyblock aware to --optarch=GENERIC (issue #3230)
      • updates
        • ...
      • new easyblocks
        • ...
      • changes
        • Always set $EBPYTHONPREFIXES instead of $PYTHONPATH (issue #2887)
      • EasyBuild 5.0 (to 5.0.x branch)
        • fix extension filter for Perl packages (PR #2699)
        • update ConfigureMake easyblock to error out on unknown configure args (PR #3025 + #3026)
        • rename run method to install_extension, and likewise for prerun to pre_install_extension, postrun to post_install_extension, and run_async to install_extension_async (PR #3064)
        • set CMake installation LIBDIR to lib by default in CMakeMake easyblock (PR #3227)
        • ~15 open PRs to replace run_cmd with run_shell_cmd in custom easyblocks
          • see also overview in issue #3089
          • all help to migrate easyblocks to run_shell_cmd is welcome!
        • ~5 open PRs to remove unused custom easyblocks
    • easyconfigs (open PRs + issues)
      • bug fixes/reports
        • ...
      • enhancements
        • ...
      • new software
        • ...
      • noteworthy software updates
        • Qt5 v5.15.11 w/ GCCcore/13.2.0 (for foss/2023b) (WIP PR #19320)
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • replace parallel by maxparallel (PR #19375)
        • detect easyconfigs where missing information about downloading sources (PR #19881)
      • closed PRs
        • ...

Q&A / others

  • EasyBuild User Meeting 2024 - agenda
    • https://easybuild.io/eum
    • suggestions for talk welcome!
    • draft agenda is being puzzled together behind the scenes
    • suggestions/ideas for talks still welcome!
  • We may be a bit too aggressive with compiler version w.r.t. being able to combine it with latest CUDA/NVCC
  • Jörg: some software requires citing the use of it in paper
    • can we try honoring this
    • for example if there's a CITATION file in the source tarballs
Clone this wiki locally