Wishlist

Here is a list of features that are desired to have in next versions of EasyBuild: (importance of them varies according to who you ask ;-)

USER-oriented features:

• Add possibility to save the logfile somewhere even when failures happen (/tmp is no good for queue submitted jobs)
• interim solution is to use -ld; OK.
• ANSWER: "/tmp" is hardcoded now, so an issue needs to be opened for this; See #84
• Consider support of "formal" language to describe package info & dependencies, eg. CUDF syntax
• http://blog.mancoosi.org
• http://www.mancoosi.org/papers/cbse11.pdf "MPM: a modular package manager"
• http://arxiv.org/abs/1109.0456 "Aligning component upgrades"
• ANSWER: may consider; TBD
• Consider category-oriented module sets (eg. module load bioinformatics-tools)
• may be done outside of EB logic, IFF the meaning is to mangle MODULEPATH? "moduleclass" can help
• ANSWER: either as said OR, produce an easyconfig with multiple dependencies; TBD
• Allow for custom-definition of easyblock paths (instead of enforcing any "a-z/" dir structure
• ANSWER: it seems to be OK w. PYTHON_PATH? nope, the a-z dirs need to have "reverse" mapping (ref. ".."); TBD
• How to organize a collection of .eb files coming from different sources? (eg. distinct git repos)
• Seen at http://joeyh.name/code/mr/ : "When updating a git repository, pull from two different upstreams and merge the two together."
• ie. can EasyBuild --robot dependency resolution support multiple paths?
• ANSWER: will support multiple directories; prefer to use many indeed
• Consider integrated support for relocatable .rpms (with --prefix), .srpms & RPM's .spec files
• in effect, let's create a handler for non "*.eb" suffixes to be managed externally; that would allow to create from eg. a .spec file (for RPMs) the equivalent .eb, then parse that from "stdio"
• ANSWER: it's already supported, but in previous versions; to be cleanup
• extreme thought: yes, this is an extreme thought: consider module search/install extensions!
• ANSWER: or try it with "module load" instead ;-)
• promote easybuild within bioinformatics community, esp. for "clinical or translational workflows", as described below: http://www.bioinfo-core.org/index.php/11th_Discussion-7_November_2011#Topic_2:_Managing_and_Tracking_Software_Updates_.28Led_by_Brent_Richter.29
• ANSWER: makes sense to approach communities; TBD
• support cmake-like builds (see example below)
• ANSWER: for the default case an easyblock is upcoming; TBD
• how to override certain steps ("/bin/true" is considered a hack as of now)
• ANSWER: to be discussed; first create wiki page with all the [different ports conventions]
• Consider the bigger picture as seen at http://eniac.cyi.ac.cy/display/UserDoc/HPC+Baseline+Configuration
• ANSWER: discussed; interesting for Flamish supercomputing activities
• BENELUX collab?
• ANSWER: discussed; TBD

SYSADMIN-oriented features:

• Define the namespace "by a standard"; eg. like [https://github.com/fgeorgatos/HPC-RFC/blob/master/0001/0001.md]
• Give freedom to define the namespace format (eg. original Package names OR the lower case version)
• allow to export namespace in both UpperCase and LowerCase conventions # a post-install hook? TBD
• Support more configurable formats of version strings (check below, how other HPC sites do it)
• nested levels (ie. split version string in sub sections)
• provide hooks for gnu/gcc/intel/pgi etc strings
• IDEA: For now, supply a "custom_namespace_function" variable, to define eg. modulename.lower();
• Provide some smart way to handle osdependencies like "tcsh" which may have no meaning on Debian # a particularly important case are the boost-devel vs boost-dev differences (rpm vs deb conventions)
• and for now allow escaping with --ignore-os-dependencies

Building GROMACS-GPU

# GROMACS-GPU uses a cmake build-generator and makefiles on unix.
# To avoid confusion build in separate folder

export OPENMM_ROOT_DIR=path_to_custom_openmm_installation
mkdir build_gromacs_gpu
cd build_gromacs_gpu
cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
make mdrun
make install-mdrun

modules namespace from various HPC sites

  ---------------------------------------------------------- --------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Example name                                               WHO                   Reference URL & Comments
  ---------------------------------------------------------- --------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  netcdf/3.6.2                                               TACC                  [[http://services.tacc.utexas.edu/index.php/using-modules][]] \\ Lmod: automatic reloading of an entire module hierarchy when a single module anywhere in the hierarchy is changed.
  hpc/netcdf-3.6.3_intel-11.0.083                           HARVARD               [[http://rc.fas.harvard.edu/faq/modulelist][]] \\ very long and complete list
  misc-libs/netcdf/3.6.3_intel                              CLUMEQ                [[https://www.clumeq.ca/wiki/index.php/ModulesDisponiblesSurColosse][]] \\ nested layout
  netcdf/intel/64/4.0                                        SARA                  [[http://www.sara.nl/systems/shared/modules/][]] \\ [[http://www.sara.nl/systems/lisa/news/modules-ng][]] \\  [[http://www.sara.nl/systems/lisa/news/amd64-phase1][]] \\  [[http://www.sara.nl/systems/lisa/news/Modules-deprecated-2011-04-04][]]
[[https://www.clumeq.ca/wiki/index.php/ModulesDisponiblesSurColosse][]] \\ nested layout
  netcdf/gnu/64/4.0                                          SARA                  [[http://www.sara.nl/systems/shared/modules/][]] \\ [[http://www.sara.nl/systems/lisa/news/modules-ng][]] \\  [[http://www.sara.nl/systems/lisa/news/amd64-phase1][]] \\  [[http://www.sara.nl/systems/lisa/news/Modules-deprecated-2011-04-04][]]
  netcdf/4.0.1                                               UIBK                  [[http://www.uibk.ac.at/zid/systeme/hpc-systeme/common/tutorials/modules-howto.html][]] \\ PREFERRED_MC
  netcdf/X.Y.Z/gnu-4.1.2                                     UIBK, too             [[http://www.uibk.ac.at/th-physik/howto/hpc/modules.html][]]
  netcdf/X.Y.Z-gnu                                           CSC                   [[http://www.csc.fi/english/pages/hippu_guide/using_hippu/modules/index_html][]]
  ofed/qlogic/gcc/64/1.2.7                                   UCL                   [[http://www.ucl.ac.uk/isd/common/research-computing/services/legion-upgrade/userguide/userenvironment][]]
  netcdf/3.6.2                                               MIT                   [[http://www.darwinproject.mit.edu/wiki/index.php/Compute_cluster_hardware/software_overview][]]
  netcdf/X.Y.Z                                               MIT, too              [[http://coyote.mit.edu/mediawiki/index.php/Modules][]]
  netcdf/3.6.2                                               CAM                   [[http://www.hpc.cam.ac.uk/user/software.html][]]
  netcdf/4.0.1_nc3                                          UTORONTO              [[https://support.scinet.utoronto.ca/wiki/index.php/Software_and_Libraries][]]
  blas/?                                                     VLSCI@AU              [[http://www.vlsci.org.au/documentation/software-applications][]]
  fluent/?                                                   EPFL                  [[http://pleiades.epfl.ch/index.php?option=com_content&task=view&id=15&Itemid=30][]]
  netcdf/4.0.0.3.1jg64                                       CSCS                  [[http://user.cscs.ch/software_and_programming_environment/compilers_and_programming/rosa_cray_xt5/modules_framework/index.html][]]
  netcdf/4.1.1/pgi/10.3/64                                   TAMU                  [[http://brazos.tamu.edu/software/modules.html][]]
  sles10.1_gnu4.1.2_shared                                 UTENNESSEE            [[http://www.nics.tennessee.edu/computing-resources/kraken/software][]]
  netcdf/4.0-pgi, /4.0-gcc , /4.1.1                          UMICH                 [[http://cac.engin.umich.edu/resources/software/][]]
  netcdf-4.0.1                                               ARSC                  [[http://www.arsc.edu/support/news/systemnews/][]]
  NetCDF/4.1.3-gnu                                           Griffith University   [[http://confluence.rcs.griffith.edu.au:8080/display/GHPC/netcdf][]]
  HMMER                                                      iCER/HPCC             [[https://wiki.hpcc.msu.edu/display/Bioinfo/Module+Files][]]
  HMMER                                                      UPPNEX                [[https://www.uppnex.uu.se/installed-software][]]
  uberftp-client-2.6                                         NCSA                  [[http://www.ncsa.illinois.edu/UserInfo/Resources/Hardware/CommonDoc/module.html][]]
  netcdf                                                     LRZ                   [[http://www.lrz.de/services/software/utilities/modules/deisa_details.html][]] \\ Ref. on DEISA setup: [[http://www.lrz.de/services/software/utilities/modules/deisa_details.html][]]
  globus                                                     PRACE sites           [[http://www.prace-ri.eu/Interactive-Access-to-HPC][]]
  globus/5.0.4 & GLOBUS-5.0                                  TACC                  [[http://www.underworldproject.org/BuildRecipes/ranger.rst][]] See Teragrid below
  netcdf/4.1.2-gnu, /4.1.2-intel, /3.6.3-gnu, /3.6.3-intel   cyi/euclid@ls2        [[http://eniac.cyi.ac.cy/display/UserDoc/HPC+Baseline+Configuration]]
  netcdf/3.6.3-gcc, netcdf-4/4.0.1-gcclf                     cyi/planck   [[http://eniac.cyi.ac.cy/display/UserDoc/HPC+Baseline+Configuration]]         
  netcdf/3.6.2-intel, /3.6.2-gnu                             ba@ls2           [[http://eniac.cyi.ac.cy/display/UserDoc/HPC+Baseline+Configuration]]     
  netcdf/4.1.3-gcc(default)                                  cytera@ls2       [[http://eniac.cyi.ac.cy/display/UserDoc/HPC+Baseline+Configuration]]     
  netcdf/1.4.3-mpich2-intel-64bit                                  FZK       [[http://www.google.com]]     
  ---------------------------------------------------------- --------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

CUDF Examples

package: m4
version: 3
depends: libc6 >= 8

package: openssl
version: 11
depends: libc6 >= 18, libssl0.9.8 >= 8, zlib1g >= 1
conflicts: ssleay < 1

package: gcc
version: 4.4.4
provides: compiler
conflicts: compiler

package: icc
version: 10.2
provides: compiler
conflicts: compiler

Build frameworks naming conventions

macports

|macports|what the function is about| |fetch|Fetch the ${distfiles} from ${master_sites} and place it in ${prefix}/var/macports/distfiles/${name}| |checksum|Compare ${checksums} specified in a Portfile to the checksums of the fetched ${distfiles}.| |extract|Unzip and untar the ${distfiles} into the path ${prefix}/var/macports/build/..../work| |patch|Apply optional patch files specified in ${patchfiles} to modify a port's source code file(s).| |configure|Execute the command configure in ${worksrcpath}.| |build|Execute ${build.cmd} in ${worksrcpath}.| |test|Execute commands to run test suites bundled with a port.| |destroot|Execute the command make install DESTDIR=${destroot}in ${worksrcpath}.|

Ref. [http://guide.macports.org/#reference.phases] -> 5.3. Port Phases

portage

|pkg_setup |Use this function to perform any miscellaneous prerequisite tasks. This might include checking for an existing configuration file.| |pkg_nofetch |Inform the user about required actions if for some reason (such as licensing issues) the sources may not be downloaded by Portage automatically. Use this in conjunction with RESTRICT="fetch". You only should display messages in this function, never call die.| |src_unpack| |Use this function to unpack your sources, apply patches, and run auxiliary programs such as the autotools. By default, this function unpacks the packages listed in A. The initial working directory is defined by WORKDIR.| |src_compile |Use this function to configure and build the package. The initial working directory is S.| |src_install |Use this function to install the package to an image in D. If your package uses automake, you can do this simply with emake DESTDIR="${D}" install. Make sure your package installs all its files using D as the root! The initial working directory is S.| |src_test |Executed only when FEATURES="test" is set and RESTRICT="test" is unset, the default of this function executes an available testing function from any Makefiles in the ${S} directory, running either "make test" or "make check" depending on what is provided. It can be overriden to create a custom test setup.| |pkg_preinst |The commands in this function are run just prior to merging a package image into the file system.| |pkg_postinst |The commands in this function are run just following merging a package image into the file system.| |pkg_prerm |The commands in this function are run just prior to unmerging a package image from the file system.| |pkg_postrm |The commands in this function are run just following unmerging a package image from the file system.| |pkg_config |You use this function to set up an initial configuration for the package after it's installed. All paths in this function should be prefixed with ROOT which points to user-specified install root which may not happen to be /. This function is only executed if and when the user runs: emerge --config =${PF}.|

Ref: [http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1#doc_chap2] -> Functions