| Author: | F.Mauger <mauger@lpccaen.in2p3.fr> |
|---|---|
| Date: | 2022-12-06 |
BxInstallers (https://github.com/BxCppDev/BxInstallers) provides some build/install/setup shell scripts for an automated installation of some software libraries of interest used in the context of the development and use of the Bayeux library.
It is mainly useful for testing and tuning the installation procedure of some third party software I need, mostly on Ubuntu Linux (18.04, 20.04, 22.04), but it may work on CentOS (not really tested).
BxInstallers does not aim to compete with industrial strength package management systems like Spack or Homebrew/Linuxbrew, nor apt or yum. I just consider this tool as well adapted to my needs in the context of my projects, my computer and the data/computing centers I use in my daily work.
The system is designed to allow the parallel installation and usage of different versions of a given software package, using different build/installation paths and distinct setup scripts. It is up to the user to properly create some high-level setup scripts that will activates the correct software stack with the proper dependency chain.
Use it at your own risk !
François Mauger, Université de Caen Normandie
Contents
Clone the BxInstallers' Git repository on your system :
$ cd ${HOME} # or wherever you want to copy BxInstallers' scripts
$ git clone https://github.com/BxCppDev/BxInstallers
$ cd BxInstallers/
$ less README.rstAs a helper package to build and install third party software, BxInstallers comes with some useful scripts that can be reused in some other context. It is thus possible to install theses tools and make them available from other tools.
From the BxInstallers' base source directory:
$ mkdir _build.d
$ cd _build.d
$ cmake -DCMAKE_INSTALL_PREFIX="/opt/sw/bxinstallers" ..
$ make install
$ cd ..Put the following line in your Bash startup script:
export PATH="/opt/sw/bxinstallers/bin:${PATH}"Assuming a package Foo is published as a source archive (typically
as foo-@version@.tar.gz) and build via CMake, it is possible to
build a skeleton of a build/install script for this package:
$ bxinstallers-mkskel --github-owner "SuperNEMO-DBD" --archive-pattern "Foo-v@PKGVERSION@.tar.gz" "Foo" "/tmp/FooInstaller"This creates the /tmp/FooInstaller directory with a skeleton
foo_installer Bash script and a few additional resource
files. Then the foo_installer should be adapted to be able to
download, configure, build and install the Foo package in some standard
location on your system (based on the env. var. BX_INSTALL_BASE_DIR).
Typically, items to be modified are:
- the original repository where the source archive can be downloaded.
Example:
https://ftp.gnu.org/gnu/gsl/ - for a GitHub repository, it is possible to specify the owner or group identifier in such a way the URL of the repo can be automatocally built: "https://github.com/@ownerName@/@packageName@/archive/"
- the pattern of the remote archive filename. Example :
"foo-@PKGVERSION@.tar.gz"where the"@PKGVERSION@"will be automatically replaced by the requested version string of the package
Once adapted to the build constraints of the package, the foo_installer script
can be used to configure, build and finally install the package.
$ cd /tmp/FooInstaller
$ ./snrs_installer --help
$ ./snrs_installer --rebuild --package-version 1.1 --with-ninjaA setup Bash script foo@.1.1.bash is also generated.
It must be saved in some specific location for
configuration files and sourced from a startup script.
BxQtInstaller: Installer for the Qt library (new and experimental)BxBoostInstaller: Installer for the Boost libraryBxCampInstaller: Installer for the CAMP libraryBxClhepInstaller: Installer for the CLHEP C++ libraryBxGeant4DatasetsInstaller: Installer for the GEANT4 datasetsBxGeant4Installer: Installer for the GEANT4 Simulation ToolkitBxRootInstaller: Installer for the ROOT libraryBxDecay0Installer: Installer for the BxDecay0 libraryBxBayeuxInstaller: Installer for the Bayeux libraryBxProtobuftoolsInstaller: Installer for the BxProtocoltools libraryBxJsontoolsInstaller: Installer for the BxJsontools libraryBxRabbitmqInstaller: Installer for the BxRabbitmq library
BxInstallers uses a set of conventional directories used to build and install libraries:
- The cache directory is the directory where source tarballs are
downloaded. By default, it is
${HOME}/bxsoftware/_cache.dfor a standard user (without root proviledges) and/var/bxsoftware/cache.dfor the root user. - The work directory is used to host build directories for the
installation of the software. By default, it is
${HOME}/bxsoftware/_work.dfor a standard user (without root proviledges) and/var/bxsoftware/work.dfor the root user. - The installation base directory is the installation base directory
for the installed software. By default, it is
${HOME}/bxsoftware/installfor a standard user (without root proviledges) and/opt/bxsoftware/installfor the root user. - The configuration base directory is the directory for the
configuration scripts associated to installed software (modules).
By default, it is
${HOME}/bxsoftware/configfor a standard user (without root proviledges) and/opt/bxsoftware/configfor the root user. - The (experimental) package directory is the directory where to
store generated Debian packages. By default, it is
${HOME}/bxsoftware/_package.dfor a standard user (without root proviledges) and/var/bxsoftware/package.dfor the root user.
These directories may need a huge amount of free storage space (several GB) because some third party software packages are really big (ROOT, Geant4 and associated datasets...) and also the building process uses quite a large amount of storage. Please make sure you can find such a large space on your disks. It is probably a bad idea to install the software stack in your home directory.
BxInstallers supports a few environment variables to locate working directories used to build and install libraries:
BX_CACHE_DIR: overrides the default cache directory (for downloading).BX_WORK_DIR: overrides the default work directory (for building).BX_INSTALL_BASE_DIR: overrides the default installation base directory (for final installation).BX_CONFIG_DIR: overrides the default configuration directory (for configuration scripts).BX_PACKAGE_DIR(optional, for experts only, experimental) : overrides the package directory to store generated Debian packages.
Each supported software package is associated to specific installation script which should, in principle, download the source archive (or the git repo) from the proper location on the web, unpack it, configure it, compile it from the working directory then install it. A setup shell script is also generated for further activation by the user.
If some extra system packages are required, some apt commands are
run on Ubuntu before the build stage (need some sudo access).
Example:
Prepare your environment : create specific working and installation directories and environment variables
$ mkdir -p /opt/sw/BxSoftware $ export BX_CACHE_DIR="/opt/sw/BxSoftware/BxCache" $ export BX_WORK_DIR="/opt/sw/BxSoftware/BxWork" $ export BX_INSTALL_BASE_DIR="/opt/sw/BxSoftware/BxInstall" $ export BX_CONFIG_DIR="/opt/sw/BxSoftware/BxConfig" $ mkdir -p ${BX_CACHE_DIR} $ mkdir -p ${BX_WORK_DIR} $ mkdir -p ${BX_INSTALL_BASE_DIR} $ mkdir -p ${BX_CONFIG_DIR}
We recommend to put the following lines in your
~/.bashrcstartup script:$ export BX_CACHE_DIR="/opt/sw/BxSoftware/BxCache" $ export BX_WORK_DIR="/opt/sw/BxSoftware/BxWork" $ export BX_INSTALL_BASE_DIR="/opt/sw/BxSoftware/BxInstall" $ export BX_CONFIG_DIR="/opt/sw/BxSoftware/BxConfig"
That will make available the working paths used to build and run the Bayeux software stack.
Install some software packages:
$ cd BxBoostInstaller/ $ ./boost_installer --package-version 1.71.0 $ cd ../BxCampInstaller/ $ ./camp_installer $ cd ../BxClhepInstaller/ $ ./clhep_installer --package-version "2.1.3.1" $ cd ../BxRootInstaller/ $ ./root_installer --package-version 6.26.06 --with-python --with-xrootd --without-buildin-xrootd
This section illustrates a typical configure-build-installation-setup procedure for the Bayeux library and all its dependencies on a Ubuntu 22.04 LTS system. We assume that Bash is the default shell.
We also assume that you are in the sudoers group or equivalent to allow some system package installation when needed.
We recommend not to use your home directory for such a big
installation but rather to create and use some other directory (here
/opt/sw) that does not need to be backuped.
Make a hosting directory for building and installing the software:
$ sudo mkdir /opt/sw
$ sudo chmod 1777 /opt/swCreate specific working and installation directories and environment variables:
$ sudo mkdir -p /opt/sw
$ sudo chmod 1777 /opt/sw
$ mkdir -p /opt/sw/BxSoftware
$ export BX_CACHE_DIR="/opt/sw/BxSoftware/BxCache"
$ export BX_WORK_DIR="/opt/sw/BxSoftware/BxWork"
$ export BX_INSTALL_BASE_DIR="/opt/sw/BxSoftware/BxInstall"
$ export BX_CONFIG_DIR="/opt/sw/BxSoftware/BxConfig"
$ mkdir -p ${BX_CACHE_DIR}
$ mkdir -p ${BX_WORK_DIR}
$ mkdir -p ${BX_INSTALL_BASE_DIR}
$ mkdir -p ${BX_CONFIG_DIR}
$ tree /opt/sw/BxSoftware$ cd ./BxClhepInstaller/
$ ./clhep_installer --package-version "2.1.4.2"
$ ls -l ${BX_CONFIG_DIR}/modules/clhep@2.1.4.2.bashUbuntu 22.04:
$ cd ../BxRootInstaller/
$ ./root_installer --package-version "6.26.06" --cxx14 --with-python --with-xrootd --without-buildin-xrootd --nprocs 4
$ ls -l ${BX_CONFIG_DIR}/modules/root@6.26.06.bash$ cd ../BxGeant4DatasetsInstaller
$ ./g4datasets_installer --package-version "9.6.4"
$ ls -l ${BX_CONFIG_DIR}/modules/g4datasets@9.6.4.bashSetup dependencies:
$ source ${BX_CONFIG_DIR}/modules/clhep@2.1.4.2.bash $ source ${BX_CONFIG_DIR}/modules/g4datasets@9.6.4.bash $ clhep_2_1_4_2_setup [info] clhep_2_1_4_2_setup: CLHEP version 2.1.4.2 is now setup ! $ which clhep-config /opt/swtest/BxSoftware/BxInstall/clhep-2.1.4.2/bin/clhep-config $ g4datasets_9_6_4_setup [info] g4datasets_9_6_4_setup: Geant4 datasets version 9.6.4 is now setup ! $ echo $G4LEDATA
Installation:
$ cd ../BxGeant4Installer /opt/swtest/BxSoftware/BxInstall/g4datasets-9.6.4/share/Geant4Datasets-9.6.4/data/G4EMLOW7.7 $ ./geant4_installer --package-version "9.6.4" $ ls -l ${BX_CONFIG_DIR}/modules/geant4@9.6.4.bash
$ cd ../BxDecay0Installer
$ ./bxdecay0_installer --package-version "1.1.0"
$ ls -l ${BX_CONFIG_DIR}/modules/bxdecay0@1.1.0.bashSetup additional dependencies, assuming CLHEP 2.1.4.2 and Geant4 datasets 9.6.4 have been setup before:
$ source ${BX_CONFIG_DIR}/modules/root@6.16.00.bash $ root_6_16_00_setup [info] root_6_16_00_setup: ROOT version 6.16.00 is now setup ! $ root-config --prefix /opt/swtest/BxSoftware/BxInstall/root-6.16.00 $ source ${BX_CONFIG_DIR}/modules/geant4@9.6.4.bash $ geant4_9_6_4_setup [info] geant4_9_6_4_setup: GEANT4 version 9.6.4 is now setup ! $ geant4-config --prefix /opt/swtest/BxSoftware/BxInstall/geant4-9.6.4/bin/.. $ source ${BX_CONFIG_DIR}/modules/bxdecay0@1.1.0.bash $ bxdecay0_1_1_0_setup [info] bxdecay0_1_1_0_setup: BxDecay0 version 1.1.0 is now setup ! $ bxdecay0-config --prefix /opt/swtest/BxSoftware/BxInstall/bxdecay0-1.1.0
Installation:
$ cd ../BxBayeuxInstaller/ $ ./bayeux_installer --package-version "3.5.2" --with-qt --with-geant4 $ ls -l ${BX_CONFIG_DIR}/modules/bayeux@3.5.0.bash
Setup:
$ source ${BX_CONFIG_DIR}/modules/bayeux@3.5.2.bash $ bayeux_3_5_2_setup [info] bayeux_3_5_0_setup: Bayeux version 3.5.2 is now setup ! $ bxquery --prefix /opt/swtest/BxSoftware/BxInstall/bayeux-3.5.2
Create a bash script :
/opt/swtest/BxSoftware/BxConfig/bxsoftware.bashexport BX_CACHE_DIR="/opt/swtest/BxSoftware/BxCache" export BX_WORK_DIR="/opt/swtest/BxSoftware/BxWork" export BX_INSTALL_BASE_DIR="/opt/swtest/BxSoftware/BxInstall" export BX_CONFIG_DIR="/opt/swtest/BxSoftware/BxConfig"Create a bash script :
/opt/swtest/BxSoftware/BxConfig/bayeux_run_setup.bash
In principle, after all the software you need has been installed and setup, you can remove the contents of the cache and work directories to save storage space on your system. I found generally useful to preserve the cache directory in order to be able to reprocess some installation off line.