Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
176 lines (129 sloc) 6.05 KB

The rchk project, https://github.com/kalibera/rchk is an important tool for detecting memory protection errors and related subtle bugs in R packages that contain compiled code and the R source code itself.

The project uses the LLVM compiler toolchain with the whole program LLVM extensions. As noted in the rchk project README, setting up the toolchain is a complicated process requiring careful attention to a very specific set of software dependencies.

The rchk project includes recipes for Docker and Vagrant systems to help automate the set up process to build either containers or virtual machines that can then be used to check R packages.

This note and the corresponding singularity.def file present an alternative simple container recipe using the Singularity container system (http://singularity.lbl.gov/). Singularity is a lightweight, serverless (that is, no daemon process), container system for GNU Linux popular in HPC settings. It's a very lightweight an minimalist containerization framework that we feel is ideal for projects like rchk.

This note outlines system requirements and installation of Singularity, building a container for rchk, and testing R packages using the container.

Installing singularity

Singularity requires a GNU Linux operating system. Most modern GNU Linux systems include Linux kernels that will work.

See http://singularity.lbl.gov/docs-installation for installation examples specific to Ubuntu and CentOS/RHEL operating systems. Alternatively, you may install Singularity directly from its source code in GitHub with (requires the git command line client, GNU make and a C compiler:

git clone https://github.com/singularityware/singularity.git
cd singularity
./autogen.sh
./configure --prefix=/usr/local
make
sudo make install

Singularity is simply a program. No daemon process/server is needed.

Building an rchk container image

The singularity.def file includes a singularity definition file for building a singularity container image based on Ubuntu 17.04 and the LLVM-4.0 and corresponding WLLVM toolchains. Singularity containers may be built as single files or, for experimentation, sandbox directories.

Note! If you're running on Red Hat or CentOS, you'll need the debootstrap program: sudo yum install debootstrap. See the Singularity documentation for more information.

Build the rchk singularity image with:

sudo singularity build rchk.img singularity.def       # makes rchk.img

See the Singularity documentation and the example below for alternative output formats (like a sandboxed directory that you can investigate easily).

The container build process concludes with a usage message, or an error if something goes wrong.

Checking a package with the rchk.img container

We've set the container up to make it easy to check R packages installed from CRAN or from a local source file. The packages are built and installed into a directory determined by the PKG_ROOT shell variable. If that variable is not set then the current working directory is used for output. Output are placed in ${PKG_ROOT}/build and ${PKG_ROOT}/lib directories, which are created if they do not exist. The container uses the http://cran.ma.imperial.ac.uk repository for network-installed packages.

Generic container invocation is:

singularity run <container image file> <package name>  [source package path]

Here is an example that checks the R package curl installed from CRAN, placing rchk output in the current directory:

singularity run rchk.img curl

Inspect the rchk output with, for instance:

cat ./lib/curl/libs/curl.so.bcheck 

## Analyzed 86 functions, traversed 864 states.

The following example checks a local source package, placing the rchk output in /tmp:

wget https://cran.r-project.org/src/contrib/irlba_2.3.1.tar.gz
PKG_ROOT=/tmp singularity run rchk.img   irlba   $(pwd)/irlba_2.3.1.tar.gz

cat /tmp/lib/irlba/libs/irlba.so.bcheck 

# Analyzed 71 functions, traversed 489 states.

Sandboxed images

Some R packages may require additional operating system library dependencies not included in the container recipe singularity.def above. You can add additional Ubuntu-packaged libraries to the container recipe before building it. Alternatively, you can build a container sandbox directory instead of a single container image file, and dynamically add required libraries to the sandbox directory as needed.

The following example builds a sandboxed container directory. We then try to install the Rmpfr library for multi-precision arithmetic, which fails due to unsatisifed library dependencies in the container image. The example proceeds to manually install the required dependencies and then chkecks the package.

Step 1. Build the sandboxed container

sudo singularity build --sandbox rchk singularity.def     # makes rchk directory

2. Try to check the Rmpfr package

PKG_ROOT=/tmp singularity run rchk  Rmpfr                 # try to check package

## ...
## ERROR: dependency ‘gmp’ is not available for package ‘Rmpfr’
## ...

3. Modify the sandboxed container to include required libraries

sudo singularity exec -w rchk /bin/bash

apt-get install -y libgmp-dev libmpfr-dev
## ... output of apt installation process

exit

Note that at this point, we can do anything to the container image that we desire, including for instance installing library dependencies manually or otherwise.

The sandboxed singularity container is the most flexible approach for checking packages with dependencies.

4. Try to check the Rmpfr package again

PKG_ROOT=/tmp singularity run rchk  Rmpfr

## ... output of R package build process, which should finish without error

# Let's check the output:
cat /tmp/lib/Rmpfr/libs/Rmpfr.so.bcheck 

## Function MPFR_as_R
##   [UP] unprotected variable exp_R while calling allocating ...
##   [UP] unprotected variable prec_R while calling allocating ...
## ... (truncated output)

Now that the library dependencies are satisfied, we're able to check the package with rchk. In this case, at least at the time of this writing, we see a few potential issues.