BiocParallel.FutureParam: Use Futures with BiocParallel

Introduction

The future package provides a generic API for using futures in R. A future is a simple yet powerful mechanism to evaluate an R expression and retrieve its value at some point in time. Futures can be resolved in many different ways depending on which strategy is used. There are various types of synchronous and asynchronous futures to choose from in the future package. Additional futures are implemented in other packages. For instance, the future.batchtools package provides futures for any type of backend that the batchtools package supports. For an introduction to futures in R, please consult the vignettes of the future package.

The BiocParallel.FutureParam package provides FutureParam, a BiocParallelParam class, for the BiocParallel package that works with any type of future. The BiocParallel.FutureParam package is cross platform just as the future package.

Below is an example showing how to use FutureParam with multiprocess futures. A multiprocess future will be evaluated in parallel using forked processes. If process forking is not supported by the operating system, then multiple background R sessions will instead be used to resolve the futures.

library("BiocParallel.FutureParam")
register(FutureParam())
plan(multiprocess)

mu <- 1.0
sigma <- 2.0
x <- bplapply(1:3, mu = mu, sigma = sigma, function(i, mu, sigma) {
  rnorm(i, mean = mu, sd = sigma)
})

FutureParam replaces existing BiocParallelParam classes

Due to the generic nature of futures, the FutureParam class provides the same functionality as many of the existing BiocParallelParam classes, e.g. SerialParam, SnowParam, MulticoreParam, BatchJobsParam and DoParParam. In addition, it provides supports for additional backends that are not yet implemented in BiocParallel, e.g. batchtools and callr.

BiocParallel usage	BiocParallel.FutureParam alternative
library("BiocParallel") register(SerialParam())	library("BiocParallel.FutureParam") register(FutureParam()) plan(sequential)
library("BiocParallel") register(MulticoreParam())	library("BiocParallel.FutureParam") register(FutureParam()) plan(multicore)
library("BiocParallel") register(SnowParam(2, type = "SOCK"))	library("BiocParallel.FutureParam") register(FutureParam()) plan(multisession, workers = 2)
library("BiocParallel") cl <- parallel::makeCluster(2, type = "SOCK") register(as(cl, "SnowParam"))	library("BiocParallel.FutureParam") register(FutureParam()) cl <- parallel::makeCluster(2, type = "SOCK") plan(cluster, workers = cl)
library("BiocParallel") register(SnowParam(4, type = "MPI"))	library("BiocParallel.FutureParam") register(FutureParam()) cl <- parallel::makeCluster(4, type = "MPI") plan(cluster, workers = cl)
library("BiocParallel") library("BatchJobs") funs <- makeClusterFunctionsSLURM("~/slurm.tmpl") register(BatchJobsParam(cluster.functions = funs))	library("BiocParallel.FutureParam") register(FutureParam()) plan(future.BatchJobs::batchjobs_slurm, pathname = "~/slurm.tmpl")
library("BiocParallel") register(BatchtoolsParam(cluster="sge", template="~/sge.tmpl"))	library("BiocParallel.FutureParam") register(FutureParam()) plan(future.batchtools::batchtools_sge, template = "~/sge.tmpl")
N/A	library("BiocParallel.FutureParam") register(FutureParam()) plan(future.callr::callr)

Installation

R package BiocParallel.FutureParam is only available via GitHub and can be installed in R as:

remotes::install_github("HenrikBengtsson/BiocParallel.FutureParam", ref="master")

Pre-release version

To install the pre-release version that is available in Git branch develop on GitHub, use:

remotes::install_github("HenrikBengtsson/BiocParallel.FutureParam", ref="develop")

This will install the package from source.

Contributions

This Git repository uses the Git Flow branching model (the git flow extension is useful for this). The develop branch contains the latest contributions and other code that will appear in the next release, and the master branch contains the code of the latest release.

Contributing to this package is easy. Just send a pull request. When you send your PR, make sure develop is the destination branch on the BiocParallel.FutureParam repository. Your PR should pass R CMD check --as-cran, which will also be checked by Travis CI and AppVeyor CI when the PR is submitted.

Software status

Resource	GitHub	GitHub Actions	Travis CI	AppVeyor CI
Platforms:	Multiple	Multiple	Linux & macOS	Windows
R CMD check
Test coverage