BatchJobs: Batch computing with R
R Shell
Latest commit f226f68 Jan 2, 2017 @mllg mllg added hint to batchtools

README.md

BatchJobs

CRAN_Status_Badge CRAN Downloads Build Status Build status Coverage Status

NOTE: Development continues in the new package batchtools

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF and Sun Grid Engine. Multicore and SSH systems are also supported.

If you use the package, please cite it and star it here

JSS link: http://www.jstatsoft.org/v64/i11

To cite BatchJobs or BatchExperiments in publications use:

  Bernd Bischl, Michel Lang, Olaf Mersmann, Jörg Rahnenführer, Claus Weihs (2015). BatchJobs and BatchExperiments: Abstraction Mechanisms for Using R in Batch Environments. Journal
  of Statistical Software, 64(11), 1-25. URL http://www.jstatsoft.org/v64/i11/.

A BibTeX entry for LaTeX users is

  @Article{,
    title = {{BatchJobs} and {BatchExperiments}: Abstraction Mechanisms for Using {R} in Batch Environments},
    author = {Bernd Bischl and Michel Lang and Olaf Mersmann and J{\"o}rg Rahnenf{\"u}hrer and Claus Weihs},
    journal = {Journal of Statistical Software},
    year = {2015},
    volume = {64},
    number = {11},
    pages = {1--25},
    url = {http://www.jstatsoft.org/v64/i11/},
  }

Core features

  • Create, submit and control R jobs on batch systems
  • Provides the functional programming tools Map, Reduce and Filter to operate on the cluster
  • Supported systems include Torque/PBS, SGE, SLURM, OpenLava and LSF
  • Support for makeshift SSH clusters and local (multicore) execution
  • Convenient collection and aggregation of results
  • Further Map and Reduce results from previous jobs as batch jobs
  • Optional mail sending using sendmailR after job completion
  • Query status of jobs and display log files inside R
  • Possibility to write your own simple cluster interface if your architecture is not supported
  • BatchExperiments extends this package with functionality required for comprehensive computer experiments and simulation studies.

Configuration

A fresh installation defaults to a local execution mode. Proceed to Configuration to set up cluster execution. The package also allows a few R options to be set, see here.

Documentation

Probably the best way to get started in 5 minutes is to read our paper or Henrik Bengtsson's intro slides. If you have more time, read our technical report. For more detailed information on the functions consult the R documentation We also provide a FAQ in our wiki. And here is a nice intro article for BatchJobs with Torque on R Bloggers.

We also have a mailing list.