BatchJobs: Batch computing with R
R Shell Rebol
Clone or download
Latest commit a3ad368 Nov 30, 2017

README.md

BatchJobs

CRAN_Status_Badge CRAN Downloads Build Status Build status Coverage Status

NOTE: Development continues in the new package batchtools

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF and Sun Grid Engine. Multicore and SSH systems are also supported.

If you use the package, please cite it and star it here

JSS link: http://www.jstatsoft.org/v64/i11

To cite BatchJobs or BatchExperiments in publications use:

Bernd Bischl, Michel Lang, Olaf Mersmann, Jörg Rahnenführer, Claus Weihs (2015). BatchJobs and BatchExperiments: Abstraction Mechanisms for Using R in Batch Environments. Journal
of Statistical Software, 64(11), 1-25. URL http://www.jstatsoft.org/v64/i11/.

A BibTeX entry for LaTeX users is

@Article{,
  title = {{BatchJobs} and {BatchExperiments}: Abstraction Mechanisms for Using {R} in Batch Environments},
  author = {Bernd Bischl and Michel Lang and Olaf Mersmann and J{\"o}rg Rahnenf{\"u}hrer and Claus Weihs},
  journal = {Journal of Statistical Software},
  year = {2015},
  volume = {64},
  number = {11},
  pages = {1--25},
  url = {http://www.jstatsoft.org/v64/i11/},
}

Core features

  • Create, submit and control R jobs on batch systems
  • Provides the functional programming tools Map, Reduce and Filter to operate on the cluster
  • Supported systems include Torque/PBS, SGE, SLURM, OpenLava and LSF
  • Support for makeshift SSH clusters and local (multicore) execution
  • Convenient collection and aggregation of results
  • Further Map and Reduce results from previous jobs as batch jobs
  • Optional mail sending using sendmailR after job completion
  • Query status of jobs and display log files inside R
  • Possibility to write your own simple cluster interface if your architecture is not supported
  • BatchExperiments extends this package with functionality required for comprehensive computer experiments and simulation studies.

Configuration

A fresh installation defaults to a local execution mode. Proceed to Configuration to set up cluster execution. The package also allows a few R options to be set, see here.

Documentation

Probably the best way to get started in 5 minutes is to read our paper or Henrik Bengtsson's intro slides. If you have more time, read our technical report. For more detailed information on the functions consult the R documentation We also provide a FAQ in our wiki. And here is a nice intro article for BatchJobs with Torque on R Bloggers.

We also have a mailing list.