Skip to content
A R package that allows users to submit parallel workloads in Azure
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Created an issue template (#207) Jan 25, 2018
.vsts Update pipeline.yml for Azure Pipelines Apr 16, 2019
R Feature: Batch Insights support (#336) Feb 5, 2019
docker-image Feature/container (#153) Nov 3, 2017
docs Feature: Batch Insights support (#336) Feb 5, 2019
inst/startup Fix: Revert Cluster Setup File (#342) Feb 7, 2019
man Documentation rewrite (#273) May 15, 2018
samples Fix: Test Coverage on Azure Pipeline CI (#348) Feb 14, 2019
vignettes Created documentation Feb 16, 2017
.Rbuildignore Added Travis CI (#23) Jun 14, 2017
.gitattributes Merge master to stable for version 0.4.3 (#132) Sep 29, 2017
.gitignore Merge master to stable for version 0.4.3 (#132) Sep 29, 2017
.lintr Enable AAD and VNet Support (#252) Apr 28, 2018
.travis.yml Fix/add task perf (#195) Jan 10, 2018 Release: v0.7.2 (#317) Oct 8, 2018 Updates to documentation Feb 17, 2017
DESCRIPTION Release: v0.7.2 (#317) Oct 8, 2018
LICENSE Initial commit Feb 15, 2017
NAMESPACE Feature/asynccluster (#197) Jan 18, 2018 Fix: Documentation typos and fixes (#307) Sep 12, 2018 add sharedKey to credentials related code and doc (#266) May 3, 2018 Feature/getstarted (#255) Apr 27, 2018

Build Status



The doAzureParallel package is a parallel backend for the widely popular foreach package. With doAzureParallel, each iteration of the foreach loop runs in parallel on an Azure Virtual Machine (VM), allowing users to scale up their R jobs to tens or hundreds of machines.

doAzureParallel is built to support the foreach parallel computing package. The foreach package supports parallel execution - it can execute multiple processes across some parallel backend. With just a few lines of code, the doAzureParallel package helps create a cluster in Azure, register it as a parallel backend, and seamlessly connects to the foreach package.

NOTE: The terms pool and cluster are used interchangably throughout this document.

Notable Features

  • Ability to use low-priority VMs for an 80% discount (link)
  • Users can bring their own Docker Image
  • AAD and VNets Support
  • Built in support for Azure Blob Storage


  • R (>= 3.3.1)
  • httr (>= 1.2.1)
  • rjson (>= 0.2.15)
  • RCurl (>= 1.95-4.8)
  • digest (>= 0.6.9)
  • foreach (>= 1.4.3)
  • iterators (>= 1.0.8)
  • bitops (>= 1.0.5)


  1. Install doAzureParallel directly from Github.
# install the package devtools

# install the doAzureParallel and rAzureBatch package
  1. Create an doAzureParallel's credentials file
  1. Login or register for an Azure Account, navigate to Azure Cloud Shell
wget -q &&
chmod 755 &&
  1. Follow the on screen prompts to create the necessary Azure resources and copy the output into your credentials file. For more information, see Getting Started Scripts.

To Learn More:

Getting Started

Import the package


Set up your parallel backend with Azure. This is your set of Azure VMs.

# 1. Generate your credential and cluster configuration files.  

# 2. Fill out your credential config and cluster config files.
# Enter your Azure Batch Account & Azure Storage keys/account-info into your credential config ("credentials.json") and configure your cluster in your cluster config ("cluster.json")

# 3. Set your credentials - you need to give the R session your credentials to interact with Azure

# 4. Register the pool. This will create a new pool if your pool hasn't already been provisioned.
cluster <- makeCluster("cluster.json")

# 5. Register the pool as your parallel backend

# 6. Check that your parallel backend has been registered

Run your parallel foreach loop with the %dopar% keyword. The foreach function will return the results of your parallel code.

number_of_iterations <- 10
results <- foreach(i = 1:number_of_iterations) %dopar% {
  # This code is executed, in parallel, across your cluster.

After you finish running your R code in Azure, you may want to shut down your cluster of VMs to make sure that you are not being charged anymore.

# shut down your pool

Table of Contents

This section will provide information about how Azure works, how best to take advantage of Azure, and best practices when using the doAzureParallel package.

  1. Azure Introduction (link)

    Using Azure Batch

  2. Getting Started (link)

    Using the Getting Started to create credentials

    i. Generate Credentials Script (link)

    • Pre-built bash script for getting Azure credentials without Azure Portal

    ii. National Cloud Support (link)

    • How to run workload in Azure national clouds
  3. Customize Cluster (link)

    Setting up your cluster to user's specific needs

    i. Virtual Machine Sizes (link)

    • How do you choose the best VM type/size for your workload?

    ii. Autoscale (link)

    • Automatically scale up/down your cluster to save time and/or money.

    iii. Building Containers (link)

    • Creating your own Docker containers for reproducibility
  4. Managing Cluster (link)

    Managing your cluster's lifespan

  5. Customize Job

    Setting up your job to user's specific needs

    i. Asynchronous Jobs (link)

    • Best practices for managing long running jobs

    ii. Foreach Azure Options (link)

    • Use Azure package-defined foreach options to improve performance and user experience

    iii. Error Handling (link)

    • How Azure handles errors in your Foreach loop?
  6. Package Management (link)

    Best practices for managing your R packages in code. This includes installation at the cluster or job level as well as how to use different package providers.

  7. Storage Management

    i. Distributing your Data (link)

    • Best practices and limitations for working with distributed data.

    ii. Persistent Storage (link)

    • Taking advantage of persistent storage for long-running jobs

    iii. Accessing Azure Storage through R (link)

    • Manage your Azure Storage files via R
  8. Performance Tuning (link)

    Best practices on optimizing your Foreach loop

  9. Debugging and Troubleshooting (link)

    Best practices on diagnosing common issues

  10. Azure Limitations (link)

    Learn about the limitations around the size of your cluster and the number of foreach jobs you can run in Azure.

Additional Documentation

Read our FAQ for known issues and common questions.

Next Steps

For more information, please visit our documentation.

You can’t perform that action at this time.