Run things in parallel from R

This page will help you set up an R script to run it in parallel locally.

If this is your first time running something in parallel on your computer, here is a little vocabulary that may help:

• A core is a general term for a processor on your computer (our CIC computers have 8). If you want to see them at work, open a new terminal and type htop. See them working at the top? You can exit with F10.
• A cluster is a collection of objecting capable of hosting cores, either a network or just the collection of cores on your personal computer. In our case, we are referring to the latter.
• A process is a single running version of R (or more generally any program). Each core runs a single process. Glossary from: https://dept.stat.lsa.umich.edu/~jerrick/courses/stat701/notes/parallel.html

The first step is to load all of the libraries you want to use in your script. You need at least "parallel". This is a very basic example, so we won't actually be using additional libraries.

library(parallel)

Set the number of cores you want to use for your cluster (available cores - 1 so you can still do other stuff)

no_cores <- detectCores() - 1```

Next, initialize your cluster
```R
cl <- makeCluster(no_cores)

If you are using additional packages, you will need to pass them through to the processes. For this example, we don't need this step, but if you do, use the following syntax. For an example of this, see our wiki page, Power Analysis (under construction)

clusterEvalQ(cl, {
  library(lme4)
  library(splines)
  library(lmerTest)
  library(ggeffects)
})

You will also need to pass the variables you want to access from your environment to the processes:

clusterExport(cl, {
  "data"
  "ANY ADDITIONAL VARIABLES"
})

There are several ways of running things in parallel. Here, we will use the parallel version of lapply to get each element in each column squared and then cubed.

answer <- parLapply(cl,data, function(x) c((x)^2, x^3))

For more advanced functions and uses, check out the Power analysis page (parallel to come): https://github.com/CobraLab/documentation/wiki/Power-Analysis