Running RangeShiftR in parallel

[pollyhipkiss]

Hi!

In the RangeShiftR paper it say RangeShiftR "has access to R’s infrastructure for parallel and cluster computing".

I am looking to run my simulations using multiple cores on a cluster, but I haven't been able to find anything in the package documentation or other publications which say how I might be able to do this?

Will RangeShiftR automatically recognise (& use) multiple cores if a job has more cores allocated, or does it need to be explicitly specified in the R script somewhere, as well as the jobscript? I've used a foreach loop to run things in parallel previously, but I am not sure how I'd apply this to the RangeShiftR simulations.

Any help would be greatly appreciated!
Thank you!

[JetteReeg]

Hi,
currently RangeShiftR is not automatically using multiple cores.
But if you have multiple scenarios (e.g. different landscape scenarios or varying RangeShifter parameters), you can make use of R packages which would run these scenarios in parallel (e.g. via a foreach loop of the doParallel package.

This is a brief outline of how I make use of a foreach loop if I have different landscape scenarios I want to run a RangeShifter simulation on:

# packages ---------------------------------------------------------------------
library(RangeShiftR)
library(doParallel)

# Rangeshifter simulations -----------------------------------------------------
# Landscape configurations
land_rep <- 1:3 # having 3 replicates of each landscape configuration
rough <- c(0.55, 0.65, 0.75, 0.85, 0.95) # different levels of roughness
loss <- c(0.95, 0.9, 0.85, 0.8, 0.75, 0.7, 0.65, 0.6) # different levels of habitat amount

sims <- expand.grid(land_rep = land_rep, rough = rough, loss = loss) # overall dataframe of my different scenarios

# setting up a cluster iwht 4 threads
nb_cpus <- 4
cl <- makeCluster(nb_cpus)
registerDoParallel(cl)

# for each landscape replicate
foreach(sim_nb = 1:nrow(sims), .packages = c("RangeShiftR", "data.table")) %dopar% {
    
    rep_nr <- sims[sim_nb,]$land_rep
    rough <- sims[sim_nb,]$rough
    loss <- sims[sim_nb,]$loss
    
    # landscape ---------------------------------------------------------------
    LandscapeFile = paste0("land_rough", rough, "_loss", loss, "_", rep_nr, ".asc") # constructing the file name of the scenario
    
    Resolution <- 400 # orig. 20; adapted so around 20 individuals survive per cell

    land <- ImportedLandscape(LandscapeFile = LandscapeFile,
                              HabPercent = F,
                              Nhabitats = 2,
                              Resolution = Resolution,
                              K_or_DensDep = c(0,28.986))
    # .....
}
# stopping the cluster as good practice
stopCluster(cl)

Best,
Jette

[pollyhipkiss]

Hi Jette,

Thanks for your speedy response! That all makes sense, and thank you for your clear code example, it is very helpful.

I had been hoping to speed up the run-time of my simulations on the cluster I am using as they currently hit the max time before completion, but it looks like this this isn't currently possible with the rangeshifter platform.

Thank you again for your help!
Polly

[JetteReeg]

Yeah, currently it is not possible to increase runtime for the RangeShiftR package. However, with the next version of RangeShifter there will be the possibility to speed up the runtime by parallel processing at least for the batch interface. I hope I can also integrate it for the R package, but I still need to have a closer look...

[pollyhipkiss]

That's great to know - I'll keep an eye out for the feature in future releases, thank you for your help!