parallel: A sequential cluster

[HenrikBengtsson]

Background

In R (>= 4.4.0), it will be possible to implement custom parallel clusters. This is because S3 generic functions sendData() and recvData() are now exported.

Wish

For troubleshooting and debugging purposes, it could be useful to have a cluster that runs sequentially in the current R session. I've implemented a prototype for this in https://github.com/HenrikBengtsson/Wishlist-for-R/blob/master/R/makeClusterSequential.R, which works like:

library(parallel)

cl <- makeClusterSequential()
print(cl)
#> A 'sequential_cluster' cluster with 1 node
#>  A 'cluster' cluster with 1 node

y <- parLapply(cl, X = 1:3, fun = sqrt)
str(y)
#> List of 3
#>  $ : num 1
#>  $ : num 1.41
#>  $ : num 1.73

Sys.getpid()
#> [1] 902511
y <- clusterEvalQ(cl, Sys.getpid())
str(y)
#> List of 1
#>  $ : int 902511

Why

This will help troubleshooting code running in parallel. For instance, we'll be able to use debug() interactively. Another example is troubleshooting code that fails when running in parallel, e.g. it might that it uses objects that must not be exported to another R process, cf. https://future.futureverse.org/articles/future-4-non-exportable-objects.html. Using makeClusterSequential() can help rule out such scenarios.

[HenrikBengtsson]

FWIW, I've added makeClusterSequential() to parallelly (>= 1.37.1-9011).