How to implement resolved() for an MPI-based cluster?

[HenrikBengtsson]

After some fixes, the develop branch now supports MPI clusters (I haven't really had a good place to check this before). However, it's not very optimal, because currently there's no efficient resolved() implementation, because I don't know yet how to do a non-blocking MPI check to see whether a particular worker is ready or not. In the meanwhile, resolved() calls value(..., signal = FALSE) and then return TRUE. This means that resolved() is blocking for MPI-based cluster futures.

Example:

library("future")
cl <- parallel::makeCluster(2L, type = "MPI")
plan(cluster, workers = cl)

> x %<-% { Sys.sleep(5); Sys.getpid() }; print(1); y %<-% { Sys.sleep(5); Sys.getpid() }; print(2); z %<-% { Sys.sleep(5); Sys.getpidWarning message:
In resolved.ClusterFuture(future) :
  resolved() is not yet implemented for workers of class 'MPInode'. Will use value() instead and return TRUE
[1] 1
Warning message:
In resolved.ClusterFuture(future) :
  resolved() is not yet implemented for workers of class 'MPInode'. Will use value() instead and return TRUE
[1] 2
Warning message:
In resolved.ClusterFuture(future) :
  resolved() is not yet implemented for workers of class 'MPInode'. Will use value() instead and return TRUE
[1] 3
> x
[1] 13023
> y
[1] 13024
> z
[1] 13023
> Sys.getpid()
[1] 13019

[HenrikBengtsson]

Probably Rmpi::mpi.iprobe().

[HenrikBengtsson]

For my own records: To run the tests/mpi.R tests, use export _R_CHECK_FULL_=true.