Missing globals in future_lapply

[conroylau]

Hi @HenrikBengtsson, hope all is well.

I am using the future_lapply function in a R package that I am currently developing. I run into a problem where the future_lapply function cannot find some functions or objects that are passed from the user to the package.

A minimal working example is as follows where func and func.square represent two functions in my package:

• The function func.main has an argument f that represents a function passed from the user.

func.main <- function(f) {
  list.return <- future.apply::future_lapply(X = 1:5,
                                             FUN = func.square,
                                             f = f)
  return(unlist(list.return))
}

• The function func.square represents a function that carries out some data processing and computations in my package. It is doing a simple arithmetic here for illustration purpose.

func.square <- function(x, f) {
  temp <- f(x)^2 + 10
  return(temp)
}

There is no issue when the function f is defined as follows and passed to func.main:

library(future)
plan(multisession, workers = 2)
f <- function(x) x^2 + 1
func.main(f)
#> [1]  14  35 110 299 686

However, an error occurs if the user specifies the function as follows where it contains a scalar a and function b that are defined outside the function f:

library(future)
plan(multisession, workers = 2)
a <- 1
b <- function(x) x^2
f <- function(x) b(x) + a
func.main(f)
#>  Error in b(x) : could not find function "b" 

While I understand that this error vanishes if I define the scalar a and function b in the future.globals option of the future_lapply function, the problem is that my package does not know the scalar a and function b in advance as it is something that the user passes to the functions in my package. I also want to maintain the minimum amount of input required from the user.

May I know what would be a workaround to this problem, or is there any method to make sure that the objects a and b can be found?

Thanks for your help in advance!

[HenrikBengtsson]

UPDATE: This is because how, and more specifically where, future.apply looks for globals. Until fixed, furrr 0.2.0 has solved this already, i.e.

func.main <- function(f) {
  list.return <- furrr::future_map(1:5,
                                   func.square,
                                   f = f)
  return(unlist(list.return))
}

will work.

[conroylau]

Thanks for the update! This works in my package.