Why does wrapping make() in a function invalidate some targets?

[tiernanmartin]

While trying to write a wrapper function for make() I noticed some puzzling behavior: making a plan using a wrapper function invalidates some targets but not all of them.

Is this a bug or a feature?

Here's a reprex using the "main" example:

library(drake)
library(tidyverse)
source("R/functions.R")
source("R/plan.R") 

options(drake_make_menu = FALSE)


# Make the plan

make(plan)
#> target raw_data
#> target data
#> target fit
#> target hist
#> target report

# Make it again (all targets should be up to date)
make(plan)
#> All targets are already up to date.

# Wrap make() inside a function and make the plan using this new function
new_make <- function(x){make(plan = x)}

new_make(plan)
#> target hist
#> target report

[wlandau]

Great question.

Related: #163, #335, and #527. This behavior is an unfixable (but perfectly avoidable) side effect of fundamental design choices:

1. Scan your environment for dependencies, and
2. Automate most of this behind the scenes.

Details

make() has an envir argument, and this is where it looks for functions and other in-memory dependencies. By default, envir is just the calling environment (parent.frame()) which is the global environment in ordinary usage. When you call make() inside a function, you are now in the function's environment, and make() does not see functions defined elsewhere. (It ignores parent environments when it looks for functions.)

Workarounds

1. Either use r_make() and friends, which do not suffer this strangeness, or
2. Pass the correct environment to make() manually.

library(drake)
load_mtcars_example()

make(my_plan)
#> target large
#> target small
#> target regression1_large
#> target regression2_large
#> target regression1_small
#> target regression2_small
#> target summ_regression1_large
#> target coef_regression1_large
#> target summ_regression2_large
#> target coef_regression2_large
#> target summ_regression1_small
#> target coef_regression1_small
#> target coef_regression2_small
#> target summ_regression2_small
#> target report

make(my_plan)
#> All targets are already up to date.

outer_make <- function(x, envir){
  make(plan = x, envir = envir)
}

envir <- environment()
outer_make(my_plan, envir)
#> All targets are already up to date.

^{Created on 2019-05-19 by the reprex package (v0.3.0)}

[tiernanmartin]

Got it!

Thanks for the clear explanation.