double-dot notation for formula

[kgoldfeld]

The double-dot notation for external variable reference is awesome. A couple of things.

(1) In my workflow, I often put the data definitions at the top, and it seems natural to sometimes first reference the external below

 #--- this doesn't work but is the way I like to do things

def <- defData(varname = "age", formula=10, dist = "nonrandom")
def <- defData(def, varname="agemult", 
               formula="age * ..age_effect", dist="nonrandom")
#> Error: Escaped variables referenced not defined (or not numeric): age_effect

age_effect <- 3
genData(2, def)
#>    id age
#> 1:  1  10
#> 2:  2  10

  #--- this does work as I am sure you know

age_effect <- 3

def <- defData(varname = "age", formula=10, dist = "nonrandom")
def <- defData(def, varname="agemult", 
               formula="age * ..age_effect", dist="nonrandom")

genData(2, def)
#>    id age agemult
#> 1:  1  10      30
#> 2:  2  10      30

(2) The strength of the double-dot notation is how it makes dynamic data definition so easy. To generate different data sets in R using different assumptions, there are at least two ways. The first approach, using for loops, works fine, but I find it a little clunky and a bit less amenable to parallelization. (And, by the way, this is the perfect kind of case where it doesn't make sense to introduce age_effect before creating def. In fact, it is just not possible - except to create a dummy version of age_effect, which is not so appealing. But, maybe there's no way around this.

  #--- this works but is a little clunky

list_of_data <- list()

age_effects <- c(0, 5, 10)
for (i in seq_along(age_effects)) {
  age_effect <- age_effects[i]
  list_of_data[[i]] <- genData(2, def)  
}

list_of_data
#> [[1]]
#>    id age agemult
#> 1:  1  10       0
#> 2:  2  10       0
#> 
#> [[2]]
#>    id age agemult
#> 1:  1  10      50
#> 2:  2  10      50
#> 
#> [[3]]
#>    id age agemult
#> 1:  1  10     100
#> 2:  2  10     100

I prefer to use lapply or mclapply - seems more compact and faster than the for loop, especially when doing 1000's of replications. But it doesn't work - and not sure how how hard it would be to make it work.

  #--- this doesn't work but is how I like to do things

myreplicate <- function(x, def) {
  age_effect <- x
  genData(2, def)
}

lapply(c(0, 5, 10), function(x) myreplicate(x, def))
#> [[1]]
#>    id age agemult
#> 1:  1  10     100
#> 2:  2  10     100
#> 
#> [[2]]
#>    id age agemult
#> 1:  1  10     100
#> 2:  2  10     100
#> 
#> [[3]]
#>    id age agemult
#> 1:  1  10     100
#> 2:  2  10     100

[assignUser]

Very happy that you like it. A quick thought from my phone, will test later: we could move the validation of ..vars from definition to generation but keep the other sanity checks (dist, valid arithmetic formula etc) that should solve your issue and would make sense I think.

[assignUser]

Yeah we can just move these and similar block in the other .check* functions into a sepreate function and use the new error system too.

simstudy/R/define_data.R

Lines 773 to 805 in bfcecd6

	naFormFuncs <-
	is.na(mget(
	formFuncs,
	ifnotfound = NA,
	mode = "function",
	envir = parent.frame(),
	inherits = TRUE
	))
	if (any(naFormFuncs)) {
	stop(paste(
	"Functions(s) referenced not defined:",
	paste(formFuncs[naFormFuncs], collapse = ", ")
	), call. = FALSE)
	}
	naDotVars <-
	is.na(mget(
	sub("..", "", formVars[dotVarsBol]),
	ifnotfound = NA,
	mode = "numeric",
	envir = parent.frame(),
	inherits = TRUE
	))
	if (any(naDotVars)) {
	stop(paste(
	"Escaped variables referenced not defined (or not numeric):",
	paste(names(naDotVars), collapse = ", ")
	),
	call. = FALSE
	)
	}

[kgoldfeld]

I noticed you never reacted to my second comment - was that intentional? Too hard to deal with?

[assignUser]

:D I will check it out once we moved the checks but I assume it has something to do with execution environments...