[ideas] tar_source and interactive debugging

[svraka]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I just discovered tar_source() recently, and ran into a small issue. Previously I used invisible(lapply(list.files(path = "R" pattern = "*.R", full.names = TRUE), source)) in my pipelines. With the latter approach it is possible to interactively debug functions sourced from R/ while having the debugger step through the source file. Meanwhile tar_source reads the scripts and then uses eval(parse(text = ...)) on the resulting R character vectors, in which case R will not be able to track where a function was defined.

Before @wlandau implemented tar_source there was some discussion on where source assigns functions in #896 and sourcing in Target Markdown was changed from sourceto eval(parse(...)) in c4f525a (although without any further explanation), so I suppose source() can be problematic.

But AFAIU using the file argument of parse() could work. It keeps the reference to the source files while having the ability to specify an environment, like this in tar_source:

-lines <- readLines(script, warn = FALSE)
-eval(parse(text = lines), envir = envir)
+eval(parse(file = script), envir = envir)

Of course, change_directory needs to be accommodated. parse() already uses readLines(..., warn = FALSE) so that's taken care of.

Would there be any drawbacks of such a change? Happy to open a PR if you think it's OK.

[wlandau]

You bring up a good idea, and I agree. Implemented in efa5da8. Does that help?

I originally had trouble with source() because it is hard to control the environment that the objects end up in. eval() helps ensure everything defined in the R scripts makes it into the environment in tar_option_get("envir").

[shikokuchuo]

@wlandau does source(local = envir) not work reliably here? Sorry I don't mean to crash this issue but I was looking at some evaluation context related stuff for the sprint. Would be worth investigating if there is an issue with source().

[wlandau]

I don't remember what the specific issue with local was in source(), only that eval() solved it. Can't seem to find my notes on that in this repo either.

[shikokuchuo]

Ok - let me know if you find anything. Eval / parse should always be a choice - if we have to resort to that here then there is probably a deficiency somewhere - and now would be a good opportunity to get that fixed!!

[shikokuchuo]

Or perhaps when you looked at it before you were trying local = TRUE rather than passing the environment explicitly? That the 'local' argument takes different types is not well known and not helped by the default being a logical value.

[wlandau]

I think it was an explicit local = envir, but I don't remember clearly enough to be sure.