Error with shiny runtime and data.table := #187

Closed
jrowen opened this Issue Jul 24, 2014 · 8 comments

Comments

Projects
None yet
4 participants

jrowen commented Jul 24, 2014

I'm running into the following error with this markdown document:


---
title: "Untitled"
output: html_document
runtime: shiny

---

```{r}
library(data.table)
DT = data.table(mtcars)
DT[, mpg:=mpg / 10]
DT
```

R Markdown output is:

  |......................                                           |  33%
  ordinary text without R code

  |...........................................                      |  67%
label: unnamed-chunk-1


processing file: test.Rmd

Quitting from lines 8-12 (test.Rmd) 
Error in `:=`(mpg, mpg/10) : 
  Check that is.data.table(DT) == TRUE. Otherwise, := and `:=`(...) are defined for use in j, once only and in particular ways. See help(":=").

This runs fine if runtime: shiny is omitted.

> sessionInfo()
R version 3.1.1 (2014-07-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] digest_0.6.4     htmltools_0.2.4  rmarkdown_0.2.50 tools_3.1.1      yaml_2.1.13 
Member

yihui commented Jul 26, 2014

Sometimes I do not understand Matt/data.table, and sometimes he does not understand me/knitr. This time it seems to be an evaluation environment problem. A fix that cures the symptom is to make parent.frame() the default environment for rmarkdown::run(), which happens to be .GlobalEnv by default (and everybody is happy in the global environment):

diff --git a/R/shiny.R b/R/shiny.R
index 20e2031..842d0eb 100644
--- a/R/shiny.R
+++ b/R/shiny.R
@@ -83,6 +83,8 @@ run <- function(file = "index.Rmd", dir = dirname(file), auto_reload = TRUE,
       "UTF-8"
     else
       render_args$encoding
+  # default environmnet to compile Rmd
+  if (is.null(render_args$envir)) render_args$envir <- parent.frame()

   onStart <- function() {
     global_r <- file.path.ci(dir, "global.R")

The real problem can be much deeper than this. I do not have time to make a full investigation...

jrowen commented Aug 6, 2014

The suggested change fixes the example above but doesn't work with interactive documents. The Rmd below results in the error: Error in eval(expr, envir, enclos) : object 'input' not found.

---
title: "Untitled"
output: html_document
runtime: shiny
---

```{r, echo=FALSE}
library(data.table)
inputPanel(
  numericInput("num", "MPG", value=10)  
)

renderTable({
  DT = data.table(mtcars)
  DT[, mpg:=mpg / 10]
  DT[mpg > input$num]
})
```
Member

yihui commented Aug 15, 2014

Then I do not know how to solve this problem. I do not fully understand the environment for evaluating the expressions in DT[...].

Similar has come up before :
http://stackoverflow.com/questions/13106018/data-table-error-when-used-through-knitr-gwidgetswww

I missed that comment from @mbask about "slidify", so I've just added that to the list making it 4 packages now on the whitelist at the top of data.table/cedta.R :

cedta.override = c("gWidgetsWWW","statET","FastRWeb","slidify")
# user may add more to this using :
# assignInNamespace("cedta.override", c(data.table:::cedta.override,"<nsname>"), "data.table")

In this case @jrowen could you try the following and if it works let me know and I'll add those upstream to the list in data.table so you don't have to do it yourself from the next data.table release.

assignInNamespace("cedta.override", c(data.table:::cedta.override,"rmarkdown"), "data.table")

If that doesn't work then we need to know the namespace name that is running the lines in the script. Turn on data.table verbosity : options(datatable.verbose=TRUE), run the script again and you should get a message : cedta decided <nsname> wasn't data.table aware. That's the name that needs to be added to the whitelist.

Alternatively, packages like these which evaluate code themselves as if they are at the prompt, can set .datatable.aware=TRUE in their namespace. But that has to be done by the package author/maintainer not the user as a workaround, since assignInNamespace() can't add new bindings to a namespace, only change existing ones.

So either I can add rmarkdown to data.table's whitelist or rmarkdown can set .datatable.aware=TRUE. Either way.

Matt

jrowen commented Aug 18, 2014

I added the assignInNamespace line right after the library(data.table) and everything appears to be running without error. There is one warning, shown below. Thanks to everyone for taking a look at this.

Warning in addResourcePath(prefix, dependency$src$file) :
  Overriding existing prefix rmarkdown-performance-0.1 => C:\Users\jrowen\AppData\Local\Temp\RtmpCMgARF\filea284e3457ea_files

@jrowen Thanks for update. Have now added rmarkdown to the whitelist in data.table v1.9.3 on github.

Sorry - I don't know about the addResourcePath warning. Suggest closing this issue and raising a new separate issue for that.

@yihui yihui added this to the v0.98-900-patch milestone Aug 20, 2014

Member

yihui commented Aug 20, 2014

@jrowen You can ignore the addResourcePath() warning. It has been fixed by rstudio/shiny#567

@yihui yihui closed this Aug 20, 2014

In case other people stumble upon this problem:

Even if your package just passes some arguments to knitr or rmarkdown, you need to make it data.table aware (the function at the bottom of this script in my example). Luckily this just means adding the line .datatable.aware = TRUE somewhere in your code.

I don't know if it's possible to make data.table smarter about this (it must lead to unforeseen and untested problems whenever any non-data.table-aware packages allows for the execution of custom code). In my case data.table was complaining about := assignment even though is.data.table was true. That seems overly sensitive, after all := has no "normal, vanilla data frame" use? But of course I see the reasoning whenever variables are accessed etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment