Could not find function "." and invalid closure when looking up for columns

[contefranz]

I am developing a package which imports data.table. I am currently using v1.10.4 on CRAN and I get strange behaviour when I want to access to columns within a data.table. The weird thing here is that it happens when I am in the debug mode due to package testing. Here is an example coming from my debug mode (I understand it is not reproducible...). The data.table is called locations:

Browse[2]> locations
##     start end
## 1:     5   7
## 2:     9  10
## 3:    21  22
Browse[2]> class( locations )
## [1] "data.table" "data.frame"
Browse[2]> locations[ , .( start, end ) ]
## Error in `[.data.frame`(x, i, j) : could not find function "."
Browse[2]> locations[ , max( start ) ]
## Error in max(start) : invalid 'type' (closure) of argument
Browse[2]> max( locations$start )
## [1] 21

Below you can find info about my R session.

Browse[2]> sessionInfo()
## R version 3.3.2 (2016-10-31)
## Platform: x86_64-apple-darwin13.4.0 (64-bit)
## Running under: macOS Sierra 10.12.3
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     

## other attached packages:
## [1] edgartools_1.0    curl_2.3          stringr_1.1.0     data.table_1.10.4

## loaded via a namespace (and not attached):
##  [1] R6_2.2.0        magrittr_1.5    tools_3.3.2     withr_1.0.2     roxygen2_6.0.1  Rcpp_0.12.9     memoise_1.0.0  
##  [8] xml2_1.1.1      stringi_1.1.2   digest_0.6.12   commonmark_1.1  devtools_1.12.0

Hope this helped!
Thanks in advance

[MichaelChirico]

What happens if you run data.table:::`[.data.table`(locations, j = .(start, end))?

[contefranz]

@MichaelChirico Well, it works actually...

Browse[2]> data.table:::`[.data.table`(locations, j = .(start, end))
##     start end
## 1:     5   7
## 2:     9  10
## 3:    21  22

Can you please explain? I see it is calling the internal function `[.data.table`] but why the usual way is broken?

[MichaelChirico]

be sure to check that data table is loaded (i.e. it's properly imported)... that's the only reason I know if why method dispatch might fail. I guess check search() before that line?

…

On Feb 14, 2017 2:12 AM, "Francesco Grossetti" ***@***.***> wrote: @MichaelChirico <https://github.com/MichaelChirico> Well, it works actually... Browse[2]> data.table:::`[.data.table`(locations, j = .(start, end)) ## start end ## 1: 5 7 ## 2: 9 10 ## 3: 21 22 Can you please explain? I see it is calling the internal function `[.data.table`] but why the usual way is broken? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2023 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHQQdZ7JbV7XmQzAI5R9Ia_FmFT0f0xfks5rcVPjgaJpZM4L_uOp> .

[contefranz]

So I think I found the issue here. It is related with the options with= which has been tweaked since version 1.9.8. What I do not understand is the different behaviour of data.table when in debug mode from when in normal interactive console. So for instance let's consider the following example after being sure that the new way is not enable with options(datatable.WhenJisSymbolThenCallingScope=FALSE).

Normal console mode

> dt = data.table( id = 1L, start = c( 9, 21, 5 ), end = c( 10, 22, 7 ) )
> dt
##    id start end
## 1:  1     9  10
## 2:  1    21  22
## 3:  1     5   7
> dt[ , start ] # prints the vector "start"
## [1]  9 21  5
> dt[ , "start" ] # prints a data.table
##    start
## 1:     9
## 2:    21
## 3:     5
> dt[ 1 ] # select the first row and prints it as a data.table
##    id start end
## 1:  1     9  10

Debug Mode

Browse[2]> dt = data.table( id = 1L, start = c( 9, 21, 5 ), end = c( 10, 22, 7 ) )
Browse[2]> dt
##    id start end
## 1:  1     9  10
## 2:  1    21  22
## 3:  1     5   7
Browse[2]> dt[ , start ] # it throws an error
## Error in .subset(x, j) : invalid subscript type 'closure'
Browse[2]> dt[ , "start" ] # it prints a vector
## [1]  9 21  5
Browse[2]> dt[ 1 ] # prints a data.table giving the first column
##    id
## 1:  1
## 2:  1
## 3:  1
Browse[2]> dt[ 1, ] # now it selects the first row
##    id start end
## 1:  1     9  10

Most importantly, it got barely impossible to create a new column by reference. See below.

Browse[2]> is.data.table( dt )
## [1] TRUE
Browse[2]> dt[ , total := start + end ] # I wrapped the error for readability
Error in `:=`(total, start + end) : Check that is.data.table(DT) == TRUE. 
Otherwise, := and `:=`(...) are defined for use in j, once only and in particular ways. 
See help(":=").

Maybe I am not getting it right, but I have already developed a package based on data.table so I have been using the debug mode a lot and didn't have any of these issues. I can see that things get better (i.e. no errors in debug mode) when I clean and rebuild the package. Are you aware of some for of conflicts with devtools v1.12.0 ?
What am I missing? @arunsrinivasan

[contefranz]

Update: I confirm that that problem is given by the check within the data.table function at line 2648, which states:

":=" <- function(...) stop('Check that is.data.table(DT) == TRUE. 
Otherwise, := and `:=`(...) are defined for use in j, once only and in particular ways. 
See help(":=").')

My function crashes when it finds the operator := (with or without breakpoints). If I launch a "Clean and Rebuild" or "Build and Reload" then the function does not a have any issue for say few times.
Any hints?

[jangorecki]

@contefranz could you confirm that issue persist in R console and is not related to IDE which you use to build and reload package?

[contefranz]

@jangorecki I just tested out and everything seems ok. As far as I can tell I think the problem was related with the devtools procedure load_all(). Is that sounds familiar to you?
Could you provide further insights please?

Thanks

[jangorecki]

@contefranz sounds familiar, I am using RStudio a lot but build and install I always do from shell thus no further insight really.

[contefranz]

Ok then. I close the issue since everything seems working now.