Error subscript out of bounds for by={list(var1, var2)}

[tdhock]

Hi I tried to use by={list(var1, var2)} and I am getting the same error message as #321 Error in bysubl[[jj + 1L]] : subscript out of bounds. I was expecting to get the same result as if I did by=list(var1, var2). If this syntax is not allowed, it would be useful to at least have a more informative error message.

Here is a MRE:

library(data.table)
dt <- data.table(
  State=c("ERROR", "COMPLETED", "ERROR"),
  ExitCode=c(1, 0, 2))

dt[, list(count=.N), by={
  list(State)
}]

dt[, list(count=.N), by={
  list(ExitCode)
}]

dt[, list(count=.N), by=list(State, ExitCode)]

dt[, list(count=.N), by={
  list(State, ExitCode)
}]

The output I got on my system was:

> dt <- data.table(
+   State=c("ERROR", "COMPLETED", "ERROR"),
+   ExitCode=c(1, 0, 2))
> dt[, list(count=.N), by={
+   list(State)
+ }]
        list count
1:     ERROR     2
2: COMPLETED     1
> dt[, list(count=.N), by={
+   list(ExitCode)
+ }]
   list count
1:    1     1
2:    0     1
3:    2     1
> dt[, list(count=.N), by=list(State, ExitCode)]
       State ExitCode count
1:     ERROR        1     1
2: COMPLETED        0     1
3:     ERROR        2     1
> dt[, list(count=.N), by={
+   list(State, ExitCode)
+ }]
Error in bysubl[[jj + 1L]] : subscript out of bounds
> 
> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: i686-pc-linux-gnu (32-bit)
Running under: Ubuntu 18.04.1 LTS

Matrix products: default
BLAS: /usr/lib/i386-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/i386-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base     

other attached packages:
[1] data.table_1.11.9       namedCapture_2017.06.01 RColorBrewer_1.1-2     
[4] lattice_0.20-35        

loaded via a namespace (and not attached):
 [1] httr_1.3.1      compiler_3.5.1  R6_2.2.2        tools_3.5.1    
 [5] withr_2.1.2     curl_3.2        memoise_1.1.0   grid_3.5.1     
 [9] knitr_1.20      git2r_0.23.0    digest_0.6.16   devtools_1.13.6
> 

[jangorecki]

it will be useful to handle such by and not raise error

[MichaelChirico]

Would be great to just use the approach in #3802 since what we're trying to accomplish is almost identical for auto-naming by and j.

Had made some progress to that effect but I guess there's some inconsistency in how by/j auto-name:

# take the first variable that is (1) not eval (#3758) and (2) starts with a character that can't start a variable name
tt = grep("^eval$|^[^[:alpha:]. ]", byvars, invert=TRUE, value=TRUE)
# byvars but exclude functions or `0`+`1` becomes `+`
tt = if (length(tt)) tt[1L] else all.vars(bysubl[[jj+1L]])[1L]

It makes sense to unify the approach (I believe) but that will have to wait for another release. Will have to hack a solution for this release 🤖