reconsider treating DT[!int] as DT[-int]?

[franknarf1]

Looking at example(data.table), there's:

library(data.table)
DT = data.table(x=rep(c("b","a","c"),each=3), y=c(1,3,6), v=1:9)
DT[!2:4]                       # all rows other than 2:4
DT[-(2:4)]                     # same

But I didn't remember that this was a feature, and I guess the inconsistency with base R is unnecessary friction for people coming from base R and/or trying to understand the behavior by looking at help("!"), DT[, !2:4], etc.^^

This came up on SO from @moodymudskipper using DT2[!lengths(some_list_col)] to select rows where the list element had nonzero length.

Edit: There is already an exception to the rule that ! triggers a not-join when i=!logi:

data.table/R/data.table.R

Line 400 in 3d54417

if (is.logical(i)) {

I guess a similar check could appear somewhere after that

    if (is.integer(i)) {
      if (notjoin) {
        notjoin = FALSE
        i = !i
      }
    }

Or more generally the condition if (!is.list(i)) would match my understanding/expectations.

As IceCreamToucan notes on SO, packages depending on this behavior could be an obstacle to making the change that I had not considered.

---

^^ Re "why DT[, !2:4]?", my usual suggestion for understanding what's going on with x[z] is to inspect x[, z] but that doesn't work in this case due to the convenience features for column selection. Maybe I should suggest x[, print(z)] instead.

[moodymudskipper]

See also this SO question : https://stackoverflow.com/questions/56990956/data-table-has-unexpected-behavior-with-in-i-when-using-list-columns

[moodymudskipper]

Regarding package dependencies It would be an easy fix for these packages though (use - instead of !). And to avoid silent horrors we might trigger an error when !'s argument has a length different from the number of rows of the table.

I don't know how feasible it is but I suspect that the occurences of [! in CRAN packages importing data.table must not be so important ? assuming 15000 packages on cran, 10% importing data.table, 10% with code containing [! that's 150 occurences to check manually to warn maintainer upstream, and maybe 30 that will need correction.

The behavior could also produce warnings first, keeping current behavior, then warnings with new behavior, then no warning at all, so package owners will have the opportunity to see adapt their code.