.SD ignores locally updated vectors in j

[jan-swissre]

I would expect both expressions give the same result.
Filtering in i is irrelevant, I have kept it to reflect my use case.

data.table(a = 1:3, b = 1:3, d = 1:3
)[a %in% 1:2, {
  b[d == 2L] <- 0L
  .SD
}]
#       a     b     d
#1:     1     1     1
#2:     2     2     2

data.table(a = 1:3, b = 1:3, d = 1:3
)[a %in% 1:2, {
  b[d == 2L] <- 0L
  .(a, b, d)
}]
#       a     b     d
#1:     1     1     1
#2:     2     0     2

1.17.0

workaround for now:

data.table::data.table(a = 1:3, b = 1:3, d = 1:3
)[a %in% 1:2, {
  b[d == 2L] <- 0L
  .SD
}, env = list(.SD = as.list(c("a","b","d")))]

[MichaelChirico]

This is a documentation issue IMO. I don't think there's any way to make this work as you'd hoped. .SD is populated up-front, not dynamically/lazily.

[venom1204]

Hi @MichaelChirico @tdhock ,

My proposed fix focuses entirely on clarifying the static snapshot nature of .SD to prevent future confusion.

Here are the specific documentation files I propose to modify:

• man/special-symbols.Rd: To add a critical note explicitly defining .SD as a static copy initialized before the j expression begins, explaining why local assignments are ignored by .SD.
• vignettes/datatable-sd-usage.Rmd: To incorporate a concrete example (like the one in the issue description) that clearly contrasts returning .SD versus returning explicit columns (.(a, b, d)) after a local vector modification.

Does this targeted documentation plan sound good to you? If so, I am happy to proceed with a Pull Request.
Thanks!

[tdhock]

local assignments are ignored by .SD.

is that true? if so an error would be better.

[jangorecki]

To not break existing workflows probably warning would be better. Question is to what degree we are able to "scan" j for assignments to columns that exist in .SD with a minimal overhead. There are many corner cases but even catching simple ones like those may cover majority of use cases.