support for dplyr::across?

[edavidaja]

across is very exciting--I was hoping to use it to solve a problem like this a bit more readably.

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(dtplyr)

lazy_dt(iris) %>% 
  mutate(
    across(starts_with("Sepal"), ~ .x^2, .names = "{col}_squared")
  )
#> Source: local data table [?? x 6]
#> Call:   copy(`_DT1`)[, `:=`(`across(starts_with("Sepal"), ~.x^2, .names = "{col}_squared")` = across(starts_with("Sepal"), 
#>     ~.x^2, .names = "{col}_squared"))]
#> Error: peek_mask() must only be used inside dplyr verbs

^{Created on 2020-04-07 by the reprex package (v0.3.0)}

Session info

sessionInfo()
#> R version 3.6.3 (2020-02-29)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 10 x64 (build 19592)
#> 
#> Matrix products: default
#> 
#> locale:
#> [1] LC_COLLATE=English_United States.1252 
#> [2] LC_CTYPE=English_United States.1252   
#> [3] LC_MONETARY=English_United States.1252
#> [4] LC_NUMERIC=C                          
#> [5] LC_TIME=English_United States.1252    
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices datasets  utils     methods   base     
#> 
#> other attached packages:
#> [1] dtplyr_1.0.1.9000 dplyr_0.8.99.9002
#> 
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_1.0.4        knitr_1.28        magrittr_1.5      tidyselect_1.0.0 
#>  [5] R6_2.4.1          rlang_0.4.5.9000  fansi_0.4.1       stringr_1.4.0    
#>  [9] highr_0.8         tools_3.6.3       data.table_1.12.8 xfun_0.12        
#> [13] cli_2.0.2         htmltools_0.4.0   ellipsis_0.3.0    yaml_2.2.1       
#> [17] digest_0.6.25     assertthat_0.2.1  tibble_3.0.0      lifecycle_0.2.0  
#> [21] crayon_1.3.4      purrr_0.3.3       vctrs_0.2.99.9010 glue_1.4.0       
#> [25] evaluate_0.14     rmarkdown_2.1     stringi_1.4.6     compiler_3.6.3   
#> [29] pillar_1.4.3      generics_0.0.2    renv_0.9.3        pkgconfig_2.0.3

[hadley]

Moderately comprehensive selection of calls that should work:

library(dtplyr)
library(dplyr, warn.conflicts = FALSE)

dt <- lazy_dt(data.frame(x = 1:5, y = 5:1))
dt %>% mutate(across(everything(), ~ .x * 2))
dt %>% summarise(across(everything(), mean))
dt %>% group_by(across(everything()))
dt %>% arrange(across(everything(), desc))

dt %>% filter(if_all(everything(), ~ . == 3))
dt %>% filter(if_any(everything(), ~ . > 3))

[hadley]

In the dev version now. It took me a surprisingly small amount of time to implement which might mean that I forgot something important, so please try it out 😄

[mjantti]

Many thanks, this works nicely!

In research papers, I use dplyr a lot to estimate multiple functions, placing each function in a list as in "fl" below. When using dtplyr, this construct seems not to work. There are workarounds but it would be nice to be able to do this:

library(dtplyr)
library(dplyr, warn.conflicts = FALSE)

fl <- list(~ .x*2, ~ .x*5)
data.frame(x = 1:5, y = 5:1) %>% mutate(across(everything(), fl))
dt %>% mutate(across(everything(), fl))

[hadley]

@mjantti in general, it's better to file a new issue, but for this specific problem, I don't currently see any way to support what you want, given the constraints of the translation.

[mjantti]

Thanks for the quick response and sorry for the breach of etiquette. Did note this is closed and also kind of guessed that had this been doable in the same way, it would have been fixed already.

[Jollywatt]

fl <- list(~ .x*2, ~ .x*5)
data.frame(x = 1:5, y = 5:1) %>% mutate(across(everything(), fl))

This is hardly a "breach of etiquette" but a cornerstone of programming; variable substitution. It strikes fear into my soul to see @hadley admitted defeat here.

Any progress on this? (Specifically, allowing dtplyr to translate aggregation functions from values.)