Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use [[ with scoped mutate #306

Closed
copernican opened this issue May 20, 2019 · 4 comments
Milestone

Comments

@copernican
Copy link

@copernican copernican commented May 20, 2019

I want to dynamically extract one or more keys from a map column using the new remote evaluation of [[. I would expect each of the cases in the reprex below to give equivalent output.

ID map column name map column quoted verb works as expected
1 x no mutate() yes
2 x no mutate_at() no
3 x yes mutate() no
4 x yes mutate_at() no
5 x no extract_keys() no
6 dim no extract_keys() yes

My best guess is that there is some interaction among [[, mutate_at(), and quasiquotation. The target DB is Presto, but this issue doesn't appear specific to Presto. Also, I suspect this is related to other recent issues, e.g., #305.

library(dplyr, warn.conflicts = FALSE)
library(dbplyr, warn.conflicts = FALSE)
library(purrr)
library(rlang, warn.conflicts = FALSE)

build_extractor <- function(map, key) {
  key_chr <- rlang::as_name(dplyr::enquo(key))
  map_quo <- rlang::enquo(map)
  rlang::list2(!!key_chr := ~ `[[`(!!map_quo, !!key_chr))
}

extract_keys <- function(.data, map, ...) {
  map_quo <- rlang::enquo(map)
  keys <- rlang::enquos(...)

  extractors <-
    keys %>%
    purrr::map(build_extractor, map = !!map_quo) %>%
    purrr::reduce(c)

  .data %>%
    dplyr::mutate_at(dplyr::vars(!!map_quo), extractors)
}

df <- tibble(x = list(c(a = 1, b = 2), c(a = 3, b = 4)))
df_dbi <- tbl_lazy(df)
map_var <- dplyr::quo(x)

df_dbi %>% mutate(a = x[["a"]], b = x[["b"]])
#> <SQL>
#> SELECT `x`, `x`.`a` AS `a`, `x`.`b` AS `b`
#> FROM `df`

df_dbi %>% mutate_at(vars(x), list(a = ~`[[`(., "a"), b = ~`[[`(., "b")))
#> Error in as.pairlist(list(...)): object 'x' not found

df_dbi %>% mutate(a = !!map_var[["a"]], b = !!map_var[["b"]])
#> <SQL>
#> SELECT `x`
#> FROM `df`

df_dbi %>% mutate_at(vars(!!map_var), list(a = ~`[[`(., "a"), b = ~`[[`(., "b")))
#> Error in as.pairlist(list(...)): object 'x' not found

df_dbi %>% extract_keys(x, a, b)
#> Error in as.pairlist(list(...)): object 'x' not found

df_dbi %>% rename(dim = x) %>% extract_keys(dim, a, b)
#> <SQL>
#> SELECT `dim`, `dim`.`a` AS `a`, `dim`.`b` AS `b`
#> FROM (SELECT `x` AS `dim`
#> FROM `df`) `dbplyr_001`

Created on 2019-05-20 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.4.0 (2017-04-21)
#>  os       Ubuntu 14.04.5 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Etc/UTC                     
#>  date     2019-05-20                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.4.0)
#>  backports     1.1.4   2019-04-10 [1] CRAN (R 3.4.0)
#>  callr         3.2.0   2019-03-15 [1] CRAN (R 3.4.0)
#>  cli           1.1.0   2019-03-19 [1] CRAN (R 3.4.0)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.4.0)
#>  DBI           1.0.0   2018-05-02 [1] CRAN (R 3.4.0)
#>  dbplyr      * 1.4.0   2019-04-23 [1] CRAN (R 3.4.0)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.4.0)
#>  devtools      2.0.2   2019-04-08 [1] CRAN (R 3.4.0)
#>  digest        0.6.19  2019-05-20 [1] CRAN (R 3.4.0)
#>  dplyr       * 0.8.1   2019-05-14 [1] CRAN (R 3.4.0)
#>  evaluate      0.13    2019-02-12 [1] CRAN (R 3.4.0)
#>  fs            1.3.1   2019-05-06 [1] CRAN (R 3.4.0)
#>  glue          1.3.1   2019-03-12 [1] CRAN (R 3.4.0)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.4.0)
#>  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.4.0)
#>  knitr         1.23    2019-05-18 [1] CRAN (R 3.4.0)
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 3.4.0)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.4.0)
#>  pillar        1.4.0   2019-05-11 [1] CRAN (R 3.4.0)
#>  pkgbuild      1.0.3   2019-03-20 [1] CRAN (R 3.4.0)
#>  pkgconfig     2.0.2   2018-08-16 [1] CRAN (R 3.4.0)
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.4.0)
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.4.0)
#>  processx      3.3.1   2019-05-08 [1] CRAN (R 3.4.0)
#>  ps            1.3.0   2018-12-21 [1] CRAN (R 3.4.0)
#>  purrr       * 0.3.2   2019-03-15 [1] CRAN (R 3.4.0)
#>  R6            2.4.0   2019-02-14 [1] CRAN (R 3.4.0)
#>  Rcpp          1.0.1   2019-03-17 [1] CRAN (R 3.4.0)
#>  remotes       2.0.4   2019-04-10 [1] CRAN (R 3.4.0)
#>  rlang       * 0.3.4   2019-04-07 [1] CRAN (R 3.4.0)
#>  rmarkdown     1.12    2019-03-14 [1] CRAN (R 3.4.0)
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.4.0)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.4.0)
#>  stringi       1.4.3   2019-03-12 [1] CRAN (R 3.4.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.4.0)
#>  testthat      2.1.1   2019-04-23 [1] CRAN (R 3.4.0)
#>  tibble        2.1.1   2019-03-16 [1] CRAN (R 3.4.0)
#>  tidyselect    0.2.5   2018-10-11 [1] CRAN (R 3.4.0)
#>  usethis       1.5.0   2019-04-07 [1] CRAN (R 3.4.0)
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.4.0)
#>  xfun          0.7     2019-05-14 [1] CRAN (R 3.4.0)
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.4.0)
#> 
#> [1] /mnt/home/sean_wilson/R/x86_64-pc-linux-gnu-library/3.4
#> [2] /usr/lib/R/library
@copernican copernican changed the title Translation of `[[` appears sensitive to map column name Possible interaction between translation of `[[` and mutate_at() May 20, 2019
@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented May 30, 2019

There's a lot going on in your reprex, but it seems like all of the problems have the same error, so I think it makes sense to fix that first:

library(dbplyr)
library(dplyr, warn.conflicts = FALSE)

lf <- lazy_frame(x = list(c(a = 1, b = 2), c(a = 3, b = 4)))
lf %>% mutate_at(vars(x), list(a = ~`[[`(., "a"), b = ~`[[`(., "b")))
#> <SQL>
#> Error in x[[]]: argument "i" is missing, with no default

Created on 2019-05-30 by the reprex package (v0.2.1.9000)

I see a different error message to you, but the most likely explanation is that I have some development version of something installed at the moment.

@hadley hadley added this to the v1.4.1 milestone May 30, 2019
@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented May 30, 2019

Simpler reprex with slightly different error:

library(dbplyr)
library(dplyr, warn.conflicts = FALSE)

lf <- lazy_frame(x = list(c(a = 1, b = 2), c(a = 3, b = 4)))
lf %>% mutate_all(list(a = ~ .$a, b = ~ .$b))
#> <SQL>
#> Error in eval_bare(x, .env): argument "y" is missing, with no default

Created on 2019-05-30 by the reprex package (v0.2.1.9000)

@hadley hadley changed the title Possible interaction between translation of `[[` and mutate_at() Can't use [[ with scoped mutate May 30, 2019
@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented Jun 4, 2019

Installed cran dplyr and now I'm getting the same error as above.

@copernican

This comment has been minimized.

Copy link
Author

@copernican copernican commented Jun 4, 2019

As a follow-up, I've discovered that using the names of other functions for the list column "avoids" the error, though the output does not appear correct, either. It just so happened that I noticed the dim case above because that was the name of a column in a database table I was using.

library(dbplyr)
library(dplyr, warn.conflicts = FALSE)

lf0 <- lazy_frame(x = list(c(a = 1, b = 2), c(a = 3, b = 4)))
lf0 %>% mutate_all(list(a = ~ .$a, b = ~ .$b))
#> Error in as.pairlist(list(...)): object 'x' not found

lf1 <- lazy_frame(match = list(c(a = 1, b = 2), c(a = 3, b = 4)))
lf1 %>% mutate_all(list(a = ~ .$a, b = ~ .$b))
#> <SQL>
#> SELECT `match`, `.`.`a` AS `a`, `.`.`b` AS `b`
#> FROM `df`

lf2 <- lazy_frame(filter = list(c(a = 1, b = 2), c(a = 3, b = 4)))  
lf2 %>% mutate_all(list(a = ~ .$a, b = ~ .$b))
#> <SQL>
#> SELECT `filter`, `.`.`a` AS `a`, `.`.`b` AS `b`
#> FROM `df`

Created on 2019-06-04 by the reprex package (v0.3.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.