.data pronoun does not work

@sverchkov commented on Jul 20, 2018, 7:10 PM UTC:

Using the .data pronoun in filter, mutate, and transmute does not work with database tables

library(dplyr)

con <- DBI::dbConnect(RSQLite::SQLite(), ":memory:")
copy_to(con, mtcars)
mtcars2 <- tbl(con, "mtcars")

### Using mutate

# works
mutate(mtcars, foo = .data$cyl)

# also works
mutate(mtcars2, foo = cyl)

# doesn't
mutate(mtcars2, foo = .data$cyl)
# Error: Column `cyl` not found in `.data`

### Using transmute

# works
transmute(mtcars, blah = .data$cyl)

# also works
transmute(mtcars, blah = cyl)

# doesn't
transmute(mtcars2, blah = .data$cyl)
# Error: Column `cyl` not found in `.data`

### Using transmute to select

# works
transmute(mtcars, .data$cyl)

# also works
transmute(mtcars2, cyl)

# doesn't
transmute(mtcars2, .data$cyl)
# Error: Column `cyl` not found in `.data`

# select doesn't have this issue, this works
select( mtcars2, .data$cyl )

### Using filter

# works
filter( mtcars, .data$cyl > 4 )

# also works
filter( mtcars2, cyl > 4 )

# doesn't
filter( mtcars2, .data$cyl > 4 )
# Error: Column `cyl` not found in `.data`

(I actually ran into this when running code with a Postgres database, so it isn't DB-specific)

This is different from issue#3370, since everything works without the .data pronoun, but might be somehow related.

This issue was moved by krlmlr from tidyverse/dplyr#3722.

[sverchkov]

Looking into it I gathered that what happens is that the error is triggered because partial_eval eventually calls lang_partial_eval which calls bare_eval because of the $ in the expression.
As far as I can tell the .data pronoun never gets installed before that so it tries to get .data$column from the empty .data pronoun that rlang creates globally, hence the message.

There's a workaround (creating a data pronoun manually) that seems to work for most dplyr verbs, but fails with select and distinct:

var_strs <- dbplyr::op_vars(mtcars2)
x <- syms( var_strs )
names( x ) <- var_strs
my_pronoun <- rlang::as_data_pronoun( x )

# Now this works, as do most other dplyr verbs
mutate( mtcars2, foo = my_pronoun$cyl )

# But these fail:
select( mtcars2, my_pronoun$cyl )
# Error: `my_pronoun$cyl` must evaluate to column positions or names, not a symbol

distinct( mtcars2, my_pronoun$cyl )
# Error in result_create(conn@ptr, statement) : no such column: my_pronoun$cyl

select fails here because, unlike the other verbs, it never calls partial_eval, and the error message is coming from tidyselect::vars_select().

distinct apparently uses the names and not the values of the object that partial_eval returns. It also means that
distinct( mtcars2, cyl = anything )
functions identically to distinct( mtcars2, cyl ), which is not consistent with how the dataframe version works.

[hadley]

Minimal reprex:

library(dplyr, warn.conflicts = FALSE)
mf <- dbplyr::memdb_frame(a = 1, b = 2)

mutate(mf, foo = .data$a)
#> Error in if (length(ans) == 0L || as.character(ans[[1L]])[1L] == "~") {: missing value where TRUE/FALSE needed

^{Created on 2019-01-02 by the reprex package (v0.2.1)}

This probably requires special treatment of the $ translation when the LHS is .data or .env

[hadley]

And hence this is part of #200