Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.data pronoun does not work #132

move bot opened this issue Jul 21, 2018 · 3 comments


Copy link

@move move bot commented Jul 21, 2018

@sverchkov commented on Jul 20, 2018, 7:10 PM UTC:

Using the .data pronoun in filter, mutate, and transmute does not work with database tables


con <- DBI::dbConnect(RSQLite::SQLite(), ":memory:")
copy_to(con, mtcars)
mtcars2 <- tbl(con, "mtcars")

### Using mutate

# works
mutate(mtcars, foo = .data$cyl)

# also works
mutate(mtcars2, foo = cyl)

# doesn't
mutate(mtcars2, foo = .data$cyl)
# Error: Column `cyl` not found in `.data`

### Using transmute

# works
transmute(mtcars, blah = .data$cyl)

# also works
transmute(mtcars, blah = cyl)

# doesn't
transmute(mtcars2, blah = .data$cyl)
# Error: Column `cyl` not found in `.data`

### Using transmute to select

# works
transmute(mtcars, .data$cyl)

# also works
transmute(mtcars2, cyl)

# doesn't
transmute(mtcars2, .data$cyl)
# Error: Column `cyl` not found in `.data`

# select doesn't have this issue, this works
select( mtcars2, .data$cyl )

### Using filter

# works
filter( mtcars, .data$cyl > 4 )

# also works
filter( mtcars2, cyl > 4 )

# doesn't
filter( mtcars2, .data$cyl > 4 )
# Error: Column `cyl` not found in `.data`

(I actually ran into this when running code with a Postgres database, so it isn't DB-specific)

This is different from issue#3370, since everything works without the .data pronoun, but might be somehow related.

This issue was moved by krlmlr from tidyverse/dplyr#3722.


This comment has been minimized.

Copy link

@sverchkov sverchkov commented Aug 17, 2018

Looking into it I gathered that what happens is that the error is triggered because partial_eval eventually calls lang_partial_eval which calls bare_eval because of the $ in the expression.
As far as I can tell the .data pronoun never gets installed before that so it tries to get .data$column from the empty .data pronoun that rlang creates globally, hence the message.

There's a workaround (creating a data pronoun manually) that seems to work for most dplyr verbs, but fails with select and distinct:

var_strs <- dbplyr::op_vars(mtcars2)
x <- syms( var_strs )
names( x ) <- var_strs
my_pronoun <- rlang::as_data_pronoun( x )

# Now this works, as do most other dplyr verbs
mutate( mtcars2, foo = my_pronoun$cyl )

# But these fail:
select( mtcars2, my_pronoun$cyl )
# Error: `my_pronoun$cyl` must evaluate to column positions or names, not a symbol

distinct( mtcars2, my_pronoun$cyl )
# Error in result_create(conn@ptr, statement) : no such column: my_pronoun$cyl

select fails here because, unlike the other verbs, it never calls partial_eval, and the error message is coming from tidyselect::vars_select().

distinct apparently uses the names and not the values of the object that partial_eval returns. It also means that
distinct( mtcars2, cyl = anything )
functions identically to distinct( mtcars2, cyl ), which is not consistent with how the dataframe version works.

sverchkov pushed a commit to sverchkov/dbplyr that referenced this issue Aug 20, 2018

This comment has been minimized.

Copy link

@hadley hadley commented Jan 2, 2019

Minimal reprex:

library(dplyr, warn.conflicts = FALSE)
mf <- dbplyr::memdb_frame(a = 1, b = 2)

mutate(mf, foo = .data$a)
#> Error in if (length(ans) == 0L || as.character(ans[[1L]])[1L] == "~") {: missing value where TRUE/FALSE needed

Created on 2019-01-02 by the reprex package (v0.2.1)

This probably requires special treatment of the $ translation when the LHS is .data or .env

@hadley hadley changed the title Using .data pronoun in filter, mutate, and transmute does not work with DBI .data pronoun does not work Jan 3, 2019

This comment has been minimized.

Copy link

@hadley hadley commented Jan 3, 2019

And hence this is part of #200

@hadley hadley added this to the v1.4.0 milestone Jan 9, 2019
@hadley hadley closed this in 7253367 Jan 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
2 participants
You can’t perform that action at this time.