Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make compatible with dbplyr #39

Closed
hadley opened this issue Mar 30, 2017 · 3 comments
Closed

Make compatible with dbplyr #39

hadley opened this issue Mar 30, 2017 · 3 comments

Comments

@hadley
Copy link
Member

@hadley hadley commented Mar 30, 2017

  1. The implementation of the dplyr database backends has moved to dbplyr. Most of the generics remain in dplyr, so that existing backends don't break.

  2. By inspecting the namespace, you can now easily see what methods you need to override for non-DBIConnections. Here's the complete list

 db_analyze
 db_begin
 db_collect
 db_commit
 db_compute
 db_copy_to
 db_create_index
 db_create_indexes
 db_create_table
 db_data_type
 db_desc
 db_drop_table
 db_explain
 db_has_table
 db_insert_into
 db_list_tables
 db_query_fields
 db_query_rows
 db_rollback
 db_save_query
 db_sql_render
 db_write_table
 sql_escape_ident
 sql_escape_string
 sql_join
 sql_select
 sql_semi_join
 sql_subquery
 sql_translate_env
  1. There are a lot of methods, but most of them simply require you to acquire the connection, then release it when it's done. To make that easier, I've written you this function to generate the wrapper functions:
 library(dplyr, warn.conflicts = FALSE)
 library(rlang)

pool_wrap <- function(f) {
 f <- enquo(f)

formals <- formals(eval_tidy(f))
 con <- names(formals)[[1]]
 passed_on <- names(formals[-1]) %>%
 set_names() %>% 
 lapply(as_symbol)

body <- expr({
 db_con <- poolCheckout(!!sym(con))
 on.exit(poolReturn(db_con))

UQE(f)(db_con, !!!passed_on)
 })

new_function(formals, body, f_env(f))
 }

pool_wrap(db_analyze)
 #> function (con, table, ...) 
 #> {
 #>     db_con <- poolCheckout(con)
 #>     on.exit(poolReturn(db_con))
 #>     db_analyze(db_con, table = table, ... = ...)
 #> }
 #> <environment: 0x7ff69c8927c8>
  1. There are at least two exceptions: copy_to() and compute() will need need to throw an error if temporary = TRUE because temporary tables are local to a connection, and there's no guarantee you'll get the same connection back next time.

I'm reasonably certain I've made all the changes to dbplyr to facilitate your Pool, but please let me know if I've missed anything. I'm also happy to hop on a call with you and @jcheng5 to answer any questions that you might have.

@GrayAlex49
Copy link

@GrayAlex49 GrayAlex49 commented Jun 12, 2017

Am I correct in assuming that since this is still open, the current pool package is not compatible with dplyr 0.7 + dbplyr?
Running src_pool(pool) %>% tbl("database_table") %>% head(5) where pool was created using dbPool as described in the Using dplyr with pool article returns the error

Error: 'src_sql' is not an exported object from 'namespace:dplyr'

Also, the examples in that article dont seem to run using dplyr 0.7

@bborgesr
Copy link
Contributor

@bborgesr bborgesr commented Jun 14, 2017

That's true. I'm working on compatibility right now, but we were out of synch for this release of dplyr (and dbplyr), so running a bit behind.. Sorry for the inconvenience!

If you want to keep using both packages now, the workaround is to use dplyr 0.5.0 (devtools::install_github("hadley/dplyr@v0.5.0")) We should have full compatibility very soon though!

@GrayAlex49
Copy link

@GrayAlex49 GrayAlex49 commented Jun 14, 2017

@bborgesr No worries, thanks for all yall do. Just wanted to make sure I hadn't misunderstood.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants
You can’t perform that action at this time.