Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`copy_to` fails to overwrite a temporary table #258

Closed
mkirzon opened this issue Mar 7, 2019 · 4 comments

Comments

@mkirzon
Copy link

@mkirzon mkirzon commented Mar 7, 2019

Using SQL Server connections, overwriting a table created by copy_to() with temporary=TRUE does not work. As a workaround, we first have to run try(db_drop_table(con, "##test")) between calls to copy_to to ensure we can overwrite the temp table.

library(tidyverse)

# Initialize 2 dataframes
df1 = data_frame(value = c(1,2,3))
df2 = data_frame(value = c(4,5,6))

# Initialize connection and upload the first dataframe
con = DBI::dbConnect(odbc::odbc(), Driver = "SQL Server", Server = "SQLServerName", Database = "DBName")
tbl_tmp = copy_to(con, df1, name = "test", temporary = TRUE)
#> Created a temporary table named: ##test

# Test the first upload
tbl_tmp
#> # Source:   table<##test> [?? x 1]
#> # Database: Microsoft SQL Server
#> #   13.00.5216[user@SQLServerName/DBName]
#>   value
#>   <dbl>
#> 1     1
#> 2     2
#> 3     3

# Overwrite original table with the new dataframe
tbl_tmp = copy_to(con, df2, name = "test", temporary = TRUE, overwrite = TRUE)
#> Created a temporary table named: ##test
#> Error: <SQL> 'CREATE TABLE "##test" (
#>   "value" FLOAT
#> )
#> '
#>   nanodbc/nanodbc.cpp:1587: 42S01: [Microsoft][ODBC SQL Server Driver][SQL Server]There is already an object named '##test' in the database.
@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented Mar 14, 2019

I don't have SQL server locally — while I've simulated the change and I'm reasonably certain it should fix the problem, I would appreciate someone (@edgararuiz ?) also checking this with a real SQL server instance.

@krlmlr

This comment has been minimized.

Copy link
Member

@krlmlr krlmlr commented Apr 8, 2019

FWIW, ## denotes global temporary tables, and # denotes local ones (https://stackoverflow.com/a/2921091/946850). Global temporary tables aren't auto-destructed when the connection terminates.

A pull request follows. I'm working on a terraformed setup of various databases on Azure for testing, happy to share.

@mkirzon

This comment has been minimized.

Copy link
Author

@mkirzon mkirzon commented May 15, 2019

I saw this was closed in the March commit and mentioned in the change log. However, the same exact reprex fails in dbplyr 1.4.0 on sql server

@florisvdh

This comment has been minimized.

Copy link

@florisvdh florisvdh commented Jun 25, 2019

Having met this same problem (with dbplyr at current master), I see that both db_has_table() and dbExistsTable() return FALSE, for a temporary table which is present in the SQL Server database. From the traceback I see that db_has_table() is evaluated before writing (even while overwrite = TRUE) and this erroneous result possibly causes the problem.

I currently use the suggested workaround from above: try(db_drop_table(con, "##test"), silent = TRUE), which works.

An extract from my traceback:

11. `db_write_table.Microsoft SQL Server`(con, table, types = types,      values = values, temporary = temporary) at dbplyr.R#90

10. db_write_table(con, table, types = types, values = values, temporary = temporary) at verb-copy-to.R#116

9. with_transaction(con, {
    if (overwrite && !is_false(db_has_table(con, table))) {
        db_drop_table(con, table, force = TRUE)
    } ... at verb-copy-to.R#110
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.