-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dbWriteTable()
too slow for remote databases and LOAD DATA LOCAL INFILE
disabled by default
#265
Comments
dbWriteTable()
too slow and LOAD DATA LOCAL INFILE
disableddbWriteTable()
too slow for remote databases and LOAD DATA LOCAL INFILE
disabled by default
Thanks. Can you please share the entire code you use to connect to your database? Please edit passwords and other sensitive information. |
Of course, thank you for the feedback @krlmlr . Please, consider this MWE: PREPARATION
METHOD 1: copy table using
METHOD 2: copy table using SQL
|
Thanks. RMariaDB has gained a What is the exact error message returned by |
Thank you, that's awesome news!
I tested for both the AWS remote server and my local MySQL database and neither worked, any idea why? |
Thanks. Can you please try remotes::install_github("r-dbi/RMariaDB#267") ? |
Awesome @krlmlr , it's working perfectly now both local and remote, thanks a lot! |
@krlmlr , do you plan to add |
I would expect it to work already. What does the error message for |
Sorry, I checked using the code below and it is indeed working flawlessly. Thanks again @krlmlr .
|
This old thread has been automatically locked. If you think you have found something related to this, please open a new issue and link to this old issue if necessary. |
Hey folks,
I believe it's a follow up to #162 and #11
DBI::dbWriteTable()
is incredibly slow and crashes for large tables (100MB+) for my AWS RDS MySQL database.LOAD DATA LOCAL INFILE
is impressively faster but it is disabled by default since MySQL 8.0.I can enable that to use
LOAD DATA LOCAL INFILE
within MySQL Workbench but I could not find a way to use it in R / DBI.Therefore I opened an issue here r-dbi/DBI#385
Any update on this matter? What is the current recommended approach to import large datasets to remote databases?
Thank you
The text was updated successfully, but these errors were encountered: