Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Progress bar for dbWriteTable? #16

Open
cboettig opened this issue Feb 2, 2021 · 1 comment
Open

Progress bar for dbWriteTable? #16

cboettig opened this issue Feb 2, 2021 · 1 comment

Comments

@cboettig
Copy link

cboettig commented Feb 2, 2021

Is it possible to show a progress bar for long-running dbWriteTable commands?

@krlmlr
Copy link
Member

krlmlr commented Sep 14, 2021

Maybe with a callback function?

If we do it, it should be part of dbAppendTable() . The way the default method is currently implemented (via sqlAppendTableTemplate()) is abysmally slow for remote connections. The only portable solution seems to be to use dbQuoteLiteral() to create a long SQL query, which is what sqlAppendTable() does (or should do, r-dbi/DBI#362).

How long may these queries become? How to make the process RAM-friendly?

All of this is a lot of work and doesn't really fit the scope of DBI; doing this in the individual backends feels even worse. Maybe there's room for a package that specializes in writing/copying large datasets across DBI backends?

@krlmlr krlmlr closed this as completed Oct 17, 2021
@krlmlr krlmlr transferred this issue from r-dbi/DBI Nov 1, 2021
@krlmlr krlmlr reopened this Nov 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants