[Proposal]: support batch loading of data #2
Labels
enhancement
New feature or request
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Contact Details
No response
Summary of your proposal
We currently support the insertAll API and soon also the storage API. What we do not yet support is batch loading as documented in https://cloud.google.com/bigquery/docs/batch-loading-data. They give an example of a single file, but we could do it for any number of files as well as any reader in general.
Need to investigate the specifics, but it does look like it is still in scope.
Motivation for your proposal
It's a different kind of use case of BQWriter, still writing data into BQ but for an entirely different purpose. For those purposes batch loading is better suited as the timing isn't as critical, and with that a reduction of cost is a given.
Alternatives for your proposal
Do not support it and explicitly document so instead.
Version
0.3.1 (Latest)
What platform are you mostly using or planning to use our software on?
MacOS
Code of Conduct
The text was updated successfully, but these errors were encountered: