Added the extract job: BigQuery -> Cloud Storage #119
Hello. Have been using this function for a while in my personal fork and always wanted to share it with the community.
This is the code for doing a BigQuery extract job.
The following code contains an example of how to use it
# specify your project ID here project <- "<my_project_id>" # specify your Cloud Storage bucket name (note the wildcard) bucket <- "gs://<my_bucket>/shakespeare*.csv" # Now run the extract_exec - it will return the number of files that were extracted extract_exec("publicdata:samples.shakespeare", project = project, destinationUris = bucket)
Shakespeare is a small dataset so you won't get charged a lot for that example.
The structure of this file is modeled after
@hadley Absolutely! Just wasn't sure if the package is still actively developed.
This particular PR was very useful for me internally since it opens way for what I called as
@realAkhmed agreed that this is indeed a faster route in general -- but CSV as a transport format isn't great for the case that your table includes nested or repeated fields.
I'd be curious what @hadley knows about potentially converting avro to a dataframe, since that would give us full fidelity exports.