Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functions to load CSV in Google Cloud Storage into a BQ table #155

Closed
wants to merge 1 commit into from

Conversation

@jarodmeng
Copy link
Contributor

commented Apr 25, 2017

Fix #126

insert_load_csv_from_cs_job() inserts an insert.load job to load a CSV file stored in Cloud Storage into BQ as a table. load_csv_from_cs() is a wrapper that waits for the insert job to finish.

@iainmwallace

This comment has been minimized.

Copy link

commented Jul 14, 2017

This is great feature - any chance that we could resolve the NEWS.md file conflict?

@everron

This comment has been minimized.

Copy link

commented Sep 1, 2017

This PR would be of great help. I am upping this. Plus, the NEWS.md conflict seems very minor

body <- list(
configuration = list(
load = list(
sourceFormat = "CSV",

This comment has been minimized.

Copy link
@byapparov

byapparov Mar 4, 2018

Contributor

Should not format be available as parameter? How one would specify NEWLINE_DELIMITED_JSON or AVRO?

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

Is there an obvious reason this shouldn't be an argument?

@@ -1,5 +1,8 @@
# Version 0.3.0.9000

* Add `insert_load_csv_from_cs_job()` and `load_csv_from_cs()` functions to help

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

Can you please move up to the current version?

#' @param table The name of the table to load data into
#' @param source_uris The fully-qualified URIs that point to a CSV file in
#' Google Cloud Storage; see
#' \href{https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.sourceUris}{the API documentation}

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

Can you please use markdown syntax here?

@@ -0,0 +1,93 @@
#' Insert a job to load a CSV file from Google Cloud Storage into a table
#'
#' This is a low-level function that creates an insert job. To wait until it is

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

Maybe this should be documented in the same file as load_csv_from_cs()?

body <- list(
configuration = list(
load = list(
sourceFormat = "CSV",

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

Is there an obvious reason this shouldn't be an argument?

#' @seealso API documentation for insert method:
#' \url{https://developers.google.com/bigquery/docs/reference/v2/jobs/insert}
#' @export
insert_load_csv_from_cs_job <- function(project, dataset, table,

This comment has been minimized.

Copy link
@hadley

hadley Mar 28, 2018

Member

I think this could just be insert_load_job()

@hadley

This comment has been minimized.

Copy link
Member

commented Apr 12, 2018

Done in bcce9b7

@hadley hadley closed this Apr 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.