-
Notifications
You must be signed in to change notification settings - Fork 3
Installation
Fabien TAYSSE edited this page May 27, 2019
·
2 revisions
-
Create a new
bq-autoload
Google Cloud Storage bucket$> gsutil mb -c regional -l europe-west1 "gs://bq-autoload"
-
Create a new
Staging
BigQuery dataset$> bq mk --dataset "Staging"
-
Clone and deploy this repository as a cloud function triggered by changes on this GCS bucket (do not forget to replace the project id)
$> git clone "https://github.com/tfabien/bigquery-autoload/" \ && cd "bigquery-autoload" \ && gcloud functions deploy "bigquery-autoload" \ --trigger-bucket "bq-autoload" \ --set-env-vars "PROJECT_ID={{YOUR_GCP_PROJECT_ID}}" \ --runtime "nodejs10" \ --memory "128MB"
Any file you upload to the bq_autoload
GCS bucket will now automatically be loaded into a BigQuery table within seconds.