A Singer target that writes data to Google BigQuery.
How to use it
Step 1: Activate the Google BigQuery API
(originally found in the Google API docs)
- Use this wizard to create or select a project in the Google Developers Console and activate the BigQuery API. Click Continue, then Go to credentials.
- On the Add credentials to your project page, click the Cancel button.
- At the top of the page, select the OAuth consent screen tab. Select an Email address, enter a Product name if not already set, and click the Save button.
- Select the Credentials tab, click the Create credentials button and select OAuth client ID.
- Select the application type Other, enter the name "Singer BigQuery Target", and click the Create button.
- Click OK to dismiss the resulting dialog.
- Click the Download button to the right of the client ID.
- Move this file to your working directory and rename it client_secrets.json.
Step 2: Configure
Create a file called
config.json in your working directory, following config.sample.json. The required parameters are the project name
project_id, the dataset name
dataset_id, and table name
Step 3: Install and Run
These commands will install
target-bigquery with pip and then run them together, piping the output of
› pip install target-bigquery tap-fixerio › tap-fixerio | target-bigquery -c config.json INFO Replicating the latest exchange rate data from fixer.io INFO Tap exiting normally
If you're using a different Tap, substitute
tap-fixerio in the final command above to the command used to run your Tap.
It is recommended to use
target-bigquery with a service account.
- Download the client_secrets.json file for your service account, and place it on the machine where
target-bigquerywill be executed.
- Set a
GOOGLE_APPLICATION_CREDENTIALSenvironment variable on the machine, where the value is the fully qualified path to client_secrets.json
It should be possible to use the oAuth flow to authenticate to GCP as well:
target-bigquerywill attempt to open a new window or tab in your default browser. If this fails, copy the URL from the console and manually open it in your browser.
- If you are not already logged into your Google account, you will be prompted to log in.
- If you are logged into multiple Google accounts, you will be asked to select one account to use for the authorization.
- Click the Accept button to allow
target-bigqueryto access your Google BigQuery table.
- You can close the tab after the signup flow is complete.
The data will be written to the table specified in your
Copyright © 2018 RealSelf, Inc.