Generate and load BigQuery tables based on Table Schema descriptors.
- implements
tableschema.Storage
interface
The package use semantic versioning. It means that major versions could include breaking changes. It's highly recommended to specify package
version range in your setup/requirements
file e.g. package>=1.0,<2.0
.
pip install tableschema-bigquery
To start using Google BigQuery service:
- Create a new project - link
- Create a service key - link
- Download json credentials and set
GOOGLE_APPLICATION_CREDENTIALS
environment variable
import io
import os
import json
from datapackage import Package
from apiclient.discovery import build
from oauth2client.client import GoogleCredentials
# Prepare BigQuery credentials
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '.credentials.json'
credentials = GoogleCredentials.get_application_default()
service = build('bigquery', 'v2', credentials=credentials)
project = json.load(io.open('.credentials.json', encoding='utf-8'))['project_id']
# Save package to BigQuery
package = Package('datapackage.json')
package.save(storage='bigquery', service=service, project=project, dataset='dataset')
# Load package from BigQuery
package = Package(storage='bigquery', service=service, project=project, dataset='dataset')
package.resources
Storage(self, service, project, dataset, prefix='')
BigQuery storage
Package implements Tabular Storage interface (see full documentation on the link):
Only additional API is documented
Arguments
- service (object): BigQuery
Service
object - project (str): BigQuery project name
- dataset (str): BigQuery dataset name
- prefix (str): prefix for all buckets
The project follows the Open Knowledge International coding standards.
Recommended way to get started is to create and activate a project virtual environment. To install package and development dependencies into active environment:
$ make install
To run tests with linting and coverage:
$ make test
Here described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted commit history.
- Initial driver realease