bqup is a backup tool for BigQuery projects. It can export a BigQuery project's structure and source code while mimicking the hierarchy of datasets and tables.
For the full story of why we made bqup, check out our blog post!
bqup can be installed using
pip install bqup
Alternatively, you can also clone the repository then run
git clone https://github.com/thinkingmachines/bqup.git cd bqup python3 setup.py install
Command line options
You can list down the options by running
bqup [-p PROJECT_ID] [-d TARGET_DIR] [-fvxr] [-e REGEX] Options: -p PROJECT_ID, --project PROJECT_ID Project ID to load. If unspecified, defaults to current project in configuration. -d TARGET_DIR, --dir TARGET_DIR The target directory where the project will be written. Defaults to current timestamp. -f --force Overwrite target directory if it exists. -v --verbose Print a summary of the loaded project. -x --schema Export table schemata as json. -r --routine Include routines in export. -e REGEX, --regex REGEX Regex pattern to filter datasets to be exported.
gcloudto run with your personal account (aka run with scissors).
Set up application-default.
gcloud auth application-default login
pip3 install wheel
pip3 install -e .
Alternatively, you can also install it using:
python3 setup.py develop
Run bqup (see Usage).
Setting up regular backups
Check out bqup-starter to set up regular bqups using GitHub's workflows!
make test to try a test upload.
make dist to upload a distribution.
Both of these will call
make build, which rebuilds the package locally.
If you wish to contribute, check out our contributing guide!
A list is maintained for all external contributors who have submitted pull requests that were subsequently approved. Users are allowed and encouraged to fork the project and submit pull requests and issues. We only request that contributions adhere to these guidelines:
The official maintainers in charge of responding to issues and merging pull requests are:
Thanks to all these wonderful people who've helped out with bqup:
bqup is maintained on a best effort basis:
- No amount of official time is currently being dedicated to the regular maintenance of this project.
- Thinking Machines does not make any guarantees about the quality of the software.
Thinking Machines reserves the rights to:
- refuse to resolve issues
- close issues without resolution
- request changes to pull requests
- reject pull requests outright