Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster table if key_properties defined #6

Closed
wants to merge 3 commits into from

Conversation

judahrand
Copy link
Collaborator

Problem

Using unclustered tables makes MERGE statements very expensive in BigQuery as they require full table scans regardless on the data to be merged. DBT has found the same thing: dbt-labs/dbt-core#2196

Proposed changes

Cluster tables when table-key-properties is defined.

Types of changes

What types of changes does your code introduce to target-bigquery?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

  • Description above provides context of the change
  • I have added tests that prove my fix is effective or that my feature works
  • Unit tests for changes (not needed for documentation changes)
  • CI checks pass with my changes
  • Bumping version in setup.py is an individual PR and not mixed with feature or bugfix PRs
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions

@judahrand judahrand temporarily deployed to Integrate Pull Request February 23, 2022 21:00 Inactive
@judahrand judahrand temporarily deployed to Integrate Pull Request February 23, 2022 21:00 Inactive
@judahrand judahrand temporarily deployed to Integrate Pull Request February 23, 2022 21:00 Inactive
@judahrand judahrand changed the title Cluster table if table-key-properties defined Cluster table if key_properties defined Feb 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant