Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Designing vendoring feature command on BigQuery #85

Open
takegue opened this issue Jan 15, 2023 · 0 comments · May be fixed by #140
Open

Designing vendoring feature command on BigQuery #85

takegue opened this issue Jan 15, 2023 · 0 comments · May be fixed by #140
Labels
enhancement New feature or request

Comments

@takegue
Copy link
Owner

takegue commented Jan 15, 2023

Vendoring is a feature to move all third-party resources, such as public datasets and UDFs, into a specified dataset.
This has several benefits that can help make your data modeling more efficient and reliable.

Background

In BigQuery, some useful UDF is published vid public dataset such as bqutil.fn, bqmake.v0 and so on.
However, these UDFs maintained by community doesn't gurantee permanent availability.
Additionaly, BigQuery resctrict to use UDF only in same-region and users often meet this restriction.

By using vendoring, you can ensure that the third-party resources you rely on are always available and that any updates or changes made to them do not affect your analysis. This can be especially important if you are working with sensitive or business-critical data.
Additionally, vendoring allows you to use third-party resources in different regions.

Usecase

# Vendoring all resources 
bqport import project.user_dataset [external_dataset]
# Update dependencies
bqport update-dependencies project.user_dataset

Support BigQuery Resources

  • Routine (UDF/Procedure)
  • (Optional) Table

Scope levels

  • (Optional) Control to exclude sources
    • Intra-project resource
    • White-list project
@takegue takegue added the enhancement New feature or request label Feb 1, 2023
@takegue takegue linked a pull request May 7, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant