Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] BigQuery Support #82

Closed
z3z1ma opened this issue Apr 1, 2023 · 4 comments
Closed

[feat] BigQuery Support #82

z3z1ma opened this issue Apr 1, 2023 · 4 comments

Comments

@z3z1ma
Copy link

z3z1ma commented Apr 1, 2023

It seems there are very few blockers for BigQuery support. I believe its just the SQL generation which uses some non ANSI casting syntax ::.

Congrats on the 3.0 upgrade, looks nice!

@z3z1ma
Copy link
Author

z3z1ma commented Apr 1, 2023

Oh actually this seems to be the bottleneck:
https://github.com/GClunies/Reflekt/blob/main/reflekt/warehouse.py#L89

I would consider, specifically for dbt users at least since I don't see Warehouse used anywhere else -- we piggyback off of existing dbt infra using something like dbt-core-interface which can be vendored.

@GClunies
Copy link
Owner

GClunies commented Apr 7, 2023

@z3z1ma Yup, agree this should not be too hard. I've never used BQ so that's the main blocker! 😆 Open to help here!

dbt-core-interface looks interesting, but its not clear to me how it helps here? Can you elaborate for me? Currently, Warehouse is used in dbt.py so that the DbtBuilder class can query the DWH for a matching table and columns as it parses each schema.

Note - in the future, reflekt could build other types of data artifacts (e.g., Looker views, etc). The end user might not use dbt at all in this scenario.

@GClunies
Copy link
Owner

GClunies commented Mar 5, 2024

Addressed in #105. FYI @z3z1ma

@GClunies GClunies closed this as completed Mar 5, 2024
@z3z1ma
Copy link
Author

z3z1ma commented Mar 10, 2024

Awesome @GClunies 💯 I will check it out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants