Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using dbt query_comment breaks project registration #31

Closed
dh-racheldean opened this issue Nov 24, 2022 · 1 comment
Closed

Using dbt query_comment breaks project registration #31

dh-racheldean opened this issue Nov 24, 2022 · 1 comment

Comments

@dh-racheldean
Copy link

Symptoms:
My dbt installation uses the query-comment config to add labels to BigQuery queries. In our dbt_project.yml we have

query-comment:
  comment: "{{ query_comment(node) }}"
  job-label: true

When trying to run a dbt-osmosis server serve...., the server throws a 500 Could not connect to Database error.

Diagnosis:
Trailing things through the code, I reached parse_project in osmosis.py. This inits the DB adapter, loads the manifest, and importantly calls save_macros_to_adapter. BUT... before we get to the save macros, the adapter setter fn calls _verify_connection which tries to query the DB before the adapter is fully initialised...

Now the adapter errors in this chunk in the dbt-bigquery connection

if (
            hasattr(self.profile, "query_comment")
            and self.profile.query_comment
            and self.profile.query_comment.job_label
        ):
            query_comment = self.query_header.comment.query_comment. 
            labels = self._labels_from_query_comment(query_comment)

failing with 'NoneType' object has no attribute 'comment' on line 6 of the snippet.
Now this code looks ugly - it's checking the contents of self.profile, then trying to access self.query-header - clearly a massive assumption is made here...

So - we have a conflict. dbt-osmosis is trying to run a query without ensuring the adapter is completely configured - and dbt-bigquery is doing a really wonky test. I'm happy to PR a deferred verify process in dbt-osmosis, or if you consider this to be more of a dbt-bigquery bug, I'll raise it with them.

@z3z1ma
Copy link
Owner

z3z1ma commented Mar 26, 2023

This looks to be resolved in latest versions. I use bigquery myself and tested this. I think the migration to the vendored dbt-core-interface smoothed out adapter management.

@z3z1ma z3z1ma closed this as completed Mar 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants