Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Regression] docs generate raises error when querying external tables #1079

Closed
2 tasks done
mikealfare opened this issue Jan 31, 2024 · 0 comments
Closed
2 tasks done
Assignees
Labels
backport 1.7.latest Tag for PR to be backported to the 1.7.latest branch regression support

Comments

@mikealfare
Copy link
Contributor

Is this a regression in a recent version of dbt-bigquery?

  • I believe this is a regression in dbt-bigquery functionality
  • I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

Creating a model dependent on an external table and running dbt docs generate raises this error:

File "/venv/dbt-1.7.0-latest/lib/python3.8/site-packages/dbt/task/generate.py", line 106, in add_column
    column_data["index"] = int(column_data["index"])
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Expected/Previous Behavior

No error should be raised and the docs should include this source, but with unknown columns, as was previously the case in 1.7.2

Steps To Reproduce

  1. Create a model dependent on an external table
  2. Run dbt docs generate

Relevant log output

No response

Environment

- OS: dbt Cloud
- Python: 3.8
- dbt-core (working version): 1.7
- dbt-bigquery (working version): 1.7.2
- dbt-core (regression version): 1.7
- dbt-bigquery (regression version): 1.7.3

Additional Context

I think we need to coalesce here:

columns.column_name,
columns.column_index,
columns.column_type,

to match (note the comment):

-- coalesce name and type for External tables - these columns are not
-- present in the COLUMN_FIELD_PATHS resultset
coalesce(columns.column_name, '<unknown>') as column_name,
-- invent a row number to account for nested fields -- BQ does
-- not treat these nested properties as independent fields
row_number() over (
partition by relation_id
order by columns.column_index, columns.column_name
) as column_index,
coalesce(columns.column_type, '<unknown>') as column_type,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.7.latest Tag for PR to be backported to the 1.7.latest branch regression support
Projects
None yet
Development

No branches or pull requests

1 participant