Skip to content

Fix!: Type inference for BQ engine adapter#3981

Merged
VaggelisD merged 3 commits intomainfrom
vaggelisd/bq_types
Mar 13, 2025
Merged

Fix!: Type inference for BQ engine adapter#3981
VaggelisD merged 3 commits intomainfrom
vaggelisd/bq_types

Conversation

@VaggelisD
Copy link
Contributor

@VaggelisD VaggelisD commented Mar 12, 2025

Fixes #3978

In the BigQuery engine adapter, type inference for the fields happens through to_standard_sql(); However, it seems like the types that do not belong to stanard SQL are not handled properly (e.g RANGE or JSON, see 4951ff9) and are instead marked as TYPE_KIND_UNSPECIFIED.

This PR adds a fallback for these types and instead uses field.field_type, as was used before:

MODEL (name vaggelis.test);

SELECT 
 RANGE('01-01-1900'::DATE, '01-01-1902'::DATE) AS my_range, 
 JSON '{"id": 10}' AS foo;
>>> bq_adapter.columns("vaggelis.test");
{'my_range': DataType(
  this=Type.RANGE,
  expressions=[
    DataType(this=Type.DATE, nested=False)],
  nested=True), 'foo': DataType(this=Type.JSON, nested=False)}

Copy link
Contributor

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's test these changes.

@VaggelisD VaggelisD merged commit f4cb59b into main Mar 13, 2025
22 checks passed
@VaggelisD VaggelisD deleted the vaggelisd/bq_types branch March 13, 2025 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigQuery RANGE type is inferred to be JSON in different places

3 participants