Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle unknown type_code for model contracts #8887

Merged
merged 5 commits into from
Oct 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixes-20231024-110151.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: Handle unknown `type_code` for model contracts
time: 2023-10-24T11:01:51.980781-06:00
custom:
Author: dbeatty10
Issue: 8877 8353
5 changes: 4 additions & 1 deletion plugins/postgres/dbt/adapters/postgres/connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,4 +204,7 @@ def get_response(cls, cursor) -> AdapterResponse:

@classmethod
def data_type_code_to_name(cls, type_code: int) -> str:
return string_types[type_code].name
if type_code in string_types:
return string_types[type_code].name
else:
return f"unknown type_code {type_code}"
Copy link
Contributor Author

@dbeatty10 dbeatty10 Oct 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rlh1994 brought up the idea of raising a debug-level warning log message in this case.

Seems like a valuable idea to explore. Maybe something like this?

            logger.debug("unknown type_code: {}", type_code)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichelleArk how would you feel about adding this ☝️ debug output when the type_code isn't found?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be a little redundant given unknown type_code will appear in the error message either way. So I'd either omit it or write a more informative message like"type_code not found in adapter-provided lookup, defaulting to 'unknown type_code'"

And we may actually need to use fire_event to fire a debuglevel event as opposed not logger.debug

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Splitting this into its own issue:

That way we can ship this fix while still giving that debug message enough attention.

76 changes: 76 additions & 0 deletions tests/functional/contracts/test_nonstandard_data_type.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
import pytest
from dbt.tests.util import run_dbt, run_dbt_and_capture


my_numeric_model_sql = """
select
12.34 as price
"""

my_money_model_sql = """
select
cast('12.34' as money) as price
"""

model_schema_money_yml = """
models:
- name: my_model
config:
contract:
enforced: true
columns:
- name: price
data_type: money
"""

model_schema_numeric_yml = """
models:
- name: my_model
config:
contract:
enforced: true
columns:
- name: price
data_type: numeric
"""


class TestModelContractUnrecognizedTypeCode1:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_money_model_sql,
"schema.yml": model_schema_money_yml,
}

def test_nonstandard_data_type(self, project):
run_dbt(["run"], expect_pass=True)


class TestModelContractUnrecognizedTypeCodeActualMismatch:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_money_model_sql,
"schema.yml": model_schema_numeric_yml,
}

def test_nonstandard_data_type(self, project):
expected_msg = "unknown type_code 790 | DECIMAL | data type mismatch"
_, logs = run_dbt_and_capture(["run"], expect_pass=False)
assert expected_msg in logs


class TestModelContractUnrecognizedTypeCodeExpectedMismatch:
@pytest.fixture(scope="class")
def models(self):
return {
"my_model.sql": my_numeric_model_sql,
"schema.yml": model_schema_money_yml,
}

def test_nonstandard_data_type(self, project):
expected_msg = "DECIMAL | unknown type_code 790 | data type mismatch"
_, logs = run_dbt_and_capture(["run"], expect_pass=False)
print(logs)
assert expected_msg in logs