Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new macro to test numeric types with specific precision and scale #263

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ For example, use `America/New_York` for East Coast Time.
- [expect_column_values_to_not_be_null](#expect_column_values_to_not_be_null)
- [expect_column_values_to_be_unique](#expect_column_values_to_be_unique)
- [expect_column_values_to_be_of_type](#expect_column_values_to_be_of_type)
- [expect_column_values_to_be_of_numeric_precision_and_scale](#expect_column_values_to_be_of_numeric_precision_and_scale)
- [expect_column_values_to_be_in_type_list](#expect_column_values_to_be_in_type_list)
- [expect_column_values_to_have_consistent_casing](#expect_column_values_to_have_consistent_casing)

Expand Down Expand Up @@ -505,6 +506,20 @@ tests:
column_type: date
```

### [expect_column_values_to_be_of_numeric_precision_and_scale](macros/schema_tests/column_values_basic/expect_column_values_to_be_of_numeric_precision_and_scale.sql)

Expect a column to be of type numeric with specific precision and scale.

*Applies to:* Column

```yaml
tests:
- dbt_expectations.expect_column_values_to_be_of_numeric_precision_and_scale:
column_type: 'numeric'
column_precision: 30
column_scale: 4
```

### [expect_column_values_to_be_in_type_list](macros/schema_tests/column_values_basic/expect_column_values_to_be_in_type_list.sql)

Expect a column to be one of a specified type list.
Expand Down
4 changes: 4 additions & 0 deletions integration_tests/models/schema_tests/data_test.sql
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ select
'2020-10-21' as date_col,
cast(0 as {{ dbt.type_float() }}) as col_numeric_a,
cast(1 as {{ dbt.type_float() }}) as col_numeric_b,
cast(1.1 as {{ dbt_expectations.type_numeric(2,1) }}) as col_numeric_c,
'a' as col_string_a,
'b' as col_string_b,
cast(null as {{ dbt.type_string() }}) as col_null,
Expand All @@ -15,6 +16,7 @@ select
'2020-10-22' as date_col,
1 as col_numeric_a,
0 as col_numeric_b,
cast(1.1 as {{ dbt_expectations.type_numeric(2,1) }}) as col_numeric_c,
'b' as col_string_a,
'ab' as col_string_b,
null as col_null,
Expand All @@ -27,6 +29,7 @@ select
'2020-10-23' as date_col,
0.5 as col_numeric_a,
0.5 as col_numeric_b,
cast(0.5 as {{ dbt_expectations.type_numeric(2,1) }}) as col_numeric_c,
'c' as col_string_a,
'abc' as col_string_b,
null as col_null,
Expand All @@ -39,6 +42,7 @@ select
'2020-10-23' as date_col,
0.5 as col_numeric_a,
0.5 as col_numeric_b,
cast(0.5 as {{ dbt_expectations.type_numeric(2,1) }}) as col_numeric_c,
'c' as col_string_a,
'abcd' as col_string_b,
null as col_null,
Expand Down
15 changes: 11 additions & 4 deletions integration_tests/models/schema_tests/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ models:
row_condition: 1=1
compare_row_condition: 1=1
- dbt_expectations.expect_table_column_count_to_equal:
value: 8
value: 9
- dbt_expectations.expect_table_column_count_to_be_between:
min_value: 1
max_value: 10
Expand All @@ -405,13 +405,13 @@ models:
- dbt_expectations.expect_table_columns_to_contain_set:
column_list: ["col_numeric_b", "col_string_a"]
- dbt_expectations.expect_table_columns_to_match_set:
column_list: ["idx", "date_col", "col_numeric_a", "col_numeric_b", "col_string_a", "col_string_b", "col_null", "col_null_2"]
column_list: ["idx", "date_col", "col_numeric_a", "col_numeric_b", "col_numeric_c", "col_string_a", "col_string_b", "col_null", "col_null_2"]
- dbt_expectations.expect_table_columns_to_match_ordered_list:
column_list: ["idx", "date_col", "col_numeric_a", "col_numeric_b", "col_string_a", "col_string_b", "col_null", "col_null_2"]
column_list: ["idx", "date_col", "col_numeric_a", "col_numeric_b", "col_numeric_c", "col_string_a", "col_string_b", "col_null", "col_null_2"]
- dbt_expectations.expect_table_column_count_to_equal_other_table:
compare_model: ref("data_test")
- dbt_expectations.expect_table_columns_to_not_contain_set:
column_list: ["col_numeric_c", "col_string_d"]
column_list: ["col_numeric_d", "col_string_d"]
- dbt_expectations.expression_is_true:
expression: (col_numeric_a + col_numeric_b = 1)
- dbt_expectations.equal_expression:
Expand Down Expand Up @@ -550,6 +550,13 @@ models:
tests:
- dbt_expectations.expect_column_values_to_be_null

- name: col_numeric_c
tests:
- dbt_expectations.expect_column_values_to_be_of_numeric_precision_and_scale:
column_type: 'numeric'
column_precision: 2
column_scale: 1

- name : data_test_factored
tests :
- dbt_expectations.expect_table_row_count_to_equal_other_table_times_factor:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{%- test expect_column_values_to_be_of_numeric_precision_and_scale(model, column_name, column_type, column_precision, column_scale) -%}
{%- if execute -%}

{%- set column_name = column_name | upper -%}
{%- set columns_in_relation = adapter.get_columns_in_relation(model) -%}

with relation_columns as (

{% for column in columns_in_relation %}
select
cast('{{ column.name | upper }}' as varchar) as relation_column,
cast('{{ column.dtype | upper }}' as varchar) as relation_column_type,
cast('{{ column.numeric_precision }}' as varchar) as relation_column_precision,
cast('{{ column.numeric_scale }}' as varchar) as relation_column_scale
{% if not loop.last %}union all{% endif %}
{% endfor %}
),
test_data as (

select
*
from
relation_columns
where
relation_column = '{{ column_name | upper }}'
and (
relation_column_type <> '{{ column_type | upper }}'
or
relation_column_precision <> '{{ column_precision }}'
or
relation_column_scale <> '{{ column_scale }}'
)

)
select *
from test_data

{%- endif -%}
{%- endtest -%}
8 changes: 8 additions & 0 deletions macros/utils/datatypes.sql
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,11 @@
{% macro postgres__type_datetime() -%}
timestamp without time zone
{%- endmacro %}

{% macro type_numeric(col_precision, col_scale) %}
{{ return(api.Column.numeric_type("numeric", col_precision, col_scale)) }}
{% endmacro %}

{% macro default__type_numeric() %}
{{ return(dbt.type_numeric()) }}
{% endmacro %}