Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

t-sql hacks that shouldn't break mainline behavior #310

Merged
merged 10 commits into from Jan 11, 2021
Merged

t-sql hacks that shouldn't break mainline behavior #310

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
418278c
t-sql hacks that shouldn't break mainline behavior
dataders Dec 23, 2020
d8e22ba
make tsql compatible
dataders Jan 8, 2021
26b218e
hail mary
dataders Jan 8, 2021
6172774
TSQL does not support 'limit 0'
dataders Jan 8, 2021
9edb1b1
return result instead of function
dataders Jan 8, 2021
1ab4972
add limit_zero to namespace
dataders Jan 8, 2021
97c4e1e
Merge branch 'master' of https://github.com/fishtown-analytics/dbt-ut…
dataders Jan 8, 2021
44ac75e
document TSQL quirks
dataders Jan 8, 2021
3486ff1
Merge branch 'master' of https://github.com/fishtown-analytics/dbt-ut…
dataders Jan 10, 2021
c84201f
posterity
dataders Jan 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion integration_tests/tests/assert_get_query_results_as_dict_objects_equal.sql
View file
Open in desktop
Expand Up @@ -77,5 +77,5 @@ Instead, we'll manually check that the values of these dictionaries are equivale
{% endif %}

{{ log(ns.err_msg, info=True) }}
select 1 {% if ns.pass %} limit 0 {% endif %}
select 1 as col_name {% if ns.pass %} where 0=1 {% endif %}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch. looks at if TSQL and BQ don't play nice together. From CircleCI

Query without FROM clause cannot have a WHERE clause

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gross. Perhaps we do need a dispatched macro {{ limit_zero() }} after all, just for the sake of integration tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where would limit_zero() live? as a macro just for integration tests, integration_tests/macros? could it be properly dispatched there?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it could live in integration_tests/macros, and look like:

{% macro limit_zero() %}
    {{ return(adapter.dispatch('limit_zero', dbt_utils._get_utils_namespaces())()) }}
{% endmacro %}

{% macro default__limit_zero() %}
    {{ return('limit 10') }}
{% endmacro %}

i.e. It dispatches via the dbt_utils_dispatch_list var for namespace purposes, but it doesn't actually "live" in / ship with the dbt_utils package.

Then, you should be able to define:

{% macro sqlserver__limit_zero() %}
  {{ return('where 0=1') }} 
{% endmacro %}

This is the same thing as, or very similar to, what we did with prep_external over in the external tables package

{% endif %}
2 changes: 1 addition & 1 deletion integration_tests/tests/logger/assert_pretty_output_msg_is_string.sql
View file
Open in desktop
@@ -1,6 +1,6 @@
{% if dbt_utils.pretty_log_format() is string %}
{# Return 0 rows for the test to pass #}
select 1 limit 0
select 1 as col_name where 0=1
{% else %}
{# Return >0 rows for the test to fail #}
select 1
Expand Down
2 changes: 1 addition & 1 deletion integration_tests/tests/logger/assert_pretty_time_is_string.sql
View file
Open in desktop
@@ -1,6 +1,6 @@
{% if dbt_utils.pretty_time() is string %}
{# Return 0 rows for the test to pass #}
select 1 limit 0
select 1 as col_name where 0=1
{% else %}
{# Return >0 rows for the test to fail #}
select 1
Expand Down
5 changes: 3 additions & 2 deletions macros/schema_tests/at_least_one.sql
View file
Open in desktop
Expand Up @@ -5,8 +5,9 @@
select count(*)
from (
select

count({{ column_name }})
{# subquery aggregate columns need aliases #}
{# thus: 'unique_values' #}
count({{ column_name }}) as unique_values

from {{ model }}

Expand Down
7 changes: 4 additions & 3 deletions macros/schema_tests/cardinality_equality.sql
View file
Open in desktop
@@ -1,5 +1,6 @@
{% macro test_cardinality_equality(model, to, field) %}

{# T-SQL doesn't let you use numbers as aliases for columns #}
{# Thus, no "GROUP BY 1" #}
{% set column_name = kwargs.get('column_name', kwargs.get('from')) %}


Expand All @@ -8,15 +9,15 @@ select
{{ column_name }},
count(*) as num_rows
from {{ model }}
group by 1
group by {{ column_name }}
),

table_b as (
select
{{ field }},
count(*) as num_rows
from {{ to }}
group by 1
group by {{ column_name }}
),

except_a as (
Expand Down
5 changes: 3 additions & 2 deletions macros/schema_tests/expression_is_true.sql
View file
Open in desktop
@@ -1,5 +1,6 @@
{% macro test_expression_is_true(model, condition='true') %}

{% macro test_expression_is_true(model, condition='1=1') %}
{# T-SQL has no boolean data type so we use 1=1 which returns TRUE #}
{# ref https://stackoverflow.com/a/7170753/3842610 #}
{% set expression = kwargs.get('expression', kwargs.get('arg')) %}

with meet_condition as (
Expand Down
2 changes: 1 addition & 1 deletion macros/schema_tests/not_constant.sql
View file
Open in desktop
Expand Up @@ -8,7 +8,7 @@ select count(*)
from (

select
count(distinct {{ column_name }})
count(distinct {{ column_name }}) as filler_column

from {{ model }}

Expand Down
4 changes: 2 additions & 2 deletions macros/schema_tests/relationships_where.sql
View file
Open in desktop
@@ -1,8 +1,8 @@
{% macro test_relationships_where(model, to, field) %}

{% set column_name = kwargs.get('column_name', kwargs.get('from')) %}
{% set from_condition = kwargs.get('from_condition', "true") %}
{% set to_condition = kwargs.get('to_condition', "true") %}
{% set from_condition = kwargs.get('from_condition', "1=1") %}
{% set to_condition = kwargs.get('to_condition', "1=1") %}

with left_table as (

Expand Down