-
Notifications
You must be signed in to change notification settings - Fork 496
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve the performance of the at_least_one test #776
Conversation
Fix ambiguous col issue
ensure case match
The failing Redshift test is unrelated to these changes. The at_least_one tests in Redshift succeed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes a ton of sense! Thanks @JoshuaHuntley 🎉
A couple of suggested changes - one that is a deferral to a different changeset (if at all... I'm not totally sold yet) and one change from a subquery to a CTE if possible.
@joellabes I made the requested changes. FYI, I'm firmly in favor of CTEs but stuck to the original approach as much as possible on the first go-round. I've flattened the code out a bit so there is now only one layer of nesting. I see your position about the all-or-nothing approach to error handling. Pushing it down to the user, especially considering our intended user base, is fair. |
Thanks @JoshuaHuntley! From the CI logs, it looks like you might have to get rid of the test that checks for the tested column being in the group_by columns as well. |
@joellabes I removed the test. There are Redshift adapter tests that are not related to this PR. Should I remove them or can we ignore them? |
@JoshuaHuntley I'm @joellabes's colleague and I was asked to lend a hand on this review. I pushed a couple commits to do the following:
The tests are all passing now ✅, so I think we'll be ready to merge soon. DetailsCase-sensitive group by testThe group by test was failing when the column to test also was included in the list of columns to group by. So I added some case-sensitive logic that checks whether it is in that list or not. Note that other macros like So users might still get an error like - name: data_test_at_least_one
columns:
- name: field
tests:
- dbt_utils.at_least_one
- dbt_utils.at_least_one:
group_by_columns: ['FIELD'] Currently, I'm more comfortable with that possibility than attempting to handle all the details and complications of a case-insensitive check. We can always update the implementation later if case-insensitive comparisons are needed.
|
resolves #775
[This is a:
All pull requests from community contributors should target the
main
branch (default).Description & motivation
Checklist
star()
source)limit_zero()
macro in place of the literal string:limit 0
dbt.type_*
macros instead of explicit datatypes (e.g.dbt.type_timestamp()
instead ofTIMESTAMP
](https://github.com/JoshuaHuntley)