Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert test to data_test #9201

Merged
merged 15 commits into from
Dec 7, 2023

Conversation

emmyoop
Copy link
Member

@emmyoop emmyoop commented Dec 4, 2023

resolves #8699

Problem

With the addition of unit_tests the config for tests is confusing as it currently does not include unit_tests, only data_tests.

Solution

Explicitly define data_tests as data_tests in both schema files and in dbt_project.yml.

Also updated unit-tests to be unit_tests in dbt_project.yml for consistency.

Continue to support the tests config for backwards compatibility but add a deprecation warning for it.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
  • This PR includes type annotations for new and modified functions

@cla-bot cla-bot bot added the cla:yes label Dec 4, 2023
Copy link
Contributor

github-actions bot commented Dec 4, 2023

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

Copy link

codecov bot commented Dec 4, 2023

Codecov Report

Attention: 32 lines in your changes are missing coverage. Please review.

Comparison is base (ca82f54) 86.89% compared to head (4613425) 64.85%.

Files Patch % Lines
core/dbt/contracts/files.py 61.11% 7 Missing ⚠️
core/dbt/contracts/graph/nodes.py 75.00% 5 Missing ⚠️
core/dbt/exceptions.py 33.33% 4 Missing ⚠️
core/dbt/parser/generic_test_builders.py 63.63% 4 Missing ⚠️
core/dbt/parser/schemas.py 70.00% 3 Missing ⚠️
core/dbt/contracts/project.py 75.00% 2 Missing ⚠️
core/dbt/parser/common.py 80.00% 2 Missing ⚠️
core/dbt/parser/schema_generic_tests.py 71.42% 2 Missing ⚠️
core/dbt/task/test.py 33.33% 2 Missing ⚠️
core/dbt/parser/sources.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                       Coverage Diff                        @@
##           unit_testing_feature_branch    #9201       +/-   ##
================================================================
- Coverage                        86.89%   64.85%   -22.05%     
================================================================
  Files                              180      180               
  Lines                            27127    27171       +44     
================================================================
- Hits                             23572    17621     -5951     
- Misses                            3555     9550     +5995     
Flag Coverage Δ
integration ?
unit 64.85% <73.33%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@emmyoop emmyoop marked this pull request as ready for review December 5, 2023 19:47
@emmyoop emmyoop requested review from a team as code owners December 5, 2023 19:47
@emmyoop emmyoop requested review from ashnm, McKnight-42, ChenyuLInx, gshank and MichelleArk and removed request for a team, McKnight-42 and ChenyuLInx December 5, 2023 19:47
@gshank
Copy link
Contributor

gshank commented Dec 5, 2023

@graciegoheen The ticket that's referenced is contradictory about whether the config key in yaml should be "data-tests" or "data_tests". Which do you want in schema.yml?

@graciegoheen
Copy link
Contributor

graciegoheen commented Dec 5, 2023

@gshank My understanding was that it had to have _ in the schema yml files and - in the dbt_project.yml file (similar to how we handle semantic models - semantic_models in schema yml, semantic-models in dbt_project.yml).

Let me know if that understanding was incorrect.

@emmyoop
Copy link
Member Author

emmyoop commented Dec 5, 2023

Using dash in the dbt_project.yml and underscore in the schema files is also the pattern used elsewhere. semantic-models are another config we uses dashes in one place and underscores in another. It was also decided to do this for unit-tests as implemented in #8988

@graciegoheen
Copy link
Contributor

@emmyoop To confirm for our docs - we're using underscores for data_tests and unit_tests everywhere?

@emmyoop
Copy link
Member Author

emmyoop commented Dec 7, 2023

@graciegoheen yes! Underscores everywhere for unit_tests and data_tests!

Nothing else was modified that's called out in #9235.

@emmyoop emmyoop merged commit a570a2c into unit_testing_feature_branch Dec 7, 2023
48 of 49 checks passed
@emmyoop emmyoop deleted the er/8698-test-config-block branch December 7, 2023 14:03
gshank added a commit that referenced this pull request Jan 16, 2024
* Initial implementation of unit testing (from pr #2911)

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>

* 8295 unit testing artifacts (#8477)

* unit test config: tags & meta (#8565)

* Add additional functional test for unit testing selection, artifacts, etc (#8639)

* Enable inline csv format in unit testing (#8743)

* Support unit testing incremental models (#8891)

* update unit test key: unit -> unit-tests (#8988)


* convert to use unit test name at top level key (#8966)

* csv file fixtures (#9044)

* Unit test support for `state:modified` and `--defer` (#9032)

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>

* Allow use of sources as unit testing inputs (#9059)

* Use daff for diff formatting in unit testing (#8984)

* Fix #8652: Use seed file from disk for unit testing if rows not specified in YAML config (#9064)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
Fix #8652: Use seed value if rows not specified

* Move unit testing to test and build commands (#9108)

* Enable unit testing in non-root packages (#9184)

* convert test to data_test (#9201)

* Make fixtures files full-fledged members of manifest and enable partial parsing (#9225)

* In build command run unit tests before models (#9273)

---------

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>
Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants