Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2997] Ability to test incremental models #8422

Closed
Tracked by #8283
gshank opened this issue Aug 16, 2023 · 4 comments
Closed
Tracked by #8283

[CT-2997] Ability to test incremental models #8422

gshank opened this issue Aug 16, 2023 · 4 comments
Assignees

Comments

@gshank
Copy link
Contributor

gshank commented Aug 16, 2023

Description

We should allow folks to add unit tests to incremental models

Acceptance criteria

  • Ability to test incremental models in both full refresh and incremental mode
  • Ability to specify the current status of an incremental model to test the incremental behavior

More on the above two points in #8275 (reply in thread):

To support this, dbt's unit testing framework would need to provide the ability to:

  • override the is_incremental macro to return True (p0), and provide the ability to specify an input fixture for {{ this }} (p1) in order to mock out the previous state of an incremental model. The expect output would be the set of rows that dbt's incremental materialization would insert/merge into the existing relation in production, as opposed to the resulting updated representation of the model.

To be moved into a separate issue:

A "current_actual" could define the current state of the model we're incrementing on, testing whether the incrementing works as expected.

@github-actions github-actions bot changed the title Ability to test incremental models [CT-2997] Ability to test incremental models Aug 16, 2023
@christopherekfeldt
Copy link

We need this! We would love to have the functionality to only test the newly added / updated data for incremental models.

@martynydbt
Copy link

martynydbt commented Sep 18, 2023

@gshank please pull "Ability to test the incremental behavior of models using incremental strategy that requires partitioning (e.g. insert overwrite in BigQuery)" out into a separate ticket

@jtcohen6
Copy link
Contributor

jtcohen6 commented Oct 2, 2023

2x2 = 4 acceptance tests for us to write as part of implementation:

  • {{ is_incremental() }} : True + False
  • {{ this }} fixture: provided or not

Discussion:

  • is_incremental would go in existing spec for overrides
  • Need to add this to spec for inputs (cc: @graciegoheen @dbeatty10)

@MichelleArk
Copy link
Contributor

Here's how i'm imagining the spec for this:

unit-tests: 
 - model: my_model
   tests: 
    - name: test_name
      given: 
      - input: this
         rows: ... 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants