Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Support individual query tag configuration for Seeds and Snapshots #48

Merged
merged 6 commits into from Nov 19, 2021

Conversation

anthu
Copy link
Contributor

@anthu anthu commented Nov 15, 2021

resolves #20

Description

This Pull Request wraps the default materialisations of Seed and Snapshot with query tags.

Configuration examples

Configuration is equal to other materialisations:

Snapshots

snapshots/snip.sql

{% snapshot snip %}
{{
  config(
    target_database='db',
    target_schema='snapshots',
    unique_key='id',
    strategy='check',
    check_cols=['col_a'],
    query_tag='SNAPSHOT'
  )
}}

select * from {{ source('db', 'table') }}
{% endsnapshot %}

(Taken from issue)

Seeds

dbt_project.yml

seeds:
  jaffle_shop:
    +query_tag: 'dbt_jaffle_seeding'

seeds/marketing/properties.yml

version: 2

seeds:
  - name: utm_parameters
    config:
      schema: seed_data
      query_tag: 'dbt_utm_seeding'

Todo

  • I have a test in mind:
  1. Run Snapshot with Query Tag
  2. Validate queries have been executed using the query_tag
select *
from table(information_schema.query_history_by_user())
where QUERY_TAG <> '';

(3. Query tag should be reset)

  • CHANGELOG.md incoming

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-snowflake next" section.

@cla-bot cla-bot bot added the cla:yes label Nov 15, 2021
@anthu anthu changed the title [WIP] Feature/query tags for seed snapshot Fix: Support individual query tag configuration for Seeds and Snapshots Nov 16, 2021
@anthu
Copy link
Contributor Author

anthu commented Nov 16, 2021

Ready for Review

@jtcohen6
Copy link
Contributor

jtcohen6 commented Nov 17, 2021

Closing and re-opening to trigger adapter integration tests, since I've added ok to test label

Edit: There are some unrelated test failures, related to recent changes to dbt-core. We'll work to get those sorted shortly

@jtcohen6 jtcohen6 closed this Nov 17, 2021
@jtcohen6 jtcohen6 reopened this Nov 17, 2021
Copy link
Contributor

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anthu This looks great — cleverly done, and nice tests to boot. Thanks for the contribution!

@jtcohen6 jtcohen6 merged commit 9194552 into dbt-labs:main Nov 19, 2021
leahwicz added a commit that referenced this pull request Dec 3, 2021
* Slack alerts for failed nightly tests (#50)

* Fix: Support individual query tag configuration for Seeds and Snapshots (#48)

* Add Query Tag for Seed

* Add Query Tag for Snapshots

* Add Changelog for #48

* Add Snapshot with Query Tag test

* Add Seed with Query Tag test

* Update connections.py (#49)

* Update connections.py

Resolves issues when the Snowflake OCSP server is not reachable and the dbt user would like to run commands without checking the OCSP server. OCSP failures/inaccessibilitiy are possible due to network routing issues or when corporate network security has exposed Snowflake (especially in privatelink) but not the OCSP server.

* Update CHANGELOG.md

Adding changelog information

* Update CHANGELOG.md

Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com>

* Update changelog and unit test

Updated changelog and unit tests as requested.

* Fix typo

Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>

* Add column comments to Snowflake views (#53)

* Add column comments to Snowflake views
* Update create_view_as macro to include column comments
* Add test for column-level view comments
* Add testing for column comments with non-lowercase column names

* Organizing adapter macros for column persistence

* Refactor query tag tests (#57)

* Refactor query tag tests

* Try this

* Try storing failures, too

* Keep trying

* Try it this way

* Bumping version to 1.0.0rc2 (#56)

* Bumping version to 1.0.0rc2

* Update changelog

Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>

Co-authored-by: Anton Huck <anthu@users.noreply.github.com>
Co-authored-by: R. Joshua Huntley <JoshuaHuntley@users.noreply.github.com>
Co-authored-by: Jeremy Cohen <jtcohen6@gmail.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Co-authored-by: Spencer Taylor <85514383+spencer-taylor-workrise@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
@anthu anthu deleted the feature/query_tags_for_seed_snapshot branch December 6, 2021 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow query_tag configuration in snapshot model for Snowflake
2 participants