snowplow-utils v0.16.4

Summary

This release adds more flexibility to the exclude_versions argument of the combine_column_versions macro for BigQuery to allow removal of any column with the suffix of the specified version.

Fixes

• Remove hardcoded structure requirement to exclude column versions.

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.16.3

Summary

This releases adds the database argument to the get_sde_or_context macro.

Features

• Add database argument to get_sde_or_context macro

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.16.2

Summary

This releases adds two new sets of macros, one is parse_agg_dict that generates aggregation sql, and the other is apply_grants which allows the usage of the snowplow__grant_select_to variable to grant select on models to any user/role in that list (does not support BigQuery). We also fix a rare edge case where custom user identifiers made from sql could lead to duplicate sessions in the lifecycle table.

Features

• Add new parse_agg_dict macro for use to generate aggregation sql in other packages
• Overwrite default dbt apply_grants macro to enable using a variable to define grant targets
• Add new default__grant_usage_on_schemas_built_into macro to add as a post-hook in package to grant usage for schemas
• Ensure only one user id even when a custom user sql is provided for the lifecycles manifest table

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.16.1

Summary

This release contains a fix to the unnest macro for redshift, which meant the query wasn't complete when using it without the index.

Fixes

• Fix bug in redshift unnest macro

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.16.0

Summary

This release contains a new array size macro as well as the optional ability for the unnest macros to also surface the index of the original array element for preserving the order. It also contains some fixes for redshift. Please note that from this version onwards this package is under the SPAL license.

Features

• Add get_array_size macro
• Add optional index to unnest

Fixes

• Fix existing tests for redshift
• Fix issue with multiple end hooks failing (Close #152)

Under the hood

• Update license to SPAL

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.15.2

Summary

This release contains a bug fix in our new base macros, which did not affect default behaviour. This also does not impact behavior in any of the Snowplow packages that rely on the utils package.

Fixes

• Fix an incorrect default value in the base_create_snowplow_events_this_run macro.

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.15.1

Summary

This release combines a few bug fixes in our new base macros, including an issue with redshift query size limits.

Fixes

• Fix issue with 8001 redshift error
• Add condition for late loading events to fix issue where session events before session start timestamp were processed
• Fix the manifest table not using the specified session timestamp field
• Provide correct error message when no user identifier is present

Under the hood

• Fix some issues in our integration tests

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.15.0

Summary

This full release introduces new "base" macros which are used to recreate the snowplow_base_events_this_run and related tables. It gives you the option to customise your session and user identifiers, your partitioning timestamps, and much more. Additionally, we remove the deprecated snowplow_incremental materialization that was removed in 0.14.0, and we add multi-version support for the get_field macro for BigQuery.

We have documentation explaining the outlined changes in detail here that you can use to explore the base macro changes in particular. For information about how to do this, see our Advanced Usage of the Utils package page and the utils configuration variables.

Finally, before we dig into the details, starting with this version all future releases of our dbt packages will be released under the Snowplow Community License instead of the Apache 2 license. For what this means for you, check out our FAQ, and our announcement blog.

Features

• Create the "base" macros to enable greater flexibility
• Update the package to be under the Snowplow Community License
• Released a fix where get_optional_fields returned field aliases even when include_field_alias was set to false (#132)
• Add multi-version support to the get_field macro for BigQuery
• Remove deprecated snowplow_incremental materialization

Under the hood

• Introduce integration tests to check customization

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.15.0-rc1

Summary

This is a pre-release version of the package, we believe it to be in working condition but you may encounter bugs and some features may change before the final release.

This release introduces new "base" macros which are used to recreate the snowplow_base_events_this_run and related tables. It gives you the option to customise your session and user identifiers, your partitioning timestamps, and much more. We have temporary docs hosted here that you can use to explore docs related to these changes. Once this is fully released, these docs will migrate over to the offical docs site

For information about how to do this, see our Advanced Usage of the Utils package page and the utils configuration variables.

Finally, before we dig into the details, starting with this version all future releases of our dbt packages will be released under the Snowplow Community License instead of the Apache 2 license. For what this means for you, check out our FAQ, and our announcement blog.

Features

• Create the "base" macros to enable greater flexibility
• Update the package to be under the Snowplow Community License
• Released a fix where get_optional_fields returned field aliases even when include_field_alias was set to false (#132)

Under the hood

• Introduce integration tests to check customization

Upgrading

To upgrade, bump the package version in your packages.yml file.

snowplow-utils v0.14.3

Summary

This version adds the ability to specify which package is returning the run limits in the print_run_limits macro, and fixes an issue with using string agg in Databricks with many null values.

Features

• Add ability to tag package in limits (Close #133)

Fixes

• Fix issue with large volume of nulls in databricks string agg

Upgrading

To upgrade bump the package version in your packages.yml file.