PR #49 includes the following updates:
- Adds passthrough column support for
USERandORGANIZATION.- Using the new
zendesk__user_passthrough_columnsandzendesk__organization_passthrough_columnsvariables, you can include custom columns from these source tables in their respective staging models. See README for more details on how to configure.
- Using the new
- Also updated the format of the pre-existing
TICKETpassthrough column variable,zendesk__ticket_passthrough_columns, to align with the newly added passthrough variables delineated above.- Previously, you could only provide a list of custom fields to be included in
stg_zendesk__ticket. Now, you have the option to provide analiasandtransform_sqlclause to be applied to each field (see README for more details). - Note: the package is and will continue to be backwards compatible with the old list-format.
- Previously, you could only provide a list of custom fields to be included in
PR #48 includes the following updates:
- Adds the
phonefield tostg_zendesk__userand ensures it is astringif the column is not found in your source data. - Adds documentation for
userfields that were previously missing yml descriptions.
PR #46 includes the following updates:
- Updated the following staging models to leverage the
{{ dbt.type_timestamp() }}macro on timestamp fields in order to ensure timestamp with no timezone is used in downstream models. This update will cause timestamps to be converted to have no timezone. If records were reported as timezone timestamps before, this will result in converted timestamp records.stg_zendesk__ticketstg_zendesk__ticket_commentstg_zendesk__ticket_field_historystg_zendesk__ticket_form_historystg_zendesk__ticket_schedulestg_zendesk__user
- Updated "Zendesk" references within the README to now refer to "Zendesk Support" in order to more accurately reflect the name of the Fivetran Zendesk Support Connector.
PR #43 introduces the following updates:
- Added the
internal_user_criteriavariable, which can be used to mark internal users whoseUSER.rolemay have changed fromagenttoend-userafter they left your organization. This variable accepts SQL that may reference any non-custom field inUSER, and it will be wrapped in acase whenstatement in thestg_zendesk__usermodel.- Example usage:
# dbt_project.yml
vars:
zendesk_source:
internal_user_criteria: "lower(email) like '%@fivetran.com' or external_id = '12345' or name in ('Garrett', 'Alfredo')" # can reference any non-custom field in USER- Output: In
stg_zendesk__user, users who match your criteria and have a role ofend-userwill have their role switched toagent. This will ensure that downstream SLA metrics are appropriately calculated.
- Updated the way we dynamically disable sources. Previously, we used a custom
meta.is_enabledflag, but, since we added this, dbt-core introduced a nativeconfig.enabledattribute. We have opted to use the dbt-native config instead. - Updated the pull request templates.
- Included auto-releaser GitHub Actions workflow to automate future releases.
PR #42 introduces the following updates:
- We have changed the identifier logic in
src_zendesk.ymlto account forgroupbeing both a Snowflake reserved word and a source table. Snowflake users will want to execute adbt run --full-refreshbefore using the new version of the package.
- Updated our
tmpmodels to utilize thedbt_utils.starmacro rather than the select * function. This removes Snowflake issues that arise when a source's dimensions change.
- Updates to the seed files and seed file configurations for the package integration tests to ensure updates are properly tested.
- Adding the
schedule_holidaysource table so that downstream models that involve business minutes calculations will accurately take holiday time into account. This staging model may be disabled by settingusing_schedulesto false. (#92)
- Incorporated the new
fivetran_utils.drop_schemas_automationmacro into the end of each Buildkite integration test job. (#37) - Updated the pull request templates. (#37)
- Updated the dbt-utils dispatch within the
stg_zendesk__ticket_schedule_tmpmodel to properly dispatchdbtas opposed todbt_utilsfor the cross-db-macros. (#32)
PR #31 includes the following breaking changes:
- Dispatch update for dbt-utils to dbt-core cross-db macros migration. Specifically
{{ dbt_utils.<macro> }}have been updated to{{ dbt.<macro> }}for the below macros:any_valuebool_orcast_bool_to_textconcatdate_truncdateadddatediffescape_single_quotesexcepthashintersectlast_daylengthlistaggpositionreplacerightsafe_castsplit_partstring_literaltype_biginttype_floattype_inttype_numerictype_stringtype_timestamparray_appendarray_concatarray_construct
- For
current_timestampandcurrent_timestamp_in_utcmacros, the dispatch AND the macro names have been updated to the below, respectively:dbt.current_timestamp_backcompatdbt.current_timestamp_in_utc_backcompat
- Dependencies on
fivetran/fivetran_utilshave been upgraded, previously[">=0.3.0", "<0.4.0"]now[">=0.4.0", "<0.5.0"].
🚨 This includes Breaking Changes! 🚨
- Updated README documentation for easier navigation and dbt package setup (#28).
- Included the
zendesk_[source_table_name]_identifiervariables for easier flexibility of the package models to refer to differently named sources tables (#28). - Databricks compatibility 🧱 (#29)
- By default, this package now builds the Zendesk staging models within a schema titled (
<target_schema>+_zendesk_source) in your target database. This was previously<target_schema>+_zendesk_staging, but we have changed it to maintain consistency with our other packges. See the README for instructions on how to configure the build schema differently.
- The
stg_zendesk__tickettable now allows for your custom passthrough columns to be added via thezendesk__ticket_passthrough_columnsvariable. You can add your passthrough columns as a list within the variable in your project configuration. (#27)
- Incorporates the
daylight_timeandtime_zonesource tables into the package. In the transform package, these tables are used to more precisely calculate business hour metrics (#62).
🎉 dbt v1.0.0 Compatibility 🎉
- Adjusts the
require-dbt-versionto now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.- For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
- For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
- Upgrades the package dependency to refer to the latest
dbt_fivetran_utils. The latestdbt_fivetran_utilspackage also has a dependency ondbt_utils[">=0.8.0", "<0.9.0"].- Please note, if you are installing a version of
dbt_utilsin yourpackages.ymlthat is not in the range above then you will encounter a package dependency error.
- Please note, if you are installing a version of
- Adjusted timestamp fields within staging models to explicitly cast the data type as
timestamp without time zone. This fixes a Redshift error where downstream datediff and dateadd functions would result in an error if the timestamp fields are synced astimestamp_tz. (#23)
- @juanbriones (#55)
Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!