partition fix

[fivetran-jamie]

Are you a current Fivetran customer?

work at fivetran

What change(s) does this PR introduce?

• add variable to limit field history to the last X years
• add variable to extend field history Y months
• cap ticket field history at its closure date (prior to adding the extension)
• a lil README formatting
• adding instructions regarding the extension + limit variables to the README

Does this PR introduce a breaking change?

• Yes (please provide breaking change details below.)
• No (please provide explanation how the change is non breaking below.)

yes in that now ticket field histories are capped at the close date, so like running a full refresh with this code will remove some rows that existed before (of post-closure tickets). no in that that may not be considered "breaking" since the tickets are closed and also in that the ticket_field_history_extension_months variable can be used to persist this behavior

thoughts? @fivetran-joemarkiewicz @fivetran-reneeli

Is this PR in response to a previously created Issue

• Yes, Issue BUG - Too many partitions produced by query, allowed 4000, query produces at least 4272 partitions #39
• No

How did you test the PR changes?

• CircleCi
• Other (please provide additional testing details below)

Select which warehouse(s) were used to test the PR

• BigQuery
• Redshift
• Snowflake
• Postgres
• Databricks
• Other (provide details below)

Provide an emoji that best describes your current mood

🐈

Feedback

We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your feedback on our existing dbt packages or what you'd like to see next.

[fivetran-reneeli]

Looks good!

[fivetran-joemarkiewicz]

@fivetran-jamie this PR looks great. I have one small request for rebasing the branch and we will include this in a breaking change for v0.7.0.

However, I set the review for requesting changes, but in actuality I would really just like for you to walk @fivetran-reneeli and I through the process of how this will help cleanup the zendesk__ticket_field_history model (ie. stop producing records after so many days of being closed). We can do this during standup tomorrow.

We can pick a ticket or two tomorrow and see with these changes how if the ticket is closed for X days then it will stop creating records, and also how it will impact records that are still open. With all the changes that will be applied to this package I want our team to get a better understanding of how all these moving pieces will work together!

Thanks!

[fivetran-jamie]

pre-change:

last day is today

post-change:

last day is 2020-02-027!

[fivetran-joemarkiewicz]

Looks great!! Thanks so much for working on this @fivetran-jamie! 🥇