Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apply data types for dates #51

Merged
merged 21 commits into from
Aug 30, 2023
Merged

Conversation

fivetran-reneeli
Copy link
Contributor

@fivetran-reneeli fivetran-reneeli commented Aug 18, 2023

PR Overview

This PR will address the following Issue/Feature: fivetran/dbt_quickbooks#96

This PR will result in the following new package version:

0.10.0

Moved to breaking, although these changes are under the hood and will not have visible impact on downstream model materializations, this will be breaking for users who still have the old timestamp version of the fields impacted.
This will also be in combo with PR #103 in the transforms-

Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:

  • Explicitly casts *_date fields as date type (due_date, transaction_date). Previously, some fields were getting interpreted as timestamps while some were interpreted as dates, leading to errors downstream on joins.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt compile
  • dbt run –full-refresh
  • dbt run
  • dbt test
  • dbt run –vars (if applicable)

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked and tagged
  • You are assigned to the corresponding issue and this PR
  • BuildKite integration tests are passing

Detailed Validation

Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":

  • You have validated these changes and assure this PR will address the respective Issue/Feature.
  • You are reasonably confident these changes will not impact any other components of this package or any dependent packages.
  • You have provided details below around the validation steps performed to gain confidence in these changes.

Models should successfully run. Additionally will ask the customer to run on this branch as well.

Standard Updates

Please acknowledge that your PR contains the following standard updates:

  • Package versioning has been appropriately indexed in the following locations:
    • indexed within dbt_project.yml
    • indexed within integration_tests/dbt_project.yml
  • CHANGELOG has individual entries for each respective change in this PR
  • [n/a] README updates have been applied (if applicable)
  • [ n/a] DECISIONLOG updates have been updated (if applicable)
  • [ n/a] Appropriate yml documentation has been added (if applicable)

dbt Docs

Please acknowledge that after the above were all completed the below were applied to your branch:

  • docs were regenerated (unless this PR does not include any code or yml updates)

If you had to summarize this PR in an emoji, which would it be?

💃

@fivetran-reneeli fivetran-reneeli self-assigned this Aug 18, 2023
@fivetran-reneeli
Copy link
Contributor Author

  • regen docs after approved

Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fivetran-reneeli! Looks mostly good.

A few comments:

  • Can you regenerate docs?
  • Can you update your issue with all the appropriate tags? [Bug] type mismatches using bigquery dbt_quickbooks#96
  • We will probably want to update the integration test file updates in dbt_quickbooks in both the seed file and the dbt_project.yml to match the updates in dbt_quickbooks_source and create a release there too. But I'd confirm with Joe just to make sure that is something we should do now or in a future sprint.

@@ -223,7 +223,7 @@ models:
description: Reference number for the transaction.
- name: vendor_id
description: Reference to the vendor the bill is linked with.
- name: due_date_at
- name: due_date
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this still due_date_at?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes I switched it back to make it standard with the other models, but then realized it's called upon downstream so I meant to revert it to due_date_at. Although, if we are making a PR in the transforms now, perhaps we should make it due_date and make the appropriate changes downstream.

I actually was wondering why it was changed to due_date_at, since it just gets renamed to due_date in transforms where it gets called.
bill_link.due_date_at as due_date,

@fivetran-joemarkiewicz Any thoughts? I don't see the reason why it was renamed to due_date_at, in stg_quickbooks__bill only

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This likely was just human error for why this one was named differently. Truthfully, I would be cautious if you are changing this as it will likely cause more scope creep than you think as you called out it is leveraged downstream. Additionally, if you change the name of this field it will be reflected as a breaking change in the source.

We can chat more about this during standup.

Copy link
Contributor Author

@fivetran-reneeli fivetran-reneeli Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to the existing issue we now have for small fixes.
fivetran/dbt_quickbooks#101

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
integration_tests/dbt_project.yml Show resolved Hide resolved
fivetran-reneeli and others added 2 commits August 22, 2023 18:14
Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>
@fivetran-reneeli
Copy link
Contributor Author

Thanks @fivetran-avinash , made the updates and in transforms as well! This is ready for re-review as well as for tranforms

Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@fivetran-avinash
Copy link
Contributor

@fivetran-reneeli LGTM

@fivetran-reneeli fivetran-reneeli merged commit 7a2ef84 into main Aug 30, 2023
1 check passed
@fivetran-reneeli fivetran-reneeli deleted the bugfix/cast_data_types_dates branch August 30, 2023 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants