Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Breaking change in stg_mixpanel___events.sql event table source #44

Open
2 of 4 tasks
chiefdatawizard opened this issue Feb 22, 2024 · 6 comments
Open
2 of 4 tasks
Labels
status:stale Issue was blocked or had no user response for more than 30 days type:wontfix This will not be worked on

Comments

@chiefdatawizard
Copy link

chiefdatawizard commented Feb 22, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

The introduction of the hardcoded source table in commit 1b66f95 [https://github.com/fivetran/dbt_mixpanel/commit/1b66f95022f7e37c78863441b1fa66b48f36f8d8#diff-06f26e4fc8545509412ed16924c7ffacb49b0057ad1f6bb83d75ba1568197428] for events introduced issues for users that used the variable event_table in their dbt_project.yml config.
image

Relevant error log or model output

No response

Expected behavior

The source event table can be determined by the global var event_table in the dbt_project.yml

dbt Project configurations

vars:
  event_table: "{{ ref('stg_mixpanel_union_old_new') }}"
  mixpanel__event_frequency_limit: 1000

Package versions

packages:
- package: fivetran/mixpanel
  version: [">=0.8.0", "<2.0.0"]

What database are you using dbt with?

snowflake

dbt Version

1.6

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance and will schedule time during our office hours for guidance
  • No.
@fivetran-catfritz
Copy link
Contributor

Hi @chiefdatawizard thanks for bringing this up! The switch to the hard-coded source was necessary to take advantage of the dbt_utils.star macro, however you can still specify a custom source event table via the identifier variable. For your case. I'd use something like:

vars:
  mixpanel_event_identifier: stg_mixpanel_union_old_new
  mixpanel_schema: <name of the schema for stg_mixpanel_union_old_new>

Please let me know if that works for you. I'll also mention that we do use minor version updates to indicate breaking changes, so we recommend limiting version ranges to patch updates such as version: [">=0.8.0", "<0.9.0"] to avoid breaking updates being automatically loaded.

@fivetran-catfritz
Copy link
Contributor

Hi @chiefdatawizard following up to my last message, I thought about it some more and realize that while the identifier will point to the alternative table, the downside is dbt won't treat it as an upstream model and know to run it first. I'll discuss this with our internal team for ideas on a remedy.

@chiefdatawizard
Copy link
Author

Hi @chiefdatawizard thanks for bringing this up! The switch to the hard-coded source was necessary to take advantage of the dbt_utils.star macro, however you can still specify a custom source event table via the identifier variable. For your case. I'd use something like:

vars:
  mixpanel_event_identifier: stg_mixpanel_union_old_new
  mixpanel_schema: <name of the schema for stg_mixpanel_union_old_new>

Please let me know if that works for you. I'll also mention that we do use minor version updates to indicate breaking changes, so we recommend limiting version ranges to patch updates such as version: [">=0.8.0", "<0.9.0"] to avoid breaking updates being automatically loaded.

Hi @fivetran-catfritz, thank you for your reply. We will look into this, to see if this is a viable option for us.

@fivetran-catfritz
Copy link
Contributor

Hi @chiefdatawizard just checking in, did my suggestion work for you?

@chiefdatawizard
Copy link
Author

Hi! Unfortunately it is still on our to-do. But when we got an answer I will let you guys know.

@fivetran-catfritz fivetran-catfritz added type:wontfix This will not be worked on status:stale Issue was blocked or had no user response for more than 30 days labels Apr 29, 2024
@fivetran-catfritz
Copy link
Contributor

Hi @chiefdatawizard let us know if you've been able to try this out. We'll come back to this if so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:stale Issue was blocked or had no user response for more than 30 days type:wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants