🚨 🚨 Source Mixpanel: fix typing #30025

davydov-d · 2023-08-31T10:00:34Z

What

Fix date field: https://github.com/airbytehq/oncall/issues/1772
Fix the primary key for the export stream to identify a record uniquely: #2939

How

Remove "format": "date-time" from date-time fields as this is not quite correct. Mixpanel's datetime data can be either of

"2021-12-16T00:00:00"
"1638298874"
"15/08/53895"

Add fields to the primary key of export stream to identify a record uniquely

🚨 User Impact 🚨

Users will have to refresh their schemas and refresh data as some of their destination columns may change their type from Data/Time/Datetime to string. Primary key for export stream changed. Breaking changes doc.

github-actions · 2023-08-31T10:00:57Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

github-actions · 2023-08-31T11:11:28Z

source-mixpanel test report (commit `fce97832a7`) - ❌

⏲️ Total pipeline duration: 67mn21s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

github-actions · 2023-09-05T14:48:27Z

source-mixpanel test report (commit `f98c7ab358`) - ❌

⏲️ Total pipeline duration: 300mn50s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	❌
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

vercel · 2023-09-27T14:21:02Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 2, 2023 7:09pm

airbyte-oss-build-runner · 2023-09-27T15:31:33Z

source-mixpanel test report (commit `f29f0b0c0a`) - ❌

⏲️ Total pipeline duration: 68mn40s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	❌
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

airbyte-oss-build-runner · 2023-09-27T16:54:10Z

source-mixpanel test report (commit `0103f3b10a`) - ✅

⏲️ Total pipeline duration: 70mn02s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

airbyte-oss-build-runner · 2023-09-28T09:54:19Z

source-mixpanel test report (commit `aee4755dea`) - ✅

⏲️ Total pipeline duration: 66mn54s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

clnoll

Looks great, just one small question about the test, @davydov-d.

clnoll · 2023-09-28T11:49:54Z

airbyte-integrations/connectors/source-mixpanel/acceptance-test-config.yml

@@ -1,6 +1,7 @@
 # See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
 # for more information about how to configure these tests
 connector_image: airbyte/source-mixpanel:dev
+# custom configuration is used for tests to speed up testing and avoid hitting rate limits


👍 thanks for adding the comment here.

airbyte-integrations/connectors/source-mixpanel/unit_tests/test_streams.py

clnoll

LGTM pending the approval of the breaking changes doc!

airbyte-oss-build-runner · 2023-09-30T01:52:03Z

source-mixpanel test report (commit `56ff05661c`) - ❌

⏲️ Total pipeline duration: 130mn08s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate metadata for source-mixpanel	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

…or-mixpanel

airbyte-oss-build-runner · 2023-10-02T18:09:22Z

source-mixpanel test report (commit `5068013a85`) - ❌

⏲️ Total pipeline duration: 65mn09s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate metadata for source-mixpanel	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

…or-mixpanel

airbyte-oss-build-runner · 2023-10-02T19:12:24Z

source-mixpanel test report (commit `5c43999ee8`) - ✅

⏲️ Total pipeline duration: 02mn11s

Step	Result
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate metadata for source-mixpanel	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

Co-authored-by: Anatolii Yatsuk <tolikyatsuk@gmail.com> Co-authored-by: Pedro S. Lopez <pedroslopez@me.com> Co-authored-by: pedroslopez <pedroslopez@users.noreply.github.com> Co-authored-by: Anatolii Yatsuk <35109939+tolik0@users.noreply.github.com>

grkhr · 2023-10-17T14:59:11Z

@tolik0 @davydov-d

Hey folks! This primary key change (for export) is still non-accurate.

airbyte/airbyte-integrations/connectors/source-mixpanel/source_mixpanel/streams/export.py

Line 78 in 5001ec1

primary_key: str = ["distinct_id", "event", "time"]

You are assuming that a user can't send multiple events with the same name in one second, but it's happening every time with high frequency events. Also properties can be different.
MixPanel uses insert_id as the unique "event". Please consider using it or just leave that choice to the end user. Now it's impossible to change primary key in UI, it's locked.

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/mixpanel labels Aug 31, 2023

davydov-d requested review from lazebnyi and darynaishchenko August 31, 2023 10:01

davydov-d marked this pull request as ready for review August 31, 2023 10:01

darynaishchenko approved these changes Sep 1, 2023

View reviewed changes

lazebnyi approved these changes Sep 4, 2023

View reviewed changes

davydov-d requested a review from erohmensing September 5, 2023 09:45

davydov-d changed the title ~~🐛 Source Mixpanel: fix typing~~ 🚨 🚨 Source Mixpanel: fix typing Sep 5, 2023

davydov-d added the breaking-change Don't merge me unless you are ready. label Sep 5, 2023

davydov-d assigned davydov-d and tolik0 Sep 8, 2023

davydov-d and others added 2 commits September 27, 2023 17:14

#1772 source mixpanel: fix typing

ec39ea7

Fix export primary key

f29f0b0

tolik0 force-pushed the ddavydov/#1772-fix-datetime-field-typing-for-mixpanel branch from e36f40d to f29f0b0 Compare September 27, 2023 14:21

vercel bot deployed to Preview September 27, 2023 14:22 View deployment

Update version

0103f3b

vercel bot deployed to Preview September 27, 2023 15:43 View deployment

Add comment for custom testing environment

aee4755

tolik0 requested a review from clnoll September 28, 2023 08:59

clnoll reviewed Sep 28, 2023

View reviewed changes

tolik0 requested a review from clnoll September 28, 2023 12:29

clnoll approved these changes Sep 28, 2023

View reviewed changes

pedroslopez added 2 commits September 29, 2023 18:31

clarify changes need to happen after applying the upgrade

b3d22a9

clarify changes need to happen after applying the upgrade

e87e3c4

vercel bot deployed to Preview September 29, 2023 22:33 View deployment

messaging tweaks

56ff056

vercel bot deployed to Preview September 29, 2023 23:41 View deployment

Merge branch 'master' into ddavydov/#1772-fix-datetime-field-typing-f…

f22c3e7

…or-mixpanel

vercel bot deployed to Preview October 2, 2023 16:51 View deployment

Automated Commit - Formatting Changes

5068013

Merge branch 'master' into ddavydov/#1772-fix-datetime-field-typing-f…

5c43999

…or-mixpanel

vercel bot deployed to Preview October 2, 2023 19:09 View deployment

pedroslopez merged commit 5001ec1 into master Oct 2, 2023
24 checks passed

pedroslopez deleted the ddavydov/#1772-fix-datetime-field-typing-for-mixpanel branch October 2, 2023 20:29

tolik0 mentioned this pull request Oct 30, 2023

🚨🚨🐛 Source Mixapanel: Delete default primary key for Export stream #31955

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚨 🚨 Source Mixpanel: fix typing #30025

🚨 🚨 Source Mixpanel: fix typing #30025

davydov-d commented Aug 31, 2023 •

edited by tolik0

Loading

github-actions bot commented Aug 31, 2023 •

edited by tolik0

Loading

github-actions bot commented Aug 31, 2023

github-actions bot commented Sep 5, 2023

vercel bot commented Sep 27, 2023 •

edited

Loading

airbyte-oss-build-runner commented Sep 27, 2023

airbyte-oss-build-runner commented Sep 27, 2023

airbyte-oss-build-runner commented Sep 28, 2023

clnoll left a comment

clnoll Sep 28, 2023

clnoll left a comment

airbyte-oss-build-runner commented Sep 30, 2023

airbyte-oss-build-runner commented Oct 2, 2023

airbyte-oss-build-runner commented Oct 2, 2023

grkhr commented Oct 17, 2023 •

edited

Loading

🚨 🚨 Source Mixpanel: fix typing #30025

🚨 🚨 Source Mixpanel: fix typing #30025

Conversation

davydov-d commented Aug 31, 2023 • edited by tolik0 Loading

What

How

🚨 User Impact 🚨

github-actions bot commented Aug 31, 2023 • edited by tolik0 Loading

Before Merging a Connector Pull Request

github-actions bot commented Aug 31, 2023

source-mixpanel test report (commit fce97832a7) - ❌

github-actions bot commented Sep 5, 2023

source-mixpanel test report (commit f98c7ab358) - ❌

vercel bot commented Sep 27, 2023 • edited Loading

airbyte-oss-build-runner commented Sep 27, 2023

source-mixpanel test report (commit f29f0b0c0a) - ❌

airbyte-oss-build-runner commented Sep 27, 2023

source-mixpanel test report (commit 0103f3b10a) - ✅

airbyte-oss-build-runner commented Sep 28, 2023

source-mixpanel test report (commit aee4755dea) - ✅

clnoll left a comment

Choose a reason for hiding this comment

clnoll Sep 28, 2023

Choose a reason for hiding this comment

clnoll left a comment

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 30, 2023

source-mixpanel test report (commit 56ff05661c) - ❌

airbyte-oss-build-runner commented Oct 2, 2023

source-mixpanel test report (commit 5068013a85) - ❌

airbyte-oss-build-runner commented Oct 2, 2023

source-mixpanel test report (commit 5c43999ee8) - ✅

grkhr commented Oct 17, 2023 • edited Loading

davydov-d commented Aug 31, 2023 •

edited by tolik0

Loading

github-actions bot commented Aug 31, 2023 •

edited by tolik0

Loading

source-mixpanel test report (commit `fce97832a7`) - ❌

source-mixpanel test report (commit `f98c7ab358`) - ❌

vercel bot commented Sep 27, 2023 •

edited

Loading

source-mixpanel test report (commit `f29f0b0c0a`) - ❌

source-mixpanel test report (commit `0103f3b10a`) - ✅

source-mixpanel test report (commit `aee4755dea`) - ✅

source-mixpanel test report (commit `56ff05661c`) - ❌

source-mixpanel test report (commit `5068013a85`) - ❌

source-mixpanel test report (commit `5c43999ee8`) - ✅

grkhr commented Oct 17, 2023 •

edited

Loading