🐛 Source Mixpanel: reimplement backoff strategy

[davydov-d]

What

https://github.com/airbytehq/oncall/issues/2363

How

The root cause of the issue is we're exceeding the number of retries because we're exceeding the rate limits. Therefore:

• Partially revert changes that were made in version 0.1.32: sleep after making every request as it is supposed by the Mixpanel API docs

This brings in a contradictory condition: one one hand we should wait after each api call, on the other hand the connector can time out when running CAT or during check() or discover() regarding changes made in version 0.1.35 (extra api calls added for both check() and discover()). That's why more changes were made:

• Sleep for 1 min after making each api call
• Timeouts for check and discover increased in the CAT config
• Skip sleeping when calling check or discover only in prod mode (do sleep when running tests)
• Use cache for the Cohorts stream
• Replace all iterators with generators where possible to not make unnecessary extra api calls
• Do not use constructions like "list(read_full_refresh(...))" to not make unnecessary extra api calls

upd:

• follow-up fix to Mixpanel: Filtering out individual items based on datetime in state #15091 as it's never been working as expected: the Export stream's cursor field is time, not date

🚨 User Impact 🚨

No negative impact

[octavia-squidington-iii]

source-stripe test report (commit 4f619c4944) - ❌

⏲️ Total pipeline duration: 33.36s

Step	Result
Validate airbyte-integrations/connectors/source-stripe/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	❌
Connector package install	✅
Build source-stripe docker image for platform linux/x86_64	✅
Unit tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-stripe test

[octavia-squidington-iii]

source-mixpanel test report (commit 4f619c4944) - ❌

⏲️ Total pipeline duration: 116mn22s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

[octavia-squidington-iii]

source-mixpanel test report (commit 6438b657a3) - ❌

⏲️ Total pipeline duration: 09mn07s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

[octavia-squidington-iii]

source-mixpanel test report (commit 2835add735) - ❌

⏲️ Total pipeline duration: 195mn35s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	❌
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

[octavia-squidington-iii]

source-mixpanel test report (commit 443781eaa9) - ❌

⏲️ Total pipeline duration: 67mn16s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

[octavia-squidington-iii]

source-mixpanel test report (commit c638de35fb) - ❌

⏲️ Total pipeline duration: 69mn22s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

[octavia-squidington-iii]

source-mixpanel test report (commit 5c326966eb) - ✅

⏲️ Total pipeline duration: 132mn36s

Step	Result
Validate airbyte-integrations/connectors/source-mixpanel/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-mixpanel docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test