🐛 Source Stripe: adding integration tests

[maxi297]

What

Adding integration tests for source-stripe events stream

How

• Add some tooling as part of Issue 32871/more integration test tooling to test events stream #33305
• Temporarily upgrade the CDK version to fix breaking change (will need to upgrade again to have access to the changes mentioned above)
• Ensuring backoff is very small to have fast running tests
• Create a ConfigBuilder to ease testing
• Add the tests

Recommended reading order

1. `airbyte-integrations/connectors/source-stripe/unit_tests/integration/test_events.py``
2. The rest

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Jan 11, 2024 3:16pm

[github-actions]

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

• PR name follows PR naming conventions
• Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
• Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
• You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
• Secrets in the connector's spec are annotated with airbyte_secret
• All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
• Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
• Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
• If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

1. Check for hidden checklists in your PR description

2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

[maxi297]

Currently failing because some of the tooling is not release until this is released

[girarda]

is this a CDK bug? why does it check the full refresh availability if we're reading in incremental mode?

[maxi297]

Not a CDK bug but a stripe one if it's one. See

airbyte/airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py

Lines 75 to 78 in 7a9ee9a

	is_available, reason = self._check_availability_for_sync_mode(stream, SyncMode.full_refresh, logger, source, None)
	if not is_available or not stream.supports_incremental:
	return is_available, reason
	return self._check_availability_for_sync_mode(stream, SyncMode.incremental, logger, source, {stream.cursor_field: 0})

Basically, as long as the stream is available in full_refresh and supports incremental, we check availability for both

[maxi297]

It seems like the team tried to add the state in the availability strategy in order to check properly (see #32223). It feels like this is a case where we instantiate things without all the information and hence we need to update the library which we agreed is not a good pattern.

My feeling is that when we will have full_refresh with state, we will need to rethink that so I would punt on that

[girarda]

the query params are derived from the stream slices, not the state, right?

[maxi297]

The stream slices is a implementation detail. At this level, the input that affect the HTTP requests is the state

[girarda]