New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source Stripe: add availability strategy #22659
Source Stripe: add availability strategy #22659
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Solid - this sort of work would have handled the bug I ran into the first time around 😄 Please add a unit test!
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py
Show resolved
Hide resolved
available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | ||
if not available: | ||
return available, reason |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | |
if not available: | |
return available, reason | |
is_available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | |
if not is_available: | |
return is_available, reason |
Small suggestion to make return is_available, reason
not look on first glance that it's returning a true
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops I meant to request changes for the test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the unit tests! Last question is about empty streams
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py
Show resolved
Hide resolved
empty_streams: ["bank_accounts", "checkout_sessions", "checkout_sessions_line_items", "external_account_bank_accounts"] | ||
# TEST 1 - Reading catalog without invoice_line_items | ||
- config_path: "secrets/config.json" | ||
configured_catalog_path: "integration_tests/non_invoice_line_items_catalog.json" | ||
empty_streams: ["transfers"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What led to these increases in empty streams?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have new account for stripe. It was populated recently by our unblockers team. There are streams which they cannot populate right now for various reasons:
bank_accounts
external_account_bank_accounts
checkout_sessions
checkout_sessions_line_items
transfers
In order to not to block development of this connector we decided to create a new task to populate those streams later. I can do nothing with it for now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, thanks for explaining, as long as that's being tracked!
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-stripe/unit_tests/test_availability_strategy.py
Outdated
Show resolved
Hide resolved
/test connector=connectors/source-stripe
Build PassedTest summary info:
|
/publish connector=connectors/source-stripe
if you have connectors that successfully published but failed definition generation, follow step 4 here |
Airbyte Code Coverage
|
cd85519
to
b93c1fe
Compare
fix changelog message Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
…availability_strategy.py assert availability result Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
/approve-and-merge reason="published before code freeze" |
What
- turn availability strategy for all streams
- prepare custom availability strategy for substreams