Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Stripe: fix stream slicing #24471

Conversation

davydov-d
Copy link
Collaborator

What

https://github.com/airbytehq/oncall/issues/1713

How

Refactoring in this PR introduced a bug: some streams yielded None as a slice instead of {}. As a rule we yield a None slice if a start_date is > now() so we dont make request to read the data in the future and pass an abnormally large incremental read test. A check in read_records skips this slice.

This wasn't caught by the CAT because we dont have those streams well seeded (perhaps it's impossible). Now the slicing is fixed.

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/stripe labels Mar 24, 2023
@davydov-d
Copy link
Collaborator Author

davydov-d commented Mar 24, 2023

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4513064439
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4513064439
🐛 https://gradle.com/s/z4fl5d4u2dijg

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Please check...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
============= 1 failed, 37 passed, 2 skipped in 200.97s (0:03:20) ==============

@pedroslopez
Copy link
Contributor

pedroslopez commented Mar 24, 2023

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4514775616
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4514775616
🐛 https://gradle.com/s/eotybcd2helgg

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Please check...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
============= 1 failed, 37 passed, 2 skipped in 199.56s (0:03:19) ==============

@pedroslopez
Copy link
Contributor

pedroslopez commented Mar 24, 2023

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4514850140
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4514850140
🐛 https://gradle.com/s/zcp2qfsonbzks

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream event...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
============= 1 failed, 37 passed, 2 skipped in 200.55s (0:03:20) ==============

@pedroslopez
Copy link
Contributor

pedroslopez commented Mar 24, 2023

Latest failure is due to expected records mismatch, which is happening on master and is a known failure, so publishing anyway for this OC fix.

@pedroslopez
Copy link
Contributor

pedroslopez commented Mar 24, 2023

/publish connector=connectors/source-stripe run-tests=false

🕑 Publishing the following connectors:
connectors/source-stripe
https://github.com/airbytehq/airbyte/actions/runs/4515002025


Connector Did it publish? Were definitions generated?
connectors/source-stripe

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@pedroslopez

This comment was marked as duplicate.

@pedroslopez pedroslopez enabled auto-merge (squash) March 24, 2023 20:34
@pedroslopez pedroslopez merged commit 94d6cc1 into master Mar 24, 2023
20 of 22 checks passed
@pedroslopez pedroslopez deleted the ddavydov/#1713-source-stripe-fix-slicing-of-single-sliced-streams branch March 24, 2023 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/stripe
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants