Source Stripe: Add New streams #27083

artem1205 · 2023-06-06T18:55:36Z

What

Resolve

How

add new streams:

Authorizations
Cards
Payment Methods
Reviews
Cardholders

refactor of existing schemas

🚨 User Impact 🚨

no breaking changes

Pre-merge Actions

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.

github-actions · 2023-06-06T18:55:56Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
You, or an Airbyter, have run /test successfully on this PR - or on a non-forked branch
You've updated the connector's metadata.yaml file (new!)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

artem1205 · 2023-06-06T20:31:17Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5193078945
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5193078945
🐛 https://gradle.com/s/5xvy323uyv5ti

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_discover[inputs0] - docker.errors.Co...
ERROR test_core.py::TestDiscovery::test_defined_cursors_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_refs_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-allOf]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-not]
ERROR test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_streams_has_sync_modes[inputs0] - doc...
ERROR test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
ERROR test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contain...
ERROR test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
ERROR test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
ERROR test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
ERROR test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
============= 1 failed, 25 passed, 2 skipped, 13 errors in 18.64s ==============

artem1205 · 2023-06-06T20:47:42Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5193206643
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5193206643
🐛 https://gradle.com/s/6b5mk6du7zrae

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - co...
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AttributeError: 'str...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
============= 2 failed, 37 passed, 2 skipped in 315.38s (0:05:15) ==============

artem1205 · 2023-06-07T13:08:45Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5200389890
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5200389890
🐛 https://gradle.com/s/lvt4pttt5r55o

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AttributeError: 'str...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:607: Backward compatibility tests are disabled for version 3.6.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
============= 1 failed, 37 passed, 3 skipped in 312.08s (0:05:12) ==============

artem1205 · 2023-06-07T15:23:33Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5201858256
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5201858256
🐛 https://gradle.com/s/ipb6ey7yq6epu

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AssertionError: All ...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:607: Backward compatibility tests are disabled for version 3.6.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
============= 1 failed, 37 passed, 3 skipped in 310.91s (0:05:10) ==============

artem1205 · 2023-06-07T21:18:26Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5204857232
❌ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5204857232
🐛 https://gradle.com/s/iihqqqnpapjhu

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream custo...
ERROR test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:607: Backward compatibility tests are disabled for version 3.6.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
========= 1 failed, 36 passed, 3 skipped, 1 error in 313.90s (0:05:13) =========

artem1205 · 2023-06-08T12:17:05Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5211085385
✅ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5211085385
Python tests coverage:

Name                                     Stmts   Miss  Cover
------------------------------------------------------------
source_stripe/source.py                     22      0   100%
source_stripe/__init__.py                    2      0   100%
source_stripe/streams.py                   368     38    90%
source_stripe/availability_strategy.py      19      2    89%
------------------------------------------------------------
TOTAL                                      411     40    90%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:607: Backward compatibility tests are disabled for version 3.6.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
================== 38 passed, 3 skipped in 334.60s (0:05:34) ===================

# Conflicts: # airbyte-integrations/connectors/source-stripe/integration_tests/expected_records.jsonl

artem1205 · 2023-06-12T17:14:58Z

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5246553526
✅ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/5246553526
Python tests coverage:

Name                                     Stmts   Miss  Cover
------------------------------------------------------------
source_stripe/source.py                     22      0   100%
source_stripe/__init__.py                    2      0   100%
source_stripe/streams.py                   372     39    90%
source_stripe/availability_strategy.py      19      2    89%
------------------------------------------------------------
TOTAL                                      415     41    90%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:607: Backward compatibility tests are disabled for version 3.6.0.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:695: This tests currently leads to too much failures. We need to fix the connectors at scale first.
================== 38 passed, 3 skipped in 368.31s (0:06:08) ===================

airbyte-integrations/connectors/source-stripe/acceptance-test-config.yml

Source Stripe: Add Stream: Cards

fb74d4b

artem1205 changed the title ~~Source Stripe: Add Stream: Cards~~ Source Stripe: Add New streams Jun 6, 2023

octavia-squidington-iii added the area/connectors Connector related issues label Jun 6, 2023

octavia-squidington-iii added the connectors/source/stripe label Jun 6, 2023

Source Stripe: update docs; tests config; bump version

84d09f4

octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Jun 6, 2023

Source Stripe: Add Stream: Payment Methods; Refactor

a93fc1c

artem1205 requested review from lazebnyi and darynaishchenko June 6, 2023 20:13

artem1205 self-assigned this Jun 6, 2023

This was linked to issues Jun 6, 2023

Source Stripe: new stream PaymentMethods #16963

Closed

Source Stripe: new stream Cards #16958

Closed

artem1205 added 4 commits June 6, 2023 22:15

Source Stripe: update docs

7083284

Source Stripe: update docs

d81269b

Source Stripe: fix unit tests

4499dc1

Source Stripe: update abnormal_state

0c25e17

Source Stripe: update schemas

07eee2c

artem1205 added 2 commits June 7, 2023 14:30

Source Stripe: update acceptance tests

a824316

Source Stripe: fix address type

7237438

Source Stripe: fix schemas

a2bd026

artem1205 added 4 commits June 7, 2023 21:17

Source Stripe: Add stream: Reviews

fceddbd

Source Stripe: update docs

a6edb05

Source Stripe: update acceptance tests

492999d

Source Stripe: update schema

d16578a

artem1205 linked an issue Jun 7, 2023 that may be closed by this pull request

Source Stripe: new stream Reviews #16977

Closed

artem1205 added 4 commits June 8, 2023 12:52

Source Stripe: add stream Cardholders

6d27ea0

Source Stripe: update expected records

a09b4ca

Source Stripe: add expected records for cardholder

dbd7881

Source Stripe: fix unit test

0d0943e

Source Stripe: add abnormal state

045f73b

artem1205 requested a review from a team June 8, 2023 12:38

lazebnyi approved these changes Jun 9, 2023

View reviewed changes

artem1205 added 2 commits June 12, 2023 12:03

Merge branch 'master' into artem1205/source-stripe-add-streams-26546

870c7ea

# Conflicts: # airbyte-integrations/connectors/source-stripe/integration_tests/expected_records.jsonl

Source Stripe: add stream: Authorizations

d441826

This was linked to issues Jun 13, 2023

Source Stripe: new stream Authorizations #26475

Closed

Source Stripe: new stream Cardholders #26476

Closed

sherifnada approved these changes Jun 14, 2023

View reviewed changes

airbyte-integrations/connectors/source-stripe/acceptance-test-config.yml Outdated Show resolved Hide resolved

Source Stripe: update comment

5456137

artem1205 merged commit 46d1f2b into master Jun 14, 2023
22 of 24 checks passed

artem1205 deleted the artem1205/source-stripe-add-streams-26546 branch June 14, 2023 15:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Stripe: Add New streams #27083

Source Stripe: Add New streams #27083

artem1205 commented Jun 6, 2023 •

edited

github-actions bot commented Jun 6, 2023 •

edited by artem1205

artem1205 commented Jun 6, 2023 •

edited by github-actions bot

artem1205 commented Jun 6, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 8, 2023 •

edited by github-actions bot

artem1205 commented Jun 12, 2023 •

edited by github-actions bot

Source Stripe: Add New streams #27083

Source Stripe: Add New streams #27083

Conversation

artem1205 commented Jun 6, 2023 • edited

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Actions

Community member or Airbyter

Airbyter

github-actions bot commented Jun 6, 2023 • edited by artem1205

Before Merging a Connector Pull Request

artem1205 commented Jun 6, 2023 • edited by github-actions bot

Build Failed

artem1205 commented Jun 6, 2023 • edited by github-actions bot

Build Failed

artem1205 commented Jun 7, 2023 • edited by github-actions bot

Build Failed

artem1205 commented Jun 7, 2023 • edited by github-actions bot

Build Failed

artem1205 commented Jun 7, 2023 • edited by github-actions bot

Build Failed

artem1205 commented Jun 8, 2023 • edited by github-actions bot

Build Passed

artem1205 commented Jun 12, 2023 • edited by github-actions bot

Build Passed

artem1205 commented Jun 6, 2023 •

edited

github-actions bot commented Jun 6, 2023 •

edited by artem1205

artem1205 commented Jun 6, 2023 •

edited by github-actions bot

artem1205 commented Jun 6, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 7, 2023 •

edited by github-actions bot

artem1205 commented Jun 8, 2023 •

edited by github-actions bot

artem1205 commented Jun 12, 2023 •

edited by github-actions bot