Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source Chargebee - Implement integration testing for otherwise untested streams [ITAS] #35509

Merged
merged 113 commits into from
Mar 13, 2024
Merged

Source Chargebee - Implement integration testing for otherwise untested streams [ITAS] #35509

merged 113 commits into from
Mar 13, 2024

Conversation

pnilan
Copy link
Contributor

@pnilan pnilan commented Feb 21, 2024 •
edited
Loading

What

  • Implements integration tests for the following streams via http_mocker:
    • addon
    • coupon
    • customer
    • event
    • hosted_page
    • plan
    • site_migration_detail
    • subscription
    • virtual_bank_account
  • Adds additional testing dependency freezegun
  • Fixes error in site_migration_detail semi-incremental implementation. Filters records based on migrated_at.
  • Unpins CDK version to ^0.67.1
  • Increments connector version to 0.4.1

Integration Test Cases

  • One-page read
  • Multi-page read
  • Correctly handles 400 response status
  • Correctly handles 401 response status
  • Correctly handles 429 response status (backoff and retry)
  • Correctly handles 500 status correctly (retries and then receives 200 response)
  • Correctly handles recurring 500 status correctly (retries, throws error)
  • Custom field transformation for subscription, customer, and coupon streams
  • Incremental: Correctly sets state when no initial state is present
  • Incremental: Correctly slices read based on previous state value

@pnilan
Set additionalProperties to true fro custom_fields object.
18a04a8
@pnilan
Updated invoice and transaction schemas to include missing fields
974cd0c
@pnilan
Added missing fields to credit_note schema
7414f3c
@pnilan
Updated subscription schema plan_unit_price field for validation …
4fbe535 
…error
@pnilan
Updated payment_source for typo in funding_type field
80c2225
@pnilan
Updated credit_note schema for validation errors
3e28a13
@pnilan
Updated item and item_price schemas for validation errors
e4ca966
@pnilan
Added missing fields to customer schema
0400d6a
@pnilan
Update customer stream to account for correct tax_provider_fields…
0bcca84 
… data types
@pnilan
Updates chargebee changelog for correct date for 0.3.0 release
c52be59
@pnilan
Remove enumeration from entity_type field for unbilled_charge stream
c774d91
@pnilan
Chore: Format code
41636d5
@pnilan
Updates Chargebee documentation to reflect current template
94eb58e
@pnilan
Updates spec to make Product Catalog an optional input
f7f5305
@pnilan
Revert start_date to required property
c7a7d4c
@pnilan
Adds suggested streams
c5a94e8
@pnilan
Update documentation to better reflect prereqs
464053c
@pnilan
Updates connector for inc syncs for gift and site_migration_detail st…
f46bb6b 
…reams
@pnilan
Updates QL to reflect silver certification
f168f15
@pnilan
Updated date format error in spec.yaml
eafbeac
@pnilan
Rollback update to destination_sync_mode in configured_catalog
b984d59
@pnilan
Merge branch 'master' into pnilan/source-chargebee-schema-update
75b25b3
@pnilan
Added error handling for attached_item stream
439c4b5
@pnilan
Removed test bypass for unbilled_charge and hosted_page streams.
3b603e8
@pnilan
Merge branch 'master' into pnilan/source-chargebee-schema-update
12fcfe9
@pnilan
Merge branch 'master' into pnilan/source-chargebee-schema-update
dd84c07
@pnilan
Add append + deduped to supported sync modes in docs
87d5c93
@pnilan
Update metadata and changelog for new minor version
2e72df0
@pnilan
Merge branch 'pnilan/source-chargebee-schema-update' of github.com:ai…
46b69eb 
…rbytehq/airbyte into pnilan/source-chargebee-schema-update
@pnilan
Update gift stream to be full_refresh.
5d7ab98
maxi297
maxi297 reviewed Mar 1, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple comments (some unrelated to your changes but highlighted by the integration tests

airbyte-integrations/connectors/source-chargebee/integration_tests/configured_catalog.json Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/source_chargebee/components.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/source_chargebee/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/source_chargebee/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/source_chargebee/manifest.yaml Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_addon.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_addon.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_addon.py Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_addon.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_coupon.py Outdated Show resolved Hide resolved
@octavia-squidington-iv octavia-squidington-iv requested a review from a team March 1, 2024 19:34
natikgadzhi
natikgadzhi reviewed Mar 3, 2024
View reviewed changes
Copy link
Contributor

@natikgadzhi natikgadzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many workspaces do we have on Chargebee?

For a mostly low-code connector, the amount of code to have strong test coverage is insanely high. That's not critique of this PR, though — this PR is great.

airbyte-integrations/connectors/source-chargebee/pyproject.toml Show resolved Hide resolved
airbyte-integrations/connectors/source-chargebee/unit_tests/integration/test_addon.py Show resolved Hide resolved
@octavia-squidington-iv octavia-squidington-iv requested a review from a team March 3, 2024 17:28
@pnilan pnilan requested a review from maxi297 March 7, 2024 15:30
lazebnyi
lazebnyi reviewed Mar 10, 2024
View reviewed changes
Copy link
Collaborator

@lazebnyi lazebnyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Maxime's comments LGTM.

maxi297
maxi297 approved these changes Mar 11, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for your diligence in addressing my comments

maxi297
maxi297 reviewed Mar 11, 2024
View reviewed changes
airbyte-integrations/connectors/source-chargebee/integration_tests/configured_catalog.json
@@ -121,11 +121,11 @@
"supported_sync_modes": ["full_refresh", "incremental"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the configured_catalog show all the streams? There are some I don't see like event and site_migration_detail

@pnilan pnilan merged commit 58b6b80 into master Mar 13, 2024
29 checks passed
@pnilan pnilan deleted the pnilan/source-chargebee-integration-tests branch March 13, 2024 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natikgadzhi natikgadzhi natikgadzhi left review comments

@maxi297 maxi297 maxi297 approved these changes

@lazebnyi lazebnyi lazebnyi left review comments

@bazarnov bazarnov bazarnov approved these changes

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/chargebee
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@pnilan @natikgadzhi @maxi297 @bazarnov @lazebnyi @octavia-squidington-iii