Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Source Shopify: Allow the known HTTP errors to be retried more than once for the BULK streams #37589

Merged
merged 17 commits into from
May 2, 2024
Merged

🐛 Source Shopify: Allow the known HTTP errors to be retried more than once for the BULK streams #37589

merged 17 commits into from
May 2, 2024

Conversation

bazarnov
Copy link
Collaborator

@bazarnov bazarnov commented Apr 25, 2024 •
edited
Loading

What

Given this connection, we frequently see the Internal Server Errors or Unknown Errors, which could and should be retried more than once to overcome the server availability and not to raise the error in between, allowing the source to retry more than once.

How

  • added bulk_retry_on_exception decorator to catch and retry the known bulk errors (max attempts = 6)
  • made small refactoring to the job_manager to:
    • simplify the checking the state of RUNNING job
    • made the private methods to be _private when they are not used elsewhere but the module itself
    • updated the unit_tests to cover the changes (the logic didn't change!)
    • added a new test to cover the retry for known HTTP errors for BULK job: test_job.py > test_retry_on_job_exception

User Impact

No impact is expected.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@bazarnov bazarnov self-assigned this Apr 25, 2024
@vercel Vercel
Copy link

vercel bot commented Apr 25, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 2, 2024 1:12pm

@bazarnov bazarnov marked this pull request as ready for review April 25, 2024 21:56
@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Apr 25, 2024
@bazarnov bazarnov requested a review from maxi297 April 25, 2024 21:57
@octavia-squidington-iv octavia-squidington-iv requested review from a team April 25, 2024 21:58
@bazarnov bazarnov requested a review from strosek April 26, 2024 11:29
maxi297
maxi297 reviewed Apr 26, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide more information on the comments and ping me right after so that I can re-review with the added information?

airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Outdated Show resolved Hide resolved
@octavia-squidington-iv octavia-squidington-iv requested a review from a team April 26, 2024 12:55
strosek
strosek approved these changes Apr 26, 2024
View reviewed changes
Copy link
Contributor

@strosek strosek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This codes looks OK, please address Max's concerns. I also added a few suggestions to improve readability.

airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Outdated Show resolved Hide resolved
@bazarnov bazarnov requested a review from maxi297 April 26, 2024 15:26
maxi297
maxi297 reviewed Apr 26, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that until I understand the flow, I'll have to mark this as "Request changes". Based on what I see, there is potential for data loss which I'm not comfortable with. We can sync on this if it's easier to address

maxi297
maxi297 requested changes Apr 26, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to add the flag based on #37589 (review)

@octavia-squidington-iv octavia-squidington-iv requested a review from a team April 26, 2024 17:11
@bazarnov
Copy link
Collaborator Author

Would be happy to discuss this once you're available.

I think that until I understand the flow, I'll have to mark this as "Request changes". Based on what I see, there is potential for data loss which I'm not comfortable with. We can sync on this if it's easier to address

Thank you for the review.

@bazarnov bazarnov requested a review from maxi297 April 29, 2024 16:08
maxi297
maxi297 reviewed Apr 30, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the newest changes. I like that:

  • method visibility (private or public) is now explicit
  • the retry is with an annotation on the _job_check_state method

I have a couple of concerns which you can push back on but once we see the same path forward, I'll approve

airbyte-integrations/connectors/source-shopify/unit_tests/graphql_bulk/test_job.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-shopify/source_shopify/shopify_graphql/bulk/job.py Show resolved Hide resolved
airbyte-integrations/connectors/source-shopify/source_shopify/streams/base_streams.py Show resolved Hide resolved
@octavia-squidington-iv octavia-squidington-iv requested a review from a team April 30, 2024 14:53
maxi297
maxi297 approved these changes May 2, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks for the changes. :shipit: !

@bazarnov bazarnov requested a review from a team May 2, 2024 12:46
@bazarnov bazarnov mentioned this pull request May 2, 2024
Closed
@bazarnov bazarnov merged commit d34b521 into master May 2, 2024
35 checks passed
@bazarnov bazarnov deleted the baz/source/shopify/add-retry-for-bulk-http-errors branch May 2, 2024 15:35
clnoll pushed a commit that referenced this pull request May 10, 2024
@bazarnov @clnoll
🐛 Source Shopify: Allow the known HTTP errors to be retried more th…
b6a1f86 
…an once for the BULK streams (#37589)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@strosek strosek strosek approved these changes

@maxi297 maxi297 maxi297 approved these changes

Assignees

@bazarnov bazarnov

Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/shopify team/critical-connectors
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@bazarnov @strosek @maxi297 @octavia-squidington-iii