Skip to content

[cherry-pick][PLUGIN-1715] Implement retry feature for BigQuery Execute Plugin#1358

Merged
vikasrathee-cs merged 1 commit intodata-integrations:release/0.23from
cloudsufi:release/0.23.1-cherry-pick-PLUGIN-1715
Feb 12, 2024
Merged

[cherry-pick][PLUGIN-1715] Implement retry feature for BigQuery Execute Plugin#1358
vikasrathee-cs merged 1 commit intodata-integrations:release/0.23from
cloudsufi:release/0.23.1-cherry-pick-PLUGIN-1715

Conversation

@psainics
Copy link
Copy Markdown
Contributor

[Cherrypick] Implement retry feature for BigQuery Execute Plugin
Commit : a5e3e7b
PR: #1334


Implement retry feature for BigQuery Execute Plugin

Jira : PLUGIN-1715

  • In general internal errors should be retried.
  • Do an exponential back-off retry on errors with reason jobBackendError , jobInternalError.
  • Backoff requirements (exponential from 1 to 32 seconds with a multiplier of 2)

Code Changes

  • Add a new set that contains all reason we should retry on.
  • A new custom exception that is raised when the error should be retried.
  • A new function to generate the exponential back- off retry policy
  • A new dependency is added dev.failsafe that handles the retry logic based on the exception raised.

Unit Tests

  • Use mocks to test the behavior of the retry.

@itsankit-google itsankit-google added the build Trigger unit test build label Feb 9, 2024
Added BQ Retry
@psainics psainics force-pushed the release/0.23.1-cherry-pick-PLUGIN-1715 branch from 9364fab to 96e0ea0 Compare February 10, 2024 09:56
@vikasrathee-cs vikasrathee-cs merged commit 47531d1 into data-integrations:release/0.23 Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Trigger unit test build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants