Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

On failed calling Rest-API wait for random moment #2121

Closed
benedeki opened this issue Aug 29, 2022 · 1 comment · Fixed by #2122
Closed

On failed calling Rest-API wait for random moment #2121

benedeki opened this issue Aug 29, 2022 · 1 comment · Fixed by #2122
Assignees
@jakipatryk
Labels
Conformance Conformance Job affected feature New feature priority: medium Important but not urgent Standardization Standardization Job affected

Comments

@benedeki
Copy link
Collaborator

Background

When a Rest-API calls fails from SparkJobs it is retried immediately, possibly creating an ingestion on the Rest endpoint.

Feature

In case of a failure & retry on Rest API call add a random length ~2 seconds sleep.
@benedeki benedeki added feature New feature Conformance Conformance Job affected Standardization Standardization Job affected priority: medium Important but not urgent labels Aug 29, 2022
jakipatryk added a commit that referenced this issue Aug 30, 2022
@jakipatryk
#2121 Implement retries backoff mechanism for CrossHostApiCaller
4f234cb
jakipatryk added a commit that referenced this issue Aug 30, 2022
@jakipatryk
#2121 Fix tests for RestDaoFactory
5f29877
@jakipatryk jakipatryk mentioned this issue Aug 30, 2022
Merged
@jakipatryk jakipatryk linked a pull request Aug 30, 2022 that will close this issue
#2121 Implement retries quadratic+randomised backoff mechanism #2122
Merged
@jakipatryk jakipatryk self-assigned this Sep 1, 2022
jakipatryk added a commit that referenced this issue Sep 1, 2022
@jakipatryk
#2121 Make randomization range growing linearly
754d651
jakipatryk added a commit that referenced this issue Sep 5, 2022
@jakipatryk
#2121 Implement retries backoff mechanism for CrossHostApiCaller
56d80f2
jakipatryk added a commit that referenced this issue Sep 5, 2022
@jakipatryk
#2121 Fix tests for RestDaoFactory
8811000
jakipatryk added a commit that referenced this issue Sep 5, 2022
@jakipatryk
#2121 Make randomization range growing linearly
ba23308
jakipatryk added a commit that referenced this issue Sep 15, 2022
@jakipatryk
#2121 Implement retries quadratic+randomised backoff mechanism (#2122)
c0c59db
@jakipatryk
Copy link
Collaborator

Release notes
dao: A random (quadratically + linearly randomized) wait has been added between retries of the same URL and between switching URLs on calling the REST API

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jakipatryk jakipatryk

Labels
Conformance Conformance Job affected feature New feature priority: medium Important but not urgent Standardization Standardization Job affected
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

#2121 Implement retries quadratic+randomised backoff mechanism AbsaOSS/enceladus
2 participants
@benedeki @jakipatryk