Fix retry_exponential_backoff divide by zero error when retry delay is zero
#17003
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edwardwang888
requested review from
ashb,
kaxil,
turbaszek and
XD-DENG
as code owners
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
uranusjr
reviewed
Jul 15, 2021
edwardwang888
force-pushed
the
fix_retry_exponential_backoff_divide_by_zero
branch
from
15dbe99
to
48bd9dd
Compare
edwardwang888
changed the title
retry_exponential_backoff divide by zero error when retry delay is zeroretry_exponential_backoff divide by zero error when retry delay is zero
edwardwang888
force-pushed
the
fix_retry_exponential_backoff_divide_by_zero
branch
3 times, most recently
from
fef1c54
to
cec61c0
Compare
This test was incorrect before. When `delay.total_seconds()` equals 1 and `self.try_number` equals 1, `min_backoff` equals 1. Thus, `modded_hash` always equals 1, regardless of the value of `ti_hash`. So it should not be a period of 1 to 15 seconds, but just 1 second. Code referenced was: airflow/airflow/models/taskinstance.py Lines 1064 to 1087
The test can be reused as the end result is the same.
edwardwang888
force-pushed
the
fix_retry_exponential_backoff_divide_by_zero
branch
from
cec61c0
to
b5acb5b
Compare
potiuk
approved these changes
Sep 29, 2021
github-actions
bot
added
the
full tests needed
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
eladkal
approved these changes
Sep 29, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When
retry_delayis zero, theretry_exponential_backoffalgorithm has a divide-by-zero error in themodded_hashcalculation, which causes the scheduler to crash.d1dceff attempted to address the divide-by-zero error by adding a ceiling function, but it doesn't account for this particular edge case. Using a lower bound instead would be a better implementation.
Fixes #17005.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.