Conversation
I think this one should finally fix the flaky test. The issue with it was that in Pytest parsing the test might occur a long time before executing it - depending how many tests are run between and how slow the test run. The original test calculated the reference time at parsing time and it assumed on hour ahead (when minutes where 0-ed were enough) for the "deferral" to happen. But if parsing was executed (thus reference time calculation happened) before end of hour and execution after end hour, then the +1 hour from beginning of the reference hour already passed. The fix is to move the reference time calculation to inside the test, so that it is not calculated a long time before the test is run. The test runs on public runners are generally slower "per container", that's why it happened more frequently on public runners.
|
I believe I found out why the async parse test was flaky finally @hussein-awala ? |
hussein-awala
left a comment
hussein-awala
left a comment
There was a problem hiding this comment.
I had this hypothesis yesterday but the date looked good in the failed ci job, but yeah what you describe completes my analysis.
|
Yeah. The tests in "Other" run for ~ 11 minutes and this one was somewhere around 90% , so chances to hit this one were quite significant. Plus - when it happened it happend in most of the tests because they were starting around the same time + it seems that for scheduled build in the night it happened always, because the workflow starts at 0:40 and it takes about 10 minutes to start the tests from starting of the CI workflow. 🤯 |
|
And I think this is what confused me with my initial attempt about But in this case it was the TIME that shifted 😱 |
ephraimbuddy
added
the
changelog:skip
I think this one should finally fix the flaky test. The issue with it was that in Pytest parsing the test might occur a long time before executing it - depending how many tests are run between and how slow the test run. The original test calculated the reference time at parsing time and it assumed on hour ahead (when minutes where 0-ed were enough) for the "deferral" to happen. But if parsing was executed (thus reference time calculation happened) before end of hour and execution after end hour, then the +1 hour from beginning of the reference hour already passed. The fix is to move the reference time calculation to inside the test, so that it is not calculated a long time before the test is run. The test runs on public runners are generally slower "per container", that's why it happened more frequently on public runners.
4 participants
I think this one should finally fix the flaky test. The issue with it was that in Pytest parsing the test might occur a long time before executing it - depending how many tests are run between and how slow the test run.
The original test calculated the reference time at parsing time and it assumed on hour ahead (when minutes where 0-ed were enough) for the "deferral" to happen. But if parsing was executed (thus reference time calculation happened) before end of hour and execution after end hour, then the +1 hour from beginning of the reference hour already passed.
The fix is to move the reference time calculation to inside the test, so that it is not calculated a long time before the test is run.
The test runs on public runners are generally slower "per container", that's why it happened more frequently on public runners.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.