You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
RunnableSummary.is_intermittent should return None when we don't have enough info to tell for sure (a bit like we do for cross-config and config-consistent failures).
The prior regressions detection logic (
# ...and it failed permanently in the first parent where it ran, it is a
# prior regression.
# Otherwise, if it passed or was intermittent, it is likely not a prior
# regression.
if (
summary.status!=Status.PASS
andnotsummary.is_intermittent
):
prior_regression=True
break
# If the failure is intermittent and it failed intermittently in a close
# parent too, it is likely a prior regression.
# We need to explore the parent's parents instead if it passed or failed
# consistently in the parent.
elifsummary.is_intermittent:
prior_regression=True
break
) should take None into account and handle all possible cases: parent None child None, parent intermittent child intermittent, parent real child real, parent None child intermittent, etc..
E.g. in a case like this:
def test_intermittent_failure_and_consistent_in_parent(create_pushes):
"""
Tests the scenario where a task failed intermittently in a push, and failed
consistently in its parent.
"""
p = create_pushes(3)
i = 1 # the index of the push we are mainly interested in
p[i - 1].tasks = [
create_task(
id=fake_id(1),
label="test-prova",
result="failed",
classification="not classified",
),
]
p[i].tasks = [
create_task(
id=fake_id(2),
label="test-prova",
result="failed",
classification="not classified",
),
create_task(
id=fake_id(3),
label="test-prova",
result="passed",
),
]
How do we know if the task wouldn't have failed intermittently in the parent push too, had it had a chance to run multiple times? We should consider this case as a prior regression (assuming we have no classifications and no backout information here, the situation is different in case we have them).
The text was updated successfully, but these errors were encountered:
RunnableSummary.is_intermittent should return None when we don't have enough info to tell for sure (a bit like we do for cross-config and config-consistent failures).
The prior regressions detection logic (
mozci/mozci/push.py
Lines 964 to 988 in 944248b
E.g. in a case like this:
How do we know if the task wouldn't have failed intermittently in the parent push too, had it had a chance to run multiple times? We should consider this case as a prior regression (assuming we have no classifications and no backout information here, the situation is different in case we have them).
The text was updated successfully, but these errors were encountered: