Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: test flaky #1113

Closed
wants to merge 1 commit into from
Closed

chore: test flaky #1113

wants to merge 1 commit into from

Conversation

sbor23
Copy link

@sbor23 sbor23 commented Jul 21, 2020

No description provided.

@@ -336,6 +336,7 @@ def test_complete_multiple_instance_task_form_work_item_next(
snapshot.assert_match(result.data)


@pytest.mark.flaky(10, 9)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean "at least 9 successes in 10 runs"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just wanted to try out if flaky helps us in any way... But doesn't seem to be the case, we don't want any flakiness.

@czosel
Copy link
Contributor

czosel commented Jul 21, 2020

Is fixing the actual flaky behavior not feasible?

@sbor23
Copy link
Author

sbor23 commented Jul 21, 2020

Is fixing the actual flaky behavior not feasible?

I don't think the code is flaky itself, because the tested result data seems fine to me.
Rather snapshottest is at fault.
It doesn't save to an OrderedDict but only to a dictionary.
And then the equality check doesn't like if the test result has a different ordering, even though it's shouldn't matter.

@sbor23 sbor23 closed this Jul 21, 2020
@sbor23 sbor23 deleted the flaky branch July 21, 2020 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants