Remove commented out pytest-rerunfailures test dependency

[pabs3]

pytest-rerunfailures is now compatible with pytest 6.1.0 and later.

Reverts: commit 8687e7f.
Reported-in: pytest-dev/pytest-rerunfailures#128
Fixed-by: pytest-dev/pytest-rerunfailures#129

[codecov]

Codecov Report

Merging #3263 (c43b365) into develop (6fc9e38) will not change coverage.
The diff coverage is n/a.

❗ Current head c43b365 differs from pull request most recent head 6f7e9f5. Consider uploading reports for the commit 6f7e9f5 to get more accurate results

@@           Coverage Diff            @@
##           develop    #3263   +/-   ##
========================================
  Coverage    79.53%   79.53%           
========================================
  Files           68       68           
  Lines        11781    11781           
========================================
  Hits          9370     9370           
  Misses        2411     2411           

Impacted Files	Coverage Δ
gensim/models/doc2vec.py	81.61% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6620df...6f7e9f5. Read the comment docs.

[mpenkov]

What's the benefit of re-running failures?

In my opinion, if a test keeps blinking, it's probably a bad test. We should improve the test so that it does not blink instead of re-running it and hoping that we get lucky on one of the attempts.

@pabs3 What are your thoughts?

[pabs3]

I'm not sure about the benefit, I just figured it was used for a reason before and only got disabled due to a bug somewhere else and that bug got fixed so now it could get enabled again. What you say sounds correct to me. If it isn't useful then I can turn this PR into a commit dropping the commented out dependency.

…

-- bye, pabs https://bonedaddy.net/pabs3/

[mpenkov]

@menshikh-iv @piskvorky What do you guys think? Can we get rid of this and run tests once only?

[piskvorky]

Do we still experience flaky tests? How bad is it – if not bad, then no reason to automate "re-runs" IMO.

[mpenkov]

I don't think it's bad. One of the wheel builds fails occasionally, but there are ways around that other than automatic re-runs (plus the re-runs don't always help).

[menshikh-iv]

If I remember correctly, rerun plugin help us to avoid "blinking" test failures, if we didn't see these tests now - we can just don't use this plugin.

there are ways around that other than automatic re-runs

@mpenkov just curious, would you mind to add details?

[mpenkov]

In order of decreasing preference, we could:

• Eliminate the external factors that cause the test to blink (are we forgetting to mock or otherwise nail down something?)
• Rewrite the test so that it does not blink (why do these test blink, anyway? Where are we dealing with non-deterministic code?)
• Mark the test as an expected failure under certain conditions
• Disable the test
• Remove the test

[gojomo]

Personally, I'd rather we default to no automatic rerunning of tests. It slows down detection of reliable failures, & might hide other stability issues better solved another way.

[pabs3]

The consensus seems to be to not re-add pytest-rerunfailures, so I have added a commit removing the commented out dependency.

[pabs3]

Can someone re-run the checks? Seems like one of them timed out and another I cannot tell why the failure happened.

[piskvorky]

The wheel-build failures are likely unrelated. @mpenkov is looking into it – thanks for your PR!
The new release is coming soon now :)

[mpenkov]

Merged. Thank you @pabs3 !