tests: retry some jest tests on failure

[connorjclark]

These are too flaky. This should help.

[connorjclark]

https://github.com/facebook/jest/pull/10678#issuecomment-734296283https://github.com/facebook/jest/pull/10678#issuecomment-734296283

[brendankenny]

Has anyone looked into why these are flaky? Does #12284 help?

[connorjclark]

Has anyone looked into why these are flaky? Does #12284 help?

havent looked. searching my memory of n=1 data points, yarn test-clients fails because the page crash. so upgrading chrome might have an impact there.

[adamraine]

What about?

$* || $* --onlyFailures

[connorjclark]

i'm gonna retry the actions a few times and see how many times it passes

(this PR only impacts some of the unit tests, not smokes...)

• passed
• failed (smoke test offline)
• failed (smoke test offline)
• failed (unit-cli Unable to connect to Chrome; smoke test offline)
• ...

[brendankenny]

we haven't landed #12284 yet :P

[connorjclark]

I'm not talking about puppeteer, I'm curious if this PR does anything.

I hit a wall with #12284 – next action item is a Chrome bisect but I'm strapped for bandwidth for another couple weeks, so I can't move forward there at the moment.

[connorjclark]

I haven't seen the unit tests fail here yet; they continue to fail elsewhere (look at latest master CI run). should be an improvement to merge this. Can investigate if still needed even after puppeteer upgrade (altho, I think some failures are not related to puppeteer, I'd have to check again..)

[brendankenny]

should be an improvement to merge this

The failures are real annoying, so SGTM if this was waiting on me :) I'd really like us to not wave off the failures as expected flakiness because I think they've definitely increased in rate recently (it would be nice if Github Actions tracked this), but I don't think there's any problem waiting for the #12284 investigation.

[brendankenny]

jest 27 is getting a new jest.retryTimes(55) that can be added in any test that would also be a nice version of this, but I didn't find an obvious roadmap, so I'm not sure when that's coming.

[connorjclark]

so SGTM if this was waiting on me :)

was actually waiting for master to be unbroked.

now, it seems windows in GH Actions doesn't use environment variables like the other environments...

[brendankenny]

now, it seems windows in GH Actions doesn't use environment variables like the other environments...

@paulirish mentioned that xvfb-run will (may?) work with inlined yarn test-viewer || yarn test-viewer --onlyFailures, which avoid the script indirection step and sidestep the windows issue

[connorjclark]

last failure was b/c of ESOCKETTIMEDOUT in smoke test yarn install :/