Reduce Travis flakiness #5930

Closed
nolanlawson opened this Issue Nov 24, 2016 · 6 comments

Projects

None yet

2 participants

@nolanlawson
Member

Based on an informal audit of Travis builds (i.e. me pointing and clicking to restart tests for the past few days), our most faily tests seem to be iPhone and IE. If we can find ways to make these tests less likely to fail I would be super happy.

Some stuff I'm trying out:

  1. Reducing memory usage (#5926, #5921)
  2. Testing out different browser/OS combos (#5928)

I would really love if our test suite were reliably green/red rather than being nearly guaranteed to have a false negative on any given run.

@nolanlawson
Member

Occasionally I also see No output has been received in the last 10m0s for the Firefox builds. I believe this is due to not being able to find the installed Firefox 48.0.

@nolanlawson nolanlawson added a commit that referenced this issue Nov 24, 2016
@nolanlawson nolanlawson (#5930) - use yarn 44140db
@nolanlawson
Member

Might be related to #5898, but IE10+Win8.1 tests are super flaky these days. I might have to deep-dive to see what's up.

@nolanlawson
Member

Wow, our build is really trolling us right now. On a typical run I have to restart like 6 jobs.

@nolanlawson
Member

OK, I'm fairly certain 4eb786b regressed us on IE10. Tests rarely seem to want to pass for that browser now.

@daleharvey
Member

Looking into this now, firefox seems far more faily than others, however I am seeing a fair amount of fails across the board as well so doing a large travis run @ #5967 and once we know / can get node stable then ill look at firefox

@nolanlawson
Member

I think it's reasonable to close this now. Done a lot of work here recently.

@nolanlawson nolanlawson closed this Dec 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment