Fix test flake #10083
Fix test flake #10083
Conversation
| @@ -30,7 +30,7 @@ if (!window.validatorLoad) { | |||
| })(); | |||
| } | |||
|
|
|||
| describe.configure().retryOnSaucelabs().run('example', function() { | |||
| describe.skip('example', function() { | |||
There was a problem hiding this comment.
Why skip all of these? Can we fix the failing example to validate or remove it from the list? If flaky on SauceLabs, I think a .skipSauceLabs() would be great to have especially for tests like this that don't really need cross-browser testing. ( as far as I remember, we run all tests including integration on Travis using latest Chrome in addition to running integration on SauceLabs, @rsimha-amp is still still the case > )
There was a problem hiding this comment.
If we skip sauce labs, when will these ever be run?
There was a problem hiding this comment.
Just verified that gulp test (which runs on Travis with latest Chrome as part of the pre-submit and unit test job) does indeed run integration tests. (it runs test/**/*)
There was a problem hiding this comment.
@aghassemi, here is where the unit tests are run for PRs and on master. We used to run using old chrome on saucelabs, but no longer do.
I didn't think we ran integration tests with gulp test, unless the --integration arg was passed in, so your comment above comes as a surprise to me. (See this.)
There was a problem hiding this comment.
gulp test uses testPaths which is test/**/*. We definitely want to keep running integration tests on Travis with latest Chrome, so maybe the job can be renamed from presubmit and unit tests to something like presubmit and all tests on latest Chrome and the integration tests job can become integrations tests on SauceLabs
There was a problem hiding this comment.
@aghassemi, fair enough. I'll review pr-check.js and do some clean up.
|
@aghassemi approval? 😛 |
* Remove video integration iframes * Skip flakey development errors integration tests * Only run vis-state tests on chrome * Skip more flakey development integration tests * fix * skipSauceLabs
* Remove video integration iframes * Skip flakey development errors integration tests * Only run vis-state tests on chrome * Skip more flakey development integration tests * fix * skipSauceLabs
Fixes #10053.