Tests: Enable applab/scenarios.feature on circle #19128
Conversation
|
Test run 54437 re-enabled scenarios.feature on Circle and on Firefox, and ran tests against ChromeLatest, Chrome44, IE, Firefox, and Safari. It had the following failures:
Good news: scenarios.feature passed on:
No evidence of problems with scenarios.feature, but it looks like it failed before running / re-running all tests. Running five browsers doesn't work so well. I'll try again with just ChromeLatest and Firefox. |
islemaster
force-pushed
the
enable-applab-scenarios-circle
branch
from
b3fb511
to
0e37996
Compare
|
Test run 54487 had the following failure:
And scenarios.feature didn't complete on either browser before the build failed with So... not sure what causes that. Going to re-run just to see if anything changes or we have a stable failure. |
|
Hi @islemaster , I think that comment of mine may be erroneous -- AFAIK we aren't using selenium-docker. The note in #11908 says the other test disabled for this reason causes 7% of circle builds to fail, and IIRC the instance you are fixing here was causing 4% of circle builds to fail. Agreed, given the new circle platform, let's re-enable and see what happens. |
|
Build 54517 fails with a merge conflict, so no useful data. The test in question has been disabled on IE for now, I'm going to work on re-enabling it and then return to this work. |
…fox] [test chrome 44] [test safari]
islemaster
force-pushed
the
enable-applab-scenarios-circle
branch
from
0e37996
to
df0dc97
Compare
|
Build 56923 had the following failures:
I'm not going to bother re-enabling this on IE just yet - I spent some time debugging yesterday and have no idea what's going on. That shouldn't block us from enabling this on Circle though. Going to do another run without IE11Win10. |
|
Build 56970 passed on Chrome, Firefox, Safari 🎉. Running a couple more trials. |
|
Build 57061 passed on Chrome, Firefox, Safari. Running one more trial... |
islemaster
changed the title
|
Sweet, build 57149 passed too. That's three in a row passing on Circle, with Firefox. I think that justifies re-enabling this. Review time! |
We had a UI test failure on the test branch this morning that didn't get caught earlier because
applab/scenarios.featureis configured not to run on Circle.When I went to take a look, I found it's also disabled on Firefox, and the following note:
The note was added by @davidsbailey in November of 2016. The issue in question is closed, but it looks like it has been closed since before Dave added the note and I'm not totally clear on whether progress has been made. That said, our situation re: Docker has changed significantly since our switch to Circle 2.0 so I figured it's worth just trying to re-enable this test.