We have one skipped test that sometimes fails on travis but doesn't seem to fail locally, and there's another one that is NOT currently skipped that has the same behavior. It's this one:
Capybara::ElementNotFound: no button with value or id or text 'unfollow-4f2c40089870160d4d000947' found
See these failing builds to see the test(s?) I'm talking about:
There's pretty much no way the changes in those builds could have caused those tests to fail, so there's something else going on here.
It's really getting annoying, so if anyone figures out what is going on here and fixes it, it would be VERY much appreciated.
I skipped the other one too.
UGH it looks like being able to find the delete link that has the text "I regret this" fails sometimes as well. I just skipped 2 more tests in acceptance/update_test.rb.
See Travis builds #129 and #135; those changes had nothing to do with being able to find the delete link.
Ok, there are more of these now, I've had it.
The new failures, and most of the old ones, appear to be tests that make external requests that would pass if you were using a previously-successful VCR cassette. That would explain why the failures are transient-- sometimes the external requests work and sometimes they don't.
I'm thinking about checking in the VCR cassettes I have in order to solve this problem. This introduces the possibility that we won't be notified through a test failure that an external site has changed, but I think the tradeoff of getting more stable test runs would be worth it.... Anyone have any thoughts before I do this?
One of the tests that we've skipped for a while does not use VCR; I'm trying a few travis runs that print out more of what the travis tests are seeing in my repo to try and figure that one out.
Adding the VCR cassettes seems to have helped some, but there are still 2 tests that fail a lot (but not every time) on travis, and never locally.
For serious, if anyone can fix this, name your reward. I will buy you coffee or beer. Or chocolate. Or pizza. Or an edible bouquet of fruit.
Which tests are still broken on this? Any?
There are four pending tests that are marked with "skip" (they show up as "S" when you run the tests right now).
I actually saw a NEW one that is NOT skipped currently the other day on travis here: http://travis-ci.org/#!/hotsh/rstat.us/jobs/2509375
Add heisenbug logging that prints to STDOUT in the cases we know of
That way, we can get a little more information when these tests fail on Travis, which appears to be the only place they ever fail. See notes in Issue #479.
@carols10cents I'm trying to reproduce this bug and I'm not seeing the skips when running rake test. I'm likely doing something wrong here. Any suggestions?
Just saw this. https://gist.github.com/jrgifford/5070939
Reporting as requested: https://gist.github.com/twopoint718/6373628
Encountered : https://gist.github.com/railsaholic/7184109