Skip to content

Add selenium with chrome to the test suite #979

Merged
merged 3 commits into from Nov 18, 2013

4 participants

@twalpole
Collaborator

@jnicklas - This PR adds selenium with chrome to the tests - There is one test failure currently

Failure/Error: expect do
expected TestApp::TestAppError but nothing was raised
# ./lib/capybara/spec/session/reset_session_spec.rb:36:in `block (2 levels) in '

This doesnt occur if just the chrome specs are run, or just the firefox, but if both are run the error occurs. I'm 99% sure this is because of the way Capybara::Server caches and looks up the port for server instances by object_id, meaning that even though I create a separate test session for chrome which creates a server object, that server object doesnt actually get booted (doesnt error either), and instead the tests are hitting the server object created for the firefox session. This doesnt make any difference for most of the tests except the one failing since it checks the internal error state of the server object it thinks is handling requests, but ends up not. Not sure what the best way to fix that error would be, any suggestions?

@jnicklas
Owner

This effectively doubles the run time of our test suite. Is that worth it? It's ridiculously long as is.

@twalpole
Collaborator

@jnicklas my end goal was to just have travis default to run a few tests on chrome that firefox has to be pending on ( specifically hover currently). While working towards that I hit upon the issue I described with the sessions, and figured that should be fixed.

@jnicklas
Owner

Yes, sounds like we should fix that. But let's hold off on adding Chrome to the test suite. I think there is a cost in discouraging contribution if the test suite is prohibitively slow.

Maybe it might be possible to have an extended suite on Travis CI? Add Chrome to the suite only on Travis?

@jnicklas
Owner

(I fixed the build on Travis btw... yay!)

@twalpole
Collaborator

@jnicklas ok, chrome tests are now only run for the travis rake task and I fixed the issue with the second server instance not booting. However now the rbx tests are failing due to timeouts, not sure why - any ideas you can provide would be appreciated

@abotalov
Collaborator
abotalov commented Mar 4, 2013

Native events are disabled by default on Firefox on Linux. I think that's why tests are failing on Firefox. To make tests not failing we may set profile.native_events = true but there may be problems with several simultaneously opened windows

@twalpole
Collaborator
twalpole commented Mar 4, 2013

@abotalov there's also no native event support on OSX and the selenium hover support has positional issues on Firefox too, so the native events are not the only reason for issues on Firefox.

@TimMoore

@twalpole is this still working for you? When I merge it into the latest master and try to run it (with Chrome 26 on Mac OS 10.8) I get a lot of timeouts waiting for Chrome to load pages:

  Timeout::Error: Timeout::Error
  occurred at /Users/tmoore/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/protocol.rb:146:in `rescue in rbuf_fill'

  1) Capybara::Session selenium_chrome #all with xpath selectors should find the first element using the given locator
     Failure/Error: @session.visit('/with_html')
     Timeout::Error:
       Timeout::Error
     # ./lib/capybara/selenium/driver.rb:43:in `visit'
     # ./lib/capybara/session.rb:193:in `visit'
     # ./lib/capybara/spec/session/all_spec.rb:3:in `block (2 levels) in <top (required)>'

etc.

@twalpole
Collaborator

@TimMoore - havent tried it in a while -- I know there were issues with the latest versions of chrome and chromedriver

@twalpole
Collaborator

@TimMoore Just tried it - same issues you're having with it just hanging when using Chrome 26.0.1410.65 and chromedriver 26.0.1383.0 . Strangely if one removes the
@browser.navigate.to('about:blank')
from Capybara::Selenium::Driver#reset! then the tests work fine (other than the reset_session tests obviously). It seems that either Chrome or chromedriver may be having an issue with changing the URL rapidly or something.

@TimMoore

I found this issue, which seems to describe the symptoms I see: https://code.google.com/p/chromedriver/issues/detail?id=23

@TimMoore

BTW, the quickest way I've found to reproduce the problem is by running:

bundle exec rspec spec/selenium_spec_chrome.rb -e'#fill_in'

That hangs consistently for me on the fourth test.

I also tried adding simple access logging to Capybara::Server::Middleware and can see that the hanging requests don't reach the server at all.

@twalpole
Collaborator

Yeah, it definitely seems to be some kind of race condition in either chrome or chromedriver, guess it could be in selenium too. Does not appear to be in Capybara

@jnicklas
Owner

Whatever happened to this issue? Our build time on Travis is 11 minutes, well under the cut off at 50 minutes, so we should be able to add a Chrome run without problem. @twalpole did you have this working? Care to bundle it up in a pull request?

@twalpole
Collaborator

@jnicklas I'll see about getting the PR updated to latest code

@twalpole
Collaborator

It appears theres a known issue with trying to run chrome on the OpenVZ setups that travis uses

@twalpole
Collaborator

@jnicklas ok this PR works again, and will run the tests against chrome as well as firefox on travis

@jnicklas
Owner

Okay merging this. At some point we'll need to do some serious debugging why Travis seems to hate JRuby and rbx.

@jnicklas jnicklas merged commit 1b73f77 into jnicklas:master Nov 18, 2013

1 check failed

Details default The Travis CI build could not complete due to an error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.