Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unknown error in child-process gem when trying to open phantomjs browser in webdriver when running on jruby #2598

Closed
Chuckv opened this Issue Feb 14, 2015 · 20 comments

Comments

Projects
None yet
3 participants
@Chuckv
Copy link

commented Feb 14, 2015

Originally reported at http://stackoverflow.com/q/23279945/409820 by a fellow using webdriver via watir-webdriver on windows 7. I was able to duplicate on windows 8 with both jruby 1.7.19 and 9.0.0.0.pre1. OTOH the two lines of IRB code below work just fine on Ruby 1.9.3, Ruby 2.0.0 and Ruby 2.1.5. Only using jruby seems to cause a problem.

Repro: (using watir-webdriver because frankly the repo steps are simpler and cleaner)

  1. download phantomJS executable from http://phantomjs.org/download.html and place in directory on path.
  2. gem install watir-webdriver
  3. In irb issue the following two commands:
require 'watir-webdriver'
b = Watir::Browser.new :phantomJS

Expected: will get a response such as

#<Watir::Browser:0x5a596fa8 url="about:blank" title="">

indicating webdriver has started a phantomjs browser session from which watir-webdriver creates a browser object (to see that the browser object works you could then do b.goto "google.com" followed by puts b.title which ought to return 'Google')

Actual: 'unknown error' with a stack trace as follows

C:\Rubies>irb
io/console not supported; tty will not be manipulated
irb(main):001:0> require 'watir-webdriver'
=> true
irb(main):002:0> b = Watir::Browser.new :phantomjs
ChildProcess::Error: Unknown error (Windows says "The operation completed successfully.", but it did not.)
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/lib.rb:325:in `handle_for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/jruby.rb:48:in `windows_handle_for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/lib.rb:306:in `handle_for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/process_builder.rb:137:in `std_stream_handle_for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/process_builder.rb:109:in `setup_io'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/process_builder.rb:32:in `start'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/windows/process.rb:68:in `launch_process'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/childprocess-0.5.5/lib/childprocess/abstract_process.rb:82:in `start'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver/phantomjs/service.rb:42:in `start'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver/phantomjs/bridge.rb:20:in `initialize'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver/common/driver.rb:45:in `for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver.rb:67:in `for'
    from C:/Rubies/jruby-1.7.19/lib/ruby/gems/shared/gems/watir-webdriver-0.6.11/lib/watir-webdriver/browser.rb:46:in `initialize'
    from (irb):2:in `evaluate'
    from org/jruby/RubyKernel.java:1107:in `eval'
    from org/jruby/RubyKernel.java:1507:in `loop'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from C:/Rubies/jruby-1.7.19/bin/jirb:13:in `(root)'irb(main):003:0> exit
@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

Hmm, I wonder if this ever worked. I'll poke around a bit, but if it's possible to narrow this down to just a childprocess call, it would speed things up a lot.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

Repro script...seems like either childprocess no longer works on JRuby on Windows, or it never did:

gem 'childprocess'
require 'child_process'
cp = ChildProcess.new('notepad.exe')
cp.start # errors
@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

Ok, I think there's two issues here.

The first is that we do not now (and may never have) provide a proper IO#fileno value for any streams on Windows. In 9k, the logic for getting the Windows "handle" instead of the FD seems to be missing. It may have been removed during 9k work. I have not checked 1.7.

This alone might not be enough of a fix, however, and based on how many places in our code expect fd to be an int (as on Unix) rather than a long (as in a Windows handle) I suspect something else regressed to cause childprocess to stop working.

cc @jarib

The logic in childprocess for getting a proper Windows handle on JRuby does not seem to work right. Given an IO, it digs out the java.nio.channels.Channel and then tries the following (in window_handle_for):

  • call #fileno on the original object. This ends up being the fallback value, which on JRuby is -1.
  • call getFDVal on the channel to get its FileDescriptor. However, this method is not visible when it is defined at all, so calling it normally from Ruby does not work.

The error that results is due to -1 bubbling through into a get_osfhandle (an FFI win32/posix bridge method I believe). If we fix the logic for getting the fileno/handle, the rest may start working properly again.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

I will try a hack to get fileno to return the Windows handle and see if that gets us farther.

@jarib

This comment has been minimized.

Copy link

commented Feb 23, 2015

Right - I basically gave up on this issue because I couldn't find a way to get the handle on Windows. If you can do it I'd be grateful.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

Sorry, I guess I never saw that issue even though you tagged me :-(

So am I getting that the FFI-based version has never worked on JRuby?

@jarib

This comment has been minimized.

Copy link

commented Feb 23, 2015

At least not with IO.pipe IIRC. It's been a while since I looked into this.

@headius headius closed this in 2e56c51 Feb 23, 2015

@headius

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

Ok, I think I figured out what broke and fixed it in 9k.

In JRuby 9k, we rewrote a lot of IO stuff, and in the process I believe I broke the "real" file descriptors for stdio channels. When we can use native IO, as on UNIX, it works fine; we have 0, 1, 2 as expected. However on Windows, where we can't (or are not yet) using native IO, we used to unwrap the given JDK streams until we found the FileChannel inside them, and then get the real fileno via reflection. I modified the code to always attempt to unwrap these channels, which appears to make childprocess work correctly (at least for the small example case I came up with).

@jarib Please give JRuby master a try with childprocess and work with me to fix any remaining issues.

@Chuckv Please confirm with JRuby master that things are working better for your case :-)

@headius headius added this to the 9.0.0.0.pre2 milestone Feb 23, 2015

@headius headius self-assigned this Feb 23, 2015

@Chuckv

This comment has been minimized.

Copy link
Author

commented Feb 24, 2015

Is this only fixed in 9 or is it fixed in 1.7 also?

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

@Chuckv I only fixed it in 9. It may be possible to fix it in 1.7 but I did not investigate that. You need it in 1.7.x, I'm guessing?

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

@Chuckv I just tested this with JRuby 1.7 HEAD and my short script worked ok. I tried to reproduce using your steps, and got the following error:

irb(main):002:0> b = Watir::Browser.new :phantomJS
ArgumentError: unknown driver: :phantomJS
        from C:/Users/headius/jruby17/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver/common/driver.rb:49:in `for'
        from C:/Users/headius/jruby17/lib/ruby/gems/shared/gems/selenium-webdriver-2.44.0/lib/selenium/webdriver.rb:67:in `for'
        from C:/Users/headius/jruby17/lib/ruby/gems/shared/gems/watir-webdriver-0.6.11/lib/watir-webdriver/browser.rb:46:in `initialize'
@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

Ok, nevermind...it's :phantomjs not :phantomJS, and I was able to reproduce under 1.7. It looks like it may be a similar issue, but caused instead by us reporting typical stdio descriptors rather than actual stdio descriptors (e.g. $stdin.fileno was 3 under the fixed 9k but is 0 under 1.7 HEAD).

@headius headius reopened this Feb 24, 2015

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

Original reported repro still fails under 9k. Continuing investigation.

@jarib

This comment has been minimized.

Copy link

commented Feb 24, 2015

I don't have access to a Windows machine to test this at the moment, but if you have suggestions for better approaches childprocess could use to do this, I'm of course happy to make the necessary changes.

@Chuckv

This comment has been minimized.

Copy link
Author

commented Feb 24, 2015

Jari, you can download VM images from MS for most combinations of Platform, WinOS, IEversion from microsoft at: https://www.modern.ie/en-us/virtualization-tools#downloads

Or if you need something that will not eventually expire, I might have a spare WinXP and or Win7 hanging around that I could donate your way. Drop me an emailwith your address. Least Ican doafter all you have done for Watir, Webdriver, etc

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

Ok, I've run into a snag. Streams created by the JDK on Windows do not appear to have filenos attached to them...only handles.

This is not to say that file descriptors aren't being used... doing a native read from fileno 0 works properly. But the streams we get back from the JVM do not have a fileno inside them, as @jarib's code was attempting to locate. They only appear to populate handle inside the JDK FileDescriptor class.

This may mean it's not possible for us to get the proper fileno for streams not created via native APIs (which is all streams on Windows right now). We could provide the handle, but probably not the fileno.

I'm trying to determine if MRI uses file descriptor APIs (_open, _read, etc) or if it uses the HANDLE APIs like _open_osfhandle to make file descriptors from them.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

Well I have a fix to report 0, 1, 2 for the stdio streams, but that's not going to help this issue. The stream that's failing in the original example is likely the pipe stream, and on JDK that too appears to have only handle and no fileno.

I think we're going to have to punt this one...the only way I could fix this to work with childprocess's logic would be to switch ALL IO logic in JRuby to use the native posix calls rather than the JDK's streams, so we actually get fileno and not just handle.

It should be possible, however, to fix this in childprocess; I can work with @jarib to dig out the handle correctly on JRuby/JVM.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

I have some interesting news.

It seems that with all my fixes in place plus a modification to how we handle the "nul:" device, @Chuckv's original example works correctly on JRuby 9k.

The stream I see failing without latest fix is one opened by phantomjs to the null device, nul: on Windows. JRuby has since 2008 special-cased /dev/null and nul: to be a bogus "NullChannel" that does not have a file descriptor or handle or anything. By removing that logic and allowing "nul:" to actually be a real stream, the win32 handle can be dug out just fine.

I'm going to explore what else this would break, but it seems like an ok change. We probably would not want to backport this to 1.7.x unless we can prove that going back to a real FileChannel versus NullChannel would not cause any unexpected consequences.

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

With the fixes I've landed on the test-windows-9k branch, the original example appears to work correctly. I'm not sure this is a fix we can make in the 1.7.x timeframe, but we can discuss that.

I still have work to do to make these changes green on *nix, but I'm going to close this and work on that separately.

@headius headius closed this Feb 24, 2015

@headius

This comment has been minimized.

Copy link
Member

commented Feb 24, 2015

The fixes have landed on master after some additional tweaks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.