Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

Closed
bigsur0 opened this issue Oct 16, 2015 · 16 comments
Closed

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

bigsur0 opened this issue Oct 16, 2015 · 16 comments
Assignees
Milestone

Comments

@bigsur0
Copy link
Contributor

@bigsur0 bigsur0 commented Oct 16, 2015

Error case followed by success case below. This issue breaks Sinatra AssetPack running on JRuby 9.0.1.0 when the Sinatra "root" directory is set using the classpath: scheme.

$ mkdir /tmp/classpath_seek
$ cd !$
$ rvm use jruby-9.0.1.0
$ touch foo.rb
$ irb
>> $CLASSPATH <<  Dir.pwd
=> ["file:/Users/me/.m2/repository/jline/jline/2.11/jline-2.11.jar", "file:/Users/me/.rvm/rubies/jruby-9.0.1.0/lib/ruby/stdlib/readline.jar", "file:/private/tmp/classpath_seek/"]
>> f = File.open('classpath:foo.rb')
=> #<File:classpath:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - classpath:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):4:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

>> f = File.open(File.join(Dir.pwd, 'foo.rb'))
=> #<File:/private/tmp/classpath_seek/foo.rb>
>> f.seek(0)
=> 0
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 16, 2015

The same problem exists for the uri:classloader scheme.

>>  f = File.open('uri:classloader:foo.rb')
=> #<File:uri:classloader:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):6:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'


>> f = File.open('uri:classloader://foo.rb')
=> #<File:uri:classloader://foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader://foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):34:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'
@bigsur0 bigsur0 changed the title Can't seek when file accessed using classpath: scheme Can't seek when file accessed using classpath: OR uri:classloader: schemes Oct 16, 2015
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 16, 2015

Same issue via this route (foo.rb in a jar):

>> f = File.open('jar:file:/Users/me/foo.jar!/foo.rb')
>> f.seek(0)
Errno::EPIPE: Broken pipe - jar:file:/Users/me/foo.jar!/foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):13:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 16, 2015

And here's a full backtrace of how we first encountered the issue, to give you a sense of how impactful this issue is.


Errno::EPIPE - Broken pipe - uri:classloader://app/assets/javascripts/models/component.js:
    org/jruby/RubyIO.java:1649:in `seek'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:110:in `block in each'
    org/jruby/RubyIO.java:1126:in `open'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:109:in `each'
    uri:classloader:/gems/rack-1.5.2/lib/rack/response.rb:36:in `initialize'
    uri:classloader:/gems/rack-1.5.2/lib/rack/mock.rb:161:in `initialize'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/mock_session.rb:32:in `request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:230:in `process_request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:57:in `get'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:109:in `block in combined'
    org/jruby/RubyArray.java:2300:in `map'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:108:in `combined'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:94:in `minify'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:32:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    uri:classloader:/gems/tilt-1.4.1/lib/tilt.rb:127:in `fetch'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:31:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    org/jruby/RubyMethod.java:111:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1603:in `block in compile!'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:985:in `route_eval'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1006:in `block in process_route'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1004:in `process_route'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:964:in `block in route!'
    org/jruby/RubyArray.java:1560:in `each'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:963:in `route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1076:in `block in dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1073:in `dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `block in call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:886:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/nulllogger.rb:9:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:180:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:2014:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `block in call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1788:in `synchronize'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:490:in `handle_request'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:361:in `process_client'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:254:in `block in run'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/thread_pool.rb:92:in `block in spawn_thread'
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 16, 2015

Confirmed that the same is true w/ JRuby 9.0.0.0

@headius
Copy link
Member

@headius headius commented Oct 19, 2015

Ok, I'm not sure the best way to handle this. Did you say it works on 1.7?

The stream we get from the in-jar files is a JarURLInputStream. The only public ancestor of that is FilterInputStream, which doesn't support seeking (it supports mark/reset but I'm not sure that's enough).

The JarURLInputStream appears to wrap a ZipFileInputStream or ZipFileInflaterInputStream. They don't support seeking either.

What is the use case you're trying to solve? It may be simpler for you to just read in-jar files into a binary String and then use StringIO to simulate random access.

@headius headius added this to the JRuby 9.0.2.0 milestone Oct 19, 2015
@headius headius self-assigned this Oct 19, 2015
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 19, 2015

Basically, Rack will read the file and seek within it, see stack trace above. So we aren't actually doing the seeking Rack is doing it transitively. On 1.7.19 this doesn't happen via the jar::file path that was produced via File.expand_path.

@headius
Copy link
Member

@headius headius commented Oct 19, 2015

@r6p Ok, I think I figured it out. Apparently I determined at one point that only selectable channels (e.g. sockets, pipes) should raise errors when you try to seek on them. I'll see about making the same change to 9k.

@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 19, 2015

Any chance for a 9.0.1.1 release w/ this fix?

@headius
Copy link
Member

@headius headius commented Oct 19, 2015

9.0.1.1 would be a security fix release. This will go into 9.0.2.0 shortly.

@headius headius closed this in a4cc5ee Oct 19, 2015
@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 19, 2015

Great thanks. Let me know if you want me to verify against a build from HEAD if that is helpful.

@headius
Copy link
Member

@headius headius commented Oct 19, 2015

@r6p Verification is always welcome :-)

@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 20, 2015

@headius I just verified with jruby-head at 4e05196. Many thanks for turning this around so quickly. Do you have a rough ETD for a 9.0.2.0 release?

@headius
Copy link
Member

@headius headius commented Oct 20, 2015

We're working on pushing out a release today.

@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 20, 2015

@headius that's awesome. Many thanks!

@bigsur0
Copy link
Contributor Author

@bigsur0 bigsur0 commented Oct 21, 2015

@headius thanks again! I just verified using the public v9.0.3.0 release.

@headius
Copy link
Member

@headius headius commented Oct 22, 2015

@r6p Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.