New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

Closed
r6p opened this Issue Oct 16, 2015 · 16 comments

Comments

Projects
None yet
2 participants
@r6p
Contributor

r6p commented Oct 16, 2015

Error case followed by success case below. This issue breaks Sinatra AssetPack running on JRuby 9.0.1.0 when the Sinatra "root" directory is set using the classpath: scheme.

$ mkdir /tmp/classpath_seek
$ cd !$
$ rvm use jruby-9.0.1.0
$ touch foo.rb
$ irb
>> $CLASSPATH <<  Dir.pwd
=> ["file:/Users/me/.m2/repository/jline/jline/2.11/jline-2.11.jar", "file:/Users/me/.rvm/rubies/jruby-9.0.1.0/lib/ruby/stdlib/readline.jar", "file:/private/tmp/classpath_seek/"]
>> f = File.open('classpath:foo.rb')
=> #<File:classpath:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - classpath:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):4:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

>> f = File.open(File.join(Dir.pwd, 'foo.rb'))
=> #<File:/private/tmp/classpath_seek/foo.rb>
>> f.seek(0)
=> 0
@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 16, 2015

Contributor

The same problem exists for the uri:classloader scheme.

>>  f = File.open('uri:classloader:foo.rb')
=> #<File:uri:classloader:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):6:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'


>> f = File.open('uri:classloader://foo.rb')
=> #<File:uri:classloader://foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader://foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):34:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'
Contributor

r6p commented Oct 16, 2015

The same problem exists for the uri:classloader scheme.

>>  f = File.open('uri:classloader:foo.rb')
=> #<File:uri:classloader:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):6:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'


>> f = File.open('uri:classloader://foo.rb')
=> #<File:uri:classloader://foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader://foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):34:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

@r6p r6p changed the title from Can't seek when file accessed using classpath: scheme to Can't seek when file accessed using classpath: OR uri:classloader: schemes Oct 16, 2015

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 16, 2015

Contributor

Same issue via this route (foo.rb in a jar):

>> f = File.open('jar:file:/Users/me/foo.jar!/foo.rb')
>> f.seek(0)
Errno::EPIPE: Broken pipe - jar:file:/Users/me/foo.jar!/foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):13:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'
Contributor

r6p commented Oct 16, 2015

Same issue via this route (foo.rb in a jar):

>> f = File.open('jar:file:/Users/me/foo.jar!/foo.rb')
>> f.seek(0)
Errno::EPIPE: Broken pipe - jar:file:/Users/me/foo.jar!/foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):13:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'
@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 16, 2015

Contributor

And here's a full backtrace of how we first encountered the issue, to give you a sense of how impactful this issue is.


Errno::EPIPE - Broken pipe - uri:classloader://app/assets/javascripts/models/component.js:
    org/jruby/RubyIO.java:1649:in `seek'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:110:in `block in each'
    org/jruby/RubyIO.java:1126:in `open'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:109:in `each'
    uri:classloader:/gems/rack-1.5.2/lib/rack/response.rb:36:in `initialize'
    uri:classloader:/gems/rack-1.5.2/lib/rack/mock.rb:161:in `initialize'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/mock_session.rb:32:in `request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:230:in `process_request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:57:in `get'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:109:in `block in combined'
    org/jruby/RubyArray.java:2300:in `map'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:108:in `combined'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:94:in `minify'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:32:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    uri:classloader:/gems/tilt-1.4.1/lib/tilt.rb:127:in `fetch'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:31:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    org/jruby/RubyMethod.java:111:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1603:in `block in compile!'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:985:in `route_eval'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1006:in `block in process_route'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1004:in `process_route'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:964:in `block in route!'
    org/jruby/RubyArray.java:1560:in `each'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:963:in `route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1076:in `block in dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1073:in `dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `block in call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:886:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/nulllogger.rb:9:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:180:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:2014:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `block in call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1788:in `synchronize'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:490:in `handle_request'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:361:in `process_client'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:254:in `block in run'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/thread_pool.rb:92:in `block in spawn_thread'
Contributor

r6p commented Oct 16, 2015

And here's a full backtrace of how we first encountered the issue, to give you a sense of how impactful this issue is.


Errno::EPIPE - Broken pipe - uri:classloader://app/assets/javascripts/models/component.js:
    org/jruby/RubyIO.java:1649:in `seek'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:110:in `block in each'
    org/jruby/RubyIO.java:1126:in `open'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:109:in `each'
    uri:classloader:/gems/rack-1.5.2/lib/rack/response.rb:36:in `initialize'
    uri:classloader:/gems/rack-1.5.2/lib/rack/mock.rb:161:in `initialize'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/mock_session.rb:32:in `request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:230:in `process_request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:57:in `get'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:109:in `block in combined'
    org/jruby/RubyArray.java:2300:in `map'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:108:in `combined'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:94:in `minify'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:32:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    uri:classloader:/gems/tilt-1.4.1/lib/tilt.rb:127:in `fetch'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:31:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    org/jruby/RubyMethod.java:111:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1603:in `block in compile!'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:985:in `route_eval'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1006:in `block in process_route'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1004:in `process_route'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:964:in `block in route!'
    org/jruby/RubyArray.java:1560:in `each'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:963:in `route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1076:in `block in dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1073:in `dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `block in call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:886:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/nulllogger.rb:9:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:180:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:2014:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `block in call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1788:in `synchronize'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:490:in `handle_request'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:361:in `process_client'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:254:in `block in run'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/thread_pool.rb:92:in `block in spawn_thread'
@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 16, 2015

Contributor

Confirmed that the same is true w/ JRuby 9.0.0.0

Contributor

r6p commented Oct 16, 2015

Confirmed that the same is true w/ JRuby 9.0.0.0

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 19, 2015

Member

Ok, I'm not sure the best way to handle this. Did you say it works on 1.7?

The stream we get from the in-jar files is a JarURLInputStream. The only public ancestor of that is FilterInputStream, which doesn't support seeking (it supports mark/reset but I'm not sure that's enough).

The JarURLInputStream appears to wrap a ZipFileInputStream or ZipFileInflaterInputStream. They don't support seeking either.

What is the use case you're trying to solve? It may be simpler for you to just read in-jar files into a binary String and then use StringIO to simulate random access.

Member

headius commented Oct 19, 2015

Ok, I'm not sure the best way to handle this. Did you say it works on 1.7?

The stream we get from the in-jar files is a JarURLInputStream. The only public ancestor of that is FilterInputStream, which doesn't support seeking (it supports mark/reset but I'm not sure that's enough).

The JarURLInputStream appears to wrap a ZipFileInputStream or ZipFileInflaterInputStream. They don't support seeking either.

What is the use case you're trying to solve? It may be simpler for you to just read in-jar files into a binary String and then use StringIO to simulate random access.

@headius headius added this to the JRuby 9.0.2.0 milestone Oct 19, 2015

@headius headius self-assigned this Oct 19, 2015

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 19, 2015

Contributor

Basically, Rack will read the file and seek within it, see stack trace above. So we aren't actually doing the seeking Rack is doing it transitively. On 1.7.19 this doesn't happen via the jar::file path that was produced via File.expand_path.

Contributor

r6p commented Oct 19, 2015

Basically, Rack will read the file and seek within it, see stack trace above. So we aren't actually doing the seeking Rack is doing it transitively. On 1.7.19 this doesn't happen via the jar::file path that was produced via File.expand_path.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 19, 2015

Member

@r6p Ok, I think I figured it out. Apparently I determined at one point that only selectable channels (e.g. sockets, pipes) should raise errors when you try to seek on them. I'll see about making the same change to 9k.

Member

headius commented Oct 19, 2015

@r6p Ok, I think I figured it out. Apparently I determined at one point that only selectable channels (e.g. sockets, pipes) should raise errors when you try to seek on them. I'll see about making the same change to 9k.

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 19, 2015

Contributor

Any chance for a 9.0.1.1 release w/ this fix?

Contributor

r6p commented Oct 19, 2015

Any chance for a 9.0.1.1 release w/ this fix?

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 19, 2015

Member

9.0.1.1 would be a security fix release. This will go into 9.0.2.0 shortly.

Member

headius commented Oct 19, 2015

9.0.1.1 would be a security fix release. This will go into 9.0.2.0 shortly.

@headius headius closed this in a4cc5ee Oct 19, 2015

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 19, 2015

Contributor

Great thanks. Let me know if you want me to verify against a build from HEAD if that is helpful.

Contributor

r6p commented Oct 19, 2015

Great thanks. Let me know if you want me to verify against a build from HEAD if that is helpful.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 19, 2015

Member

@r6p Verification is always welcome :-)

Member

headius commented Oct 19, 2015

@r6p Verification is always welcome :-)

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 20, 2015

Contributor

@headius I just verified with jruby-head at 4e05196. Many thanks for turning this around so quickly. Do you have a rough ETD for a 9.0.2.0 release?

Contributor

r6p commented Oct 20, 2015

@headius I just verified with jruby-head at 4e05196. Many thanks for turning this around so quickly. Do you have a rough ETD for a 9.0.2.0 release?

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 20, 2015

Member

We're working on pushing out a release today.

Member

headius commented Oct 20, 2015

We're working on pushing out a release today.

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 20, 2015

Contributor

@headius that's awesome. Many thanks!

Contributor

r6p commented Oct 20, 2015

@headius that's awesome. Many thanks!

@r6p

This comment has been minimized.

Show comment
Hide comment
@r6p

r6p Oct 21, 2015

Contributor

@headius thanks again! I just verified using the public v9.0.3.0 release.

Contributor

r6p commented Oct 21, 2015

@headius thanks again! I just verified using the public v9.0.3.0 release.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Oct 22, 2015

Member

@r6p Thanks!

Member

headius commented Oct 22, 2015

@r6p Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment