Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

bigsur0 · 2015-10-16T00:15:34Z

Error case followed by success case below. This issue breaks Sinatra AssetPack running on JRuby 9.0.1.0 when the Sinatra "root" directory is set using the classpath: scheme.

$ mkdir /tmp/classpath_seek
$ cd !$
$ rvm use jruby-9.0.1.0
$ touch foo.rb
$ irb
>> $CLASSPATH <<  Dir.pwd
=> ["file:/Users/me/.m2/repository/jline/jline/2.11/jline-2.11.jar", "file:/Users/me/.rvm/rubies/jruby-9.0.1.0/lib/ruby/stdlib/readline.jar", "file:/private/tmp/classpath_seek/"]
>> f = File.open('classpath:foo.rb')
=> #<File:classpath:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - classpath:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):4:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

>> f = File.open(File.join(Dir.pwd, 'foo.rb'))
=> #<File:/private/tmp/classpath_seek/foo.rb>
>> f.seek(0)
=> 0

The text was updated successfully, but these errors were encountered:

bigsur0 · 2015-10-16T00:37:15Z

The same problem exists for the uri:classloader scheme.

>>  f = File.open('uri:classloader:foo.rb')
=> #<File:uri:classloader:foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader:foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):6:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'


>> f = File.open('uri:classloader://foo.rb')
=> #<File:uri:classloader://foo.rb>
>> f.seek(0)
Errno::EPIPE: Broken pipe - uri:classloader://foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):34:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

bigsur0 · 2015-10-16T03:45:37Z

Same issue via this route (foo.rb in a jar):

>> f = File.open('jar:file:/Users/me/foo.jar!/foo.rb')
>> f.seek(0)
Errno::EPIPE: Broken pipe - jar:file:/Users/me/foo.jar!/foo.rb
    from org/jruby/RubyIO.java:1649:in `seek'
    from (irb):13:in `<eval>'
    from org/jruby/RubyKernel.java:978:in `eval'
    from org/jruby/RubyKernel.java:1291:in `loop'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from org/jruby/RubyKernel.java:1098:in `catch'
    from /Users/me/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `<top>'

bigsur0 · 2015-10-16T16:18:55Z

And here's a full backtrace of how we first encountered the issue, to give you a sense of how impactful this issue is.


Errno::EPIPE - Broken pipe - uri:classloader://app/assets/javascripts/models/component.js:
    org/jruby/RubyIO.java:1649:in `seek'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:110:in `block in each'
    org/jruby/RubyIO.java:1126:in `open'
    uri:classloader:/gems/rack-1.5.2/lib/rack/file.rb:109:in `each'
    uri:classloader:/gems/rack-1.5.2/lib/rack/response.rb:36:in `initialize'
    uri:classloader:/gems/rack-1.5.2/lib/rack/mock.rb:161:in `initialize'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/mock_session.rb:32:in `request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:230:in `process_request'
    uri:classloader:/gems/rack-test-0.6.2/lib/rack/test.rb:57:in `get'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:109:in `block in combined'
    org/jruby/RubyArray.java:2300:in `map'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:108:in `combined'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/package.rb:94:in `minify'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:32:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    uri:classloader:/gems/tilt-1.4.1/lib/tilt.rb:127:in `fetch'
    uri:classloader:/gems/sinatra-assetpack-0.3.3/lib/sinatra/assetpack/class_methods.rb:31:in `block in GET (?-mix:^\/assets\/application(?:.[a-f0-9]{32})?.js$)'
    org/jruby/RubyMethod.java:111:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1603:in `block in compile!'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:985:in `route_eval'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:966:in `block in route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1006:in `block in process_route'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1004:in `process_route'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:964:in `block in route!'
    org/jruby/RubyArray.java:1560:in `each'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:963:in `route!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1076:in `block in dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1073:in `dispatch!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `block in call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `block in invoke'
    org/jruby/RubyKernel.java:1098:in `catch'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1058:in `invoke'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:898:in `call!'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:886:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
    uri:classloader:/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/nulllogger.rb:9:in `call'
    uri:classloader:/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:180:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:2014:in `call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `block in call'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1788:in `synchronize'
    uri:classloader:/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:490:in `handle_request'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:361:in `process_client'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/server.rb:254:in `block in run'
    org/jruby/RubyProc.java:308:in `call'
    uri:classloader:/gems/puma-2.8.2-java/lib/puma/thread_pool.rb:92:in `block in spawn_thread'

bigsur0 · 2015-10-16T21:01:19Z

Confirmed that the same is true w/ JRuby 9.0.0.0

headius · 2015-10-19T17:05:09Z

Ok, I'm not sure the best way to handle this. Did you say it works on 1.7?

The stream we get from the in-jar files is a JarURLInputStream. The only public ancestor of that is FilterInputStream, which doesn't support seeking (it supports mark/reset but I'm not sure that's enough).

The JarURLInputStream appears to wrap a ZipFileInputStream or ZipFileInflaterInputStream. They don't support seeking either.

What is the use case you're trying to solve? It may be simpler for you to just read in-jar files into a binary String and then use StringIO to simulate random access.

bigsur0 · 2015-10-19T17:41:03Z

Basically, Rack will read the file and seek within it, see stack trace above. So we aren't actually doing the seeking Rack is doing it transitively. On 1.7.19 this doesn't happen via the jar::file path that was produced via File.expand_path.

headius · 2015-10-19T18:46:54Z

@r6p Ok, I think I figured it out. Apparently I determined at one point that only selectable channels (e.g. sockets, pipes) should raise errors when you try to seek on them. I'll see about making the same change to 9k.

bigsur0 · 2015-10-19T18:55:00Z

Any chance for a 9.0.1.1 release w/ this fix?

headius · 2015-10-19T19:44:16Z

9.0.1.1 would be a security fix release. This will go into 9.0.2.0 shortly.

bigsur0 · 2015-10-19T20:33:32Z

Great thanks. Let me know if you want me to verify against a build from HEAD if that is helpful.

headius · 2015-10-19T20:43:35Z

@r6p Verification is always welcome :-)

bigsur0 · 2015-10-20T05:09:47Z

@headius I just verified with jruby-head at 4e05196. Many thanks for turning this around so quickly. Do you have a rough ETD for a 9.0.2.0 release?

headius · 2015-10-20T15:43:24Z

We're working on pushing out a release today.

bigsur0 · 2015-10-20T15:59:51Z

@headius that's awesome. Many thanks!

bigsur0 · 2015-10-21T23:45:13Z

@headius thanks again! I just verified using the public v9.0.3.0 release.

headius · 2015-10-22T12:23:00Z

@r6p Thanks!

bigsur0 changed the title ~~Can't seek when file accessed using classpath: scheme~~ Can't seek when file accessed using classpath: OR uri:classloader: schemes Oct 16, 2015

headius added this to the JRuby 9.0.2.0 milestone Oct 19, 2015

headius self-assigned this Oct 19, 2015

headius added java integration JRuby 9000 labels Oct 19, 2015

headius closed this as completed in a4cc5ee Oct 19, 2015

ikaronen-relex mentioned this issue Aug 16, 2021

Seeking the DB file does not work in a bundled JRuby application, crashes randomly discourse/mini_mime#37

Open

This was referenced Oct 25, 2022

Make sure the DB file is seekable, use a StringIO wrapper if it's not discourse/mini_mime#38

Open

Raise EPIPE for all unseekable channels #7415

Open

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 20, 2015

headius commented Oct 20, 2015

bigsur0 commented Oct 20, 2015

bigsur0 commented Oct 21, 2015

headius commented Oct 22, 2015

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

Can't seek when file accessed using classpath: OR uri:classloader: schemes #3399

Comments

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

bigsur0 commented Oct 16, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 19, 2015

headius commented Oct 19, 2015

bigsur0 commented Oct 20, 2015

headius commented Oct 20, 2015

bigsur0 commented Oct 20, 2015

bigsur0 commented Oct 21, 2015

headius commented Oct 22, 2015