Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File#pos fails for files greater than java.lang.Integer::MAX_VALUE on 64-bit systems #3664

Closed
byteit101 opened this issue Feb 13, 2016 · 6 comments
Milestone

Comments

@byteit101
Copy link
Member

@byteit101 byteit101 commented Feb 13, 2016

When writing files greater than 2GiB, the Tempfile#pos value throws an error.

require 'tempfile'
halfG = "12345" * 100_000_000
tmp = Tempfile.new("demo")
9.times do
    puts tmp.write(halfG)
    begin
        puts tmp.pos # will error once > 2GiB
    rescue
        puts "Error, can still write"
        puts tmp.write(halfG)
        puts tmp.size # works fine
        puts tmp.pos # will error
    end
end

JRuby 9.0.5.0 (2G heap):

$ ruby -J-Xmx2000m bug-demo.rb
500000000
500000000
500000000
1000000000
500000000
1500000000
500000000
2000000000
500000000
Error, can still write
500000000
3000000000
Errno::ENOENT: No such file or directory - /tmp/demo20160213-14805-1qqz35b
                   pos at org/jruby/RubyIO.java:1532
  block in bug-demo.rb at bug-demo.rb:13
                 times at org/jruby/RubyFixnum.java:296
                 <top> at bug-demo.rb:5

MRI:

$ ruby bug-demo.rb 
500000000
500000000
500000000
1000000000
500000000
1500000000
500000000
2000000000
500000000
2500000000
500000000
3000000000
500000000
3500000000
500000000
4000000000
500000000
4500000000
@byteit101 byteit101 changed the title Tempfile#pos fails for files greater than java.lang.Integer::MAX_VALUE File#pos fails for files greater than java.lang.Integer::MAX_VALUE Feb 13, 2016
@byteit101
Copy link
Member Author

@byteit101 byteit101 commented Feb 13, 2016

Also just realized that this is in File#pos too, not just Tempfile.

tmp = File.new("/tmp/demo", "w")

(other code is the same)

@byteit101
Copy link
Member Author

@byteit101 byteit101 commented Feb 13, 2016

Digging in, this looks to be a jnr-posix issue as all the declarations are ints, not longs. Should I cross-post this issue in jnr?

https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/io/PosixShim.java#L69
https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java#L153

@headius
Copy link
Member

@headius headius commented Feb 14, 2016

@byteit101 Yeah we'll need to fix those APIs to be longs. Anything that's using NIO channels only should be long already.

@byteit101
Copy link
Member Author

@byteit101 byteit101 commented Feb 14, 2016

I just tried this on a 32-bit system, and it worked as expected, which I did not expect

@byteit101 byteit101 changed the title File#pos fails for files greater than java.lang.Integer::MAX_VALUE File#pos fails for files greater than java.lang.Integer::MAX_VALUE on 64-bit systems Feb 14, 2016
@headius headius added this to the JRuby 9.1.1.0 milestone Mar 8, 2016
@headius headius modified the milestones: JRuby 9.1.0.0, JRuby 9.1.1.0 Apr 20, 2016
@headius
Copy link
Member

@headius headius commented Apr 20, 2016

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435, but the remaining API fixes still need to be done.

@headius headius closed this Apr 20, 2016
@headius
Copy link
Member

@headius headius commented Apr 20, 2016

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.