New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File#pos fails for files greater than java.lang.Integer::MAX_VALUE on 64-bit systems #3664

Closed
byteit101 opened this Issue Feb 13, 2016 · 6 comments

Comments

Projects
None yet
2 participants
@byteit101
Member

byteit101 commented Feb 13, 2016

When writing files greater than 2GiB, the Tempfile#pos value throws an error.

require 'tempfile'
halfG = "12345" * 100_000_000
tmp = Tempfile.new("demo")
9.times do
    puts tmp.write(halfG)
    begin
        puts tmp.pos # will error once > 2GiB
    rescue
        puts "Error, can still write"
        puts tmp.write(halfG)
        puts tmp.size # works fine
        puts tmp.pos # will error
    end
end

JRuby 9.0.5.0 (2G heap):

$ ruby -J-Xmx2000m bug-demo.rb
500000000
500000000
500000000
1000000000
500000000
1500000000
500000000
2000000000
500000000
Error, can still write
500000000
3000000000
Errno::ENOENT: No such file or directory - /tmp/demo20160213-14805-1qqz35b
                   pos at org/jruby/RubyIO.java:1532
  block in bug-demo.rb at bug-demo.rb:13
                 times at org/jruby/RubyFixnum.java:296
                 <top> at bug-demo.rb:5

MRI:

$ ruby bug-demo.rb 
500000000
500000000
500000000
1000000000
500000000
1500000000
500000000
2000000000
500000000
2500000000
500000000
3000000000
500000000
3500000000
500000000
4000000000
500000000
4500000000

@byteit101 byteit101 changed the title from Tempfile#pos fails for files greater than java.lang.Integer::MAX_VALUE to File#pos fails for files greater than java.lang.Integer::MAX_VALUE Feb 13, 2016

@byteit101

This comment has been minimized.

Show comment
Hide comment
@byteit101

byteit101 Feb 13, 2016

Member

Also just realized that this is in File#pos too, not just Tempfile.

tmp = File.new("/tmp/demo", "w")

(other code is the same)

Member

byteit101 commented Feb 13, 2016

Also just realized that this is in File#pos too, not just Tempfile.

tmp = File.new("/tmp/demo", "w")

(other code is the same)

@byteit101

This comment has been minimized.

Show comment
Hide comment
@byteit101

byteit101 Feb 13, 2016

Member

Digging in, this looks to be a jnr-posix issue as all the declarations are ints, not longs. Should I cross-post this issue in jnr?

https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/io/PosixShim.java#L69
https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java#L153

Member

byteit101 commented Feb 13, 2016

Digging in, this looks to be a jnr-posix issue as all the declarations are ints, not longs. Should I cross-post this issue in jnr?

https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/io/PosixShim.java#L69
https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java#L153

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 14, 2016

Member

@byteit101 Yeah we'll need to fix those APIs to be longs. Anything that's using NIO channels only should be long already.

Member

headius commented Feb 14, 2016

@byteit101 Yeah we'll need to fix those APIs to be longs. Anything that's using NIO channels only should be long already.

@byteit101

This comment has been minimized.

Show comment
Hide comment
@byteit101

byteit101 Feb 14, 2016

Member

I just tried this on a 32-bit system, and it worked as expected, which I did not expect

Member

byteit101 commented Feb 14, 2016

I just tried this on a 32-bit system, and it worked as expected, which I did not expect

@byteit101 byteit101 changed the title from File#pos fails for files greater than java.lang.Integer::MAX_VALUE to File#pos fails for files greater than java.lang.Integer::MAX_VALUE on 64-bit systems Feb 14, 2016

@headius headius added this to the JRuby 9.1.1.0 milestone Mar 8, 2016

@headius headius modified the milestones: JRuby 9.1.0.0, JRuby 9.1.1.0 Apr 20, 2016

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Apr 20, 2016

Member

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435, but the remaining API fixes still need to be done.

Member

headius commented Apr 20, 2016

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435, but the remaining API fixes still need to be done.

@headius headius closed this Apr 20, 2016

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Apr 20, 2016

Member

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435.

Member

headius commented Apr 20, 2016

See #3817 which aggregates ongoing work to fix seek/tell/pos logic.

Some of the errors here will be fixed along with #3435.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment