Skip to content

Commit

Permalink
Fix JRUBY-4913
Browse files Browse the repository at this point in the history
FileUtils.cp failed if file source is too big (>= 1GB)

On 32-bit JVMs, FileChannel.transferTo would attempt to mmap a
a larger addressable space than the process could handle. This
caused the error. My fix is to catch that error and then attempt
the copy again in 100MB chunks. 100MB may still fail if we are
close to the process's addressable memory limit, but it's much
more reliable than what we had before.
  • Loading branch information
headius committed Jan 17, 2012
1 parent 5f98b23 commit 1b8ce57
Showing 1 changed file with 20 additions and 1 deletion.
21 changes: 20 additions & 1 deletion src/org/jruby/RubyIO.java
Expand Up @@ -3905,7 +3905,26 @@ public static IRubyObject copy_stream(ThreadContext context, IRubyObject recv,
try {
long size = f1.size();

f1.transferTo(f2.position(), size, f2);
// handle large files on 32-bit JVMs (JRUBY-4913)
try {
f1.transferTo(f2.position(), size, f2);
} catch (IOException ioe) {
// if the failure is "Cannot allocate memory", do the transfer in 100MB max chunks
if (ioe.getMessage().equals("Cannot allocate memory")) {
long _100M = 100 * 1024 * 1024;
while (size > 0) {
if (size > _100M) {
f1.transferTo(f2.position(), _100M, f2);
size -= _100M;
} else {
f1.transferTo(f2.position(), size, f2);
break;
}
}
} else {
throw ioe;
}
}

return context.getRuntime().newFixnum(size);
} catch (IOException ioe) {
Expand Down

0 comments on commit 1b8ce57

Please sign in to comment.