Errno::ENOENT: No such file or directory - \\server creating a directory in a UNC path #1727

Closed
impurist opened this Issue Jun 5, 2014 · 7 comments

Comments

Projects
None yet
5 participants
@impurist

impurist commented Jun 5, 2014

When I call
FileUtils.mkdir_p("//Server/share/my_new_directory")
it raise the Errno::ENOENT: No such file or directory - \Server

This code worked perfectly in MRI.
Otherwise loving JRuby...

additional info:
jruby 1.7.12 (1.9.3p392) 2014-04-15 643e292 on Java HotSpot(TM) 64-Bit Server VM 1.8.0-b132 +indy [Windows 7-amd64]

@janisvi

This comment has been minimized.

Show comment
Hide comment
@janisvi

janisvi Sep 17, 2015

We are building downlowdable application and some of our customer are getting the same error. I got the same issue on my test windows. I used jruby 1.7.22 and tested on Windows7 (Microsoft Windows [Version 6.1.7601])

We tracked it down to jnr.posix.WindowsPOSIX.mkdir function and jnr.posixWindowsLibC._wmkdir funcion. Before creating directroy path is changed to UTF-16 and prepended with //?/. With prepended value directory creation is failing. Here is excerpt from irb how we tested it:

JRuby.runtime.getPosix.java_class.declared_field("posix")
f = _
f.accessible = true
f.value(JRuby.runtime.getPosix)
checked_posix = _
checked_posix.java_class.declared_field("posix")
f = _
f.accessible = true
f.value(checked_posix)
window_posix = _
window_posix.java_class.declared_method("wlibc")
m = _
m.accessible = true
m.invoke(window_posix)
wlibc = _

m = wlibc.java_class.declared_instance_methods[-6]
m.accessible?
m.accessible = true
m.invoke(wlibc, Java::jnr.posix.WString.path("//JWIN7/jira_h/tmp"))

This method allways returns -1. Then we tried create directory without prepending //?/ then it succeeds:

Java::jnr.posix.WString.java_class.declared_constructors[0]
c = _
c.accessible = true
m.invoke(wlibc, c.new_instance("//JWIN7/jira_h/tmp"))

this returned 0 and directory was created.

janisvi commented Sep 17, 2015

We are building downlowdable application and some of our customer are getting the same error. I got the same issue on my test windows. I used jruby 1.7.22 and tested on Windows7 (Microsoft Windows [Version 6.1.7601])

We tracked it down to jnr.posix.WindowsPOSIX.mkdir function and jnr.posixWindowsLibC._wmkdir funcion. Before creating directroy path is changed to UTF-16 and prepended with //?/. With prepended value directory creation is failing. Here is excerpt from irb how we tested it:

JRuby.runtime.getPosix.java_class.declared_field("posix")
f = _
f.accessible = true
f.value(JRuby.runtime.getPosix)
checked_posix = _
checked_posix.java_class.declared_field("posix")
f = _
f.accessible = true
f.value(checked_posix)
window_posix = _
window_posix.java_class.declared_method("wlibc")
m = _
m.accessible = true
m.invoke(window_posix)
wlibc = _

m = wlibc.java_class.declared_instance_methods[-6]
m.accessible?
m.accessible = true
m.invoke(wlibc, Java::jnr.posix.WString.path("//JWIN7/jira_h/tmp"))

This method allways returns -1. Then we tried create directory without prepending //?/ then it succeeds:

Java::jnr.posix.WString.java_class.declared_constructors[0]
c = _
c.accessible = true
m.invoke(wlibc, c.new_instance("//JWIN7/jira_h/tmp"))

this returned 0 and directory was created.

@rsim

This comment has been minimized.

Show comment
Hide comment
@rsim

rsim Sep 17, 2015

At first one fix for the irb script (to get the _wmkdir method):

m = wlibc.java_class.declared_instance_methods.detect{|m| m.name=="_wmkdir"}

Found out in https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath that for UNC path // should be replaced with //?/UNC/. So in the previous example the following works:

m.invoke(wlibc, c.new_instance("//?/UNC/JWIN7/jira_h/tmp"))

rsim commented Sep 17, 2015

At first one fix for the irb script (to get the _wmkdir method):

m = wlibc.java_class.declared_instance_methods.detect{|m| m.name=="_wmkdir"}

Found out in https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath that for UNC path // should be replaced with //?/UNC/. So in the previous example the following works:

m.invoke(wlibc, c.new_instance("//?/UNC/JWIN7/jira_h/tmp"))

@enebo enebo added this to the JRuby 1.7.23 milestone Sep 22, 2015

@enebo enebo added the core label Sep 22, 2015

@enebo

This comment has been minimized.

Show comment
Hide comment
@enebo

enebo Sep 22, 2015

Member

This is fixed in jnr-posix and I will be updating JRuby 1.7 and master to use the new version soon (just release jnr-ffi and Maven takes hours for new artifacts to show up on central).

Member

enebo commented Sep 22, 2015

This is fixed in jnr-posix and I will be updating JRuby 1.7 and master to use the new version soon (just release jnr-ffi and Maven takes hours for new artifacts to show up on central).

@mkristian

This comment has been minimized.

Show comment
Hide comment
@janisvi

This comment has been minimized.

Show comment
Hide comment
@janisvi

janisvi Sep 23, 2015

There is one more issue with mkdir_p. With fixed UNC path names mkdir_p will work only for the case when a path exists or only the last folder from the path is missing. Those two cases are handled differently - https://github.com/jruby/jruby/blob/1.7.22/lib/ruby/1.9/fileutils.rb#L206

For rest of the cases, path is split to folders using File.dirname and it looks like it is returning incorrect results for UNC path names (https://github.com/jruby/jruby/blob/1.7.22/core/src/main/java/org/jruby/RubyFile.java#L712)

Right now it is returning only host name if called for share:

File.dirname("//host/share") # => "//host"

however mkdir_p expects that smallest path returned from File.dirname will be directory (https://github.com/jruby/jruby/blob/1.7.22/lib/ruby/1.9/fileutils.rb#L223).

File.directory("//host") # => false
File.directory("//host/share" # => true

If File.dirname behaviour will be changed to:

File.dirname("//host/share") # => "//host/share"

then mkdir_p will work.

janisvi commented Sep 23, 2015

There is one more issue with mkdir_p. With fixed UNC path names mkdir_p will work only for the case when a path exists or only the last folder from the path is missing. Those two cases are handled differently - https://github.com/jruby/jruby/blob/1.7.22/lib/ruby/1.9/fileutils.rb#L206

For rest of the cases, path is split to folders using File.dirname and it looks like it is returning incorrect results for UNC path names (https://github.com/jruby/jruby/blob/1.7.22/core/src/main/java/org/jruby/RubyFile.java#L712)

Right now it is returning only host name if called for share:

File.dirname("//host/share") # => "//host"

however mkdir_p expects that smallest path returned from File.dirname will be directory (https://github.com/jruby/jruby/blob/1.7.22/lib/ruby/1.9/fileutils.rb#L223).

File.directory("//host") # => false
File.directory("//host/share" # => true

If File.dirname behaviour will be changed to:

File.dirname("//host/share") # => "//host/share"

then mkdir_p will work.

@janisvi

This comment has been minimized.

Show comment
Hide comment
@janisvi

janisvi Sep 25, 2015

We have customers waiting to install our product on Windows with UNC file paths so I created simple File.dirname (https://github.com/jruby/jruby/blob/1.7.22/core/src/main/java/org/jruby/RubyFile.java#L732) patch for our needs.

if (startsWithDriveLetterOnWindows && index == 2) {
    // Include additional path separator
    // (so that dirname of "C:\file.txt" is  "C:\", not "C:")
    index++;
}

// PATCH: don't remove //host/share from windows UNC paths "//host/share"
// but sill remove last path element if path is longer
if ( name.startsWith("//")) {
   index = name.length();
   String[] splitted = name.split(Pattern.quote("/"));
   if (splitted.length > 4) {
      index = name.lastIndexOf("/");
   }
}
// if (jfilename.startsWith("\\\\")) {
//     index = jfilename.length();
//     String[] splitted = jfilename.split(Pattern.quote("\\"));
//     int last = splitted.length-1;
//     if (splitted[last].contains(".")) {
//         index = jfilename.lastIndexOf("\\");
//     }
// }

result = jfilename.substring(0, index);

I also created simple patch for WindowsHelpers.toWPath (https://github.com/jnr/jnr-posix/blob/3.0.12/src/main/java/jnr/posix/util/WindowsHelpers.java#L24) method to work correctly for Windows UNC short paths -

public static byte[] toWPath(String path) {
    boolean absolute = new File(path).isAbsolute();
    // PATCH: commented out windows long path prefix. It is not working with UNC paths (//host/share)
    // if (absolute) {
    //     path = "//?/" + path;
    // }
    return toWString(path);
}

janisvi commented Sep 25, 2015

We have customers waiting to install our product on Windows with UNC file paths so I created simple File.dirname (https://github.com/jruby/jruby/blob/1.7.22/core/src/main/java/org/jruby/RubyFile.java#L732) patch for our needs.

if (startsWithDriveLetterOnWindows && index == 2) {
    // Include additional path separator
    // (so that dirname of "C:\file.txt" is  "C:\", not "C:")
    index++;
}

// PATCH: don't remove //host/share from windows UNC paths "//host/share"
// but sill remove last path element if path is longer
if ( name.startsWith("//")) {
   index = name.length();
   String[] splitted = name.split(Pattern.quote("/"));
   if (splitted.length > 4) {
      index = name.lastIndexOf("/");
   }
}
// if (jfilename.startsWith("\\\\")) {
//     index = jfilename.length();
//     String[] splitted = jfilename.split(Pattern.quote("\\"));
//     int last = splitted.length-1;
//     if (splitted[last].contains(".")) {
//         index = jfilename.lastIndexOf("\\");
//     }
// }

result = jfilename.substring(0, index);

I also created simple patch for WindowsHelpers.toWPath (https://github.com/jnr/jnr-posix/blob/3.0.12/src/main/java/jnr/posix/util/WindowsHelpers.java#L24) method to work correctly for Windows UNC short paths -

public static byte[] toWPath(String path) {
    boolean absolute = new File(path).isAbsolute();
    // PATCH: commented out windows long path prefix. It is not working with UNC paths (//host/share)
    // if (absolute) {
    //     path = "//?/" + path;
    // }
    return toWString(path);
}
@enebo

This comment has been minimized.

Show comment
Hide comment
@enebo

enebo Apr 7, 2016

Member

@janisvi I opened up #3786 for your second issue (and I also resolved it just now). I separated them because they are unrelated and I wanted both behaviors captured in our release notes.

Member

enebo commented Apr 7, 2016

@janisvi I opened up #3786 for your second issue (and I also resolved it just now). I separated them because they are unrelated and I wanted both behaviors captured in our release notes.

@enebo enebo closed this Apr 7, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment