Strange access issue with Ruby and network drives

[orgads]

Hi,

I'm trying to access 2 network drives from msys2 shell using ruby. One works, while the other doesn't.

The same command works with msys1.

ruby -e 'puts Dir.glob("//server1/some/path/*")' # works
ruby -e 'puts Dir.glob("//server2/other/path/*")' # works with msys1, fails with msys2

Needless to say, the same version of Ruby is used on both environments.

Process Monitor shows that for server2 CreateFile is invoked for //server2/other/path/ with a trailing slash, which results in Access Denied, while with msys1 or cmd, and for server1 even with msys2, there isn't a trailing slash.

The path length is the same for both servers (1 share name, 1 path under it).

I have no idea what type of servers these are.

Is there a way to debug this problem?

[mingwandroid]

Where'd you get your ruby from?

[orgads]

RubyInstaller. Version 2.1.5p273.

[mingwandroid]

Do both folders contain files or is one of them empty?

[mingwandroid]

Can you show the output from:
ruby -e 'print ["//server2/other/path/"]'
and:
ruby -e 'print ["//server1/other/path/"]'

[orgads]

Both have files obviously.

Did you mean Dir["//server1/other/path/"]?

[orgads]

$ ruby -e 'print Dir["//server1/some/path/"]'
["//server1/some/path/"]
$ ruby -e 'print Dir["//server2/other/path/"]'
["//server2/other/path/"]

[mingwandroid]

It is not obvious at all, and why would it be?

I wanted you to reply with what I asked for as I want to see if MSYS2 mangles your arguments.

[orgads]

The output is just the same without Dir.

If there weren't files and it would return an empty list, why would I file an issue?

[elieux]

Can you check the command lines that are used to invoke ruby in Process Monitor? (It's the "Process Start" event.)

[mingwandroid]

You weren't clear about what "One works, while the other doesn't" meant.

Github mangled my "*"'s, I wanted to see the output of:

ruby -e 'print ["//server1/some/path/*"]'
ruby -e 'print ["//server2/other/path/*"]'

.. but the Process Start command lines should be enough anyway.

[orgads]

The Process Start looks the same for both paths:

D:\Ruby\bin\ruby.exe -e "print Dir[\"//server1/some/path/*\"]"
D:\Ruby\bin\ruby.exe -e "print Dir[\"//server2/other/path/*\"]"

[elieux]

Do they look the same when started from MSYS and MSYS2?

[orgads]

Exactly the same. It's the same also on cmd.

[elieux]

I wonder what else could be affecting the results. What terminals are you using?

Can you also try from Cygwin?

[orgads]

Doesn't work in cygwin (both 32 and 64 bit).

[elieux]

And terminals? Is it possible that the difference between the working environments and the non-woking ones is in mintty?

[orgads]

On cygwin I use the default terminal. Which other terminals are available?

[elieux]

The two basic options are the Windows console host and mintty. I don't know which one is the default for Cygwin, but MSYS2 uses mintty as default and AFAIK MSYS1 uses the Windows console.

[orgads]

The default of MSYS1 (which works) looks exactly like the default for Cygwin (which fails). MSYS2 uses mintty here, so it doesn't look related to the terminal type.

[mingwandroid]

Cygwin and MSYS2 both use a very recent mintty, MSYS1 uses a very old mintty.

[orgads]

Is there a way to pick an alternative shell with cygwin?

[mingwandroid]

From cmd.exe, do:

C:\msys64\usr\bin\bash --login -i

.. and see what results you get.

[orgads]

Fails.

[mingwandroid]

.. and finally, same thing without --login -i

[orgads]

Fails.

[mingwandroid]

So bash or MSYS2 is somehow poisoning the environment .. which MSYS2 are you running? The WIP git-for-Windows one or ours?

[orgads]

Tried both. But official cygwin also fails.

[mingwandroid]

I'm fairly out of ideas; if you are able to post detailed procmon logs somewhere then we can scan those for differences.

[orgads]

This is a full procmon log filtered by ruby.exe process name.

The working server is //netapp1/CM/CompilationResults, the failing one is //aclnas01/versions/CompilationResults.

The log contains 2 successful executions with MSYS1 which succeed (1 for each server), then a successful execution with MSYS2 to netapp1, and the last one is for aclnas01 which fails (notice ACCESS DENIED at line 9902).

[mingwandroid]

The servers seem to have different access permissions.
Working one has:

"16:11:16.2122568","ruby.exe","6904","QueryNetworkOpenInformationFile","\\netapp1\CM\CompilationResults","SUCCESS","CreationTime: 20/04/2004 14:51:32, LastAccessTime: 07/05/2015 16:11:05, LastWriteTime: 13/04/2015 17:02:45, ChangeTime: 07/05/2015 00:41:01, AllocationSize: 01/01/1601 03:00:00, EndOfFile: 01/01/1601 03:00:00, FileAttributes: D"

Non working one has:

"16:11:18.9542818","ruby.exe","8676","QueryNetworkOpenInformationFile","\\aclnas01\versions\CompilationResults","SUCCESS","CreationTime: 27/01/2015 23:03:32, LastAccessTime: 07/05/2015 16:11:09, LastWriteTime: 28/01/2015 17:24:09, ChangeTime: 24/02/2015 16:23:19, AllocationSize: 01/01/1601 03:00:00, EndOfFile: 01/01/1601 03:00:00, FileAttributes: DA"

A is archiving .. are these both just normal smb shares? Can you remove archiving flag to see if it makes any difference?

[mingwandroid]

This may be of interest too:

http://blogs.msdn.com/b/oldnewthing/archive/2009/02/19/9432917.aspx

A could mean "needs archiving" but it could also mean encrypted or compressed.

[mingwandroid]

Also, MSYS1:

"16:11:09.1113553","ruby.exe","3036","CreateFile","\\aclnas01\versions\CompilationResults\","SUCCESS","Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"

vs MSYS2:

"16:11:18.9558312","ruby.exe","8676","CreateFile","\\aclnas01\versions\CompilationResults\","ACCESS DENIED","Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Open For Backup, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a"

I'm not sure what this "Open For Backup" means, but it does sound archiving-related.

[mingwandroid]

This thread looks very interesting in this regard:

https://cygwin.com/ml/cygwin/2008-02/msg00076.html

[elieux]

I'm not sure what this "Open For Backup" means, but it does sound archiving-related.

AFAIK, this is primarily used for archiving purposes, but is also used by Cygwin to gain access to files that would be inaccessible without it (in case the user/process possesses SeBackupPrivilege), probably to emulate the all-doors-open rights of root accounts in the POSIX world.

[elieux]

https://cygwin.com/ml/cygwin/2008-02/msg00076.htm

Out of curiosity, I built the MSYS2 runtime with all occurrences of "Open for Backup" flag removed: msys-runtime, msys2-runtime.sig, msys-runtime-devel, msys2-runtime-devel.sig. I'm not sure if I actually succeeded, but give it a try.

[orgads]

@elieux: Thanks for your effort. Unfortunately, it doesn't solve the issue.

[orgads]

Can you remove archiving flag to see if it makes any difference?

@mingwandroid I only have read access to these servers. I can't test that.

[orgads]

Ok, I was able to reproduce with a minimal working example:

#include <stdio.h>
#include <windows.h>

int main()
{
    const TCHAR *name = TEXT("//aclnas01/versions/CompilationResults");
    WIN32_FIND_DATA fd;
    printf("%d\n", FindFirstFile(name, &fd) != INVALID_HANDLE_VALUE); // Fails on cygwin
    return 0;
}

Any ideas how to investigate it further?

[orgads]

2 days ago I posted this bug report to the cygwin mailing list. It still wasn't answered.

Some more insights follow.

Using Wireshark, I discovered that aclnas01 uses SMB, while netapp1 uses SMB2.

For some reason, when executed from command prompt, a FIND_FIRST2 request is sent and the server replies, while with cygwin a "NT Create AndX" request is sent and denied.

To minimize the captured packets, I executed the application twice, and captured only the second execution (to avoid session initiation).

Working capture
Failing capture

Does cygwin implement SMB on its own, or is this difference a
side-effect of the "Backup Intent" flag?

[orgads]

I just found another SMB1 linux server, which does work.

It first has a "NT Create AndX" request for path , which succeeds. Then it issues Trans2 FIND_FIRST2 request for the real path (\a).

This issue might be related to the Archive bit which is set on aclnas01.

[orgads]

Hi,

After investigation, I found that the root cause for this problem is
set_cygwin_privileges, which sets SE_RESTORE_PRIVILEGE and
SE_BACKUP_PRIVILEGE for the process during initialization.

Commenting out these 2 lines solves the problem for me.

I have no idea if it breaks anything.

[orgads]

Ok, I was able to fix the issue by removing the privileges in my script.

If anyone with this problem reaches here, the script follows:

  def remove_privileges
    return unless RUBY_PLATFORM.include?('mingw')
    require 'win32/api'

    _AdjustTokenPrivileges = Win32::API.new('AdjustTokenPrivileges', 'LBPLPP', 'B', 'advapi32')
    _OpenProcessToken  = Win32::API.new('OpenProcessToken', 'LLP', 'B', 'advapi32')
    _GetCurrentProcess = Win32::API.new('GetCurrentProcess', 'V', 'L', 'kernel32')
    _TOKEN_ADJUST_PRIVILEGES = 0x0020

    token = [0].pack('l')
    _OpenProcessToken.call(_GetCurrentProcess.call, _TOKEN_ADJUST_PRIVILEGES, token)
    token = token.unpack('l').first
    _AdjustTokenPrivileges.call(token, true, 0, 0, 0, 0)
  end