cmd/go: creates executables with non-standard permissions on NTFS

[steamonimo]

What version of Go are you using (go version)?

1.9.1

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

Windows 2012 Server R2

What did you do?

Compiling source and creating executable

What did you expect to see?

At minimum I expect that the created executable will inherit the rights from the folder it was created in.
If the executable did exists before then it would be even better to preserve its current NTFS settings (that could be very different from the rest)

What did you see instead?

The created executable is currently breaking the inheritance of NTFS permissions. The resulting executable is restricted to the current user and system. In contrast the standard windows behaviour for newly created files is that the file will always inherit its permissions from its parent folder. With Windows 7 files that are moved will also inherit the permissions of the new target folder. With executables running in different and very restricted user accounts the current behaviour of breaking the inheritance is not helpful. In more general terms not respecting the current NTFS permissions is not recommendable.

[alexbrainman]

What did you do?

Compiling source and creating executable

Please, provide the steps that we need to do to reproduce your problem. What commands do we run? Assume we know nothing about this "inheritance of NTFS permissions" subject (because we don't). Thank you.

The created executable is currently breaking the inheritance of NTFS permissions. The resulting executable is restricted to the current user and system. In contrast the standard windows behaviour for newly created files is that the file will always inherit its permissions from its parent folder.

Please, provide some references that support your claim. I do not know anything about the subject, so it is hard for me to judge without some authoritive documentation. Thank you.

Alex

With executables running in different and very restricted user accounts the current behaviour of breaking the inheritance is not helpful.

Can you, please, explain this. What is the exact problem?

Alex

[steamonimo]

Please, provide the steps that we need to do to reproduce your problem. What commands do we run? Assume we know nothing about this "inheritance of NTFS permissions" subject (because we don't). Thank you.

On Windows the command go build x.go will create an executable file. If the file did exist before then this file will be overwritten. This newly created file has NTFS permissions that will be different from its parent folder because permission inheritance is disabled.

Can you, please, explain this. What is the exact problem?
The problem is the file systems with Access Control Lists are still rarely used on the Linux platform therefore the awareness of developers is not very high. NTFS is one variant of ACL based file systems. Here the inheritance of file permissions is an integral part of respecting the security architecture of its users. With security architecture I mean the specific rights that are applied to directories. With new files inheriting the permissions from the parent folder the security architecture is preserved.

In my case the golang executable is executed from a Windows service. This service runs with least necessary privileges to execute the task. This is done by executing the service in a very limited user account. Under normal circumstances I would just modify the privileges of the parent folder of the exe once to allow this restricted user to always read/execute the files in this folder. The go build would produce an executable that would inherit these permissions. Therefore my service would execute the golang application without problems. But with the current go build and its broken inheritance I have to manually modify the NTFS permissions, add the user and allow him read/execute the created executable.

In contrast the Microsoft Compiler does exatly what I expected the go command too: it will inherit the NTFS permissions from the parent folder where the executable has been created. This inheritance is the standard behaviour of Windows: just create a new file and look at its NTFS permissions - all inherited from the parent folder. Copy an existing file to a new directory with different NTFS permissions - the moved file will now inherit from this parent.

This behaviour prevents to create loopholes in the security architecture of the file system. For example on servers where folders are shared with many users. Here the permissions on the folders will control which user group has access. Files moved or newly created there will inherit the permissions from the parent folder. Therefore their permissions are as expected from the security standpoint. Respecting the NTFS permissions instead of replacing them with your own is a standard procedure recommended in "Writing Secure Code" by Microsoft.

[alexbrainman]

On Windows the command go build x.go will create an executable file. If the file did exist before then this file will be overwritten. This newly created file has NTFS permissions that will be different from its parent folder because permission inheritance is disabled.

I need to see it all for myself. What are the steps for me to see this? How do I see that created file has NTFS permissions that are different from its parent folder. You say "permission inheritance is disabled". Where is it disabled? How do I see it? What tools do I use it to see it?

Thank you

Alex

[steamonimo]

I suggest the following steps:

• take a simple go project as your base project. You put this project in directory A. Then compile an executable in this directory A.

• now visit the directory A with the windows explorer and open the properties of directory A. Right click on directory A and select properties in the menu. In the tab security you see that the directory A has very specific NTFS permissions.

• in the windows explorer you now click on the executable and select right click > properties as before. Now in the tab security you see the NTFS permissions. There you can see that the groups from the directory are not there. Instead just the current user, system and administrators have access. This is because the inheritance of the NTFS permissions has been disabled for this executable. You can verify this by clicking "Advanced" below. In the following dialog you see the button "Enable Inheritance". This shows you that currently the inhertance of NTFS permissions from the parent directory is indeed disabled - and this is the root of the problem.

• now the point is that the Compiler creates the executable very likely with the CreateFile API function. This function has a parameter called lpSecurityAttributes. The structure you have to pass there is rather complex. It controls which NTFS permissions are applied to the created file: which users/groups have access, which rights they have and if inheritance is enabled. One solution is to at least enable inheritance.

However the solution is much easier: instead of passing a complex and error prone structure to lpSecurityAttributes of CreateFile you just pass NULL. This will create an executable file that inherits all NTFS permissions from its parent directory A - and since the current user can already read and write in directory A - otherwise he could not edit files there - the inherited permissions will also allow the created file to be executed. This way NTFS permissions are preserved and respected.

Now you could argue: what if the user is compiling to a directory where the NTFS permissions of the parent folder would not allow the file to be executed? If this is an concern then at least enable inheritance in the more complex lpSecurityAttributes you are already passing. This way the file will inherit from the parent and in addition the rights from the lpSecurityAttributes are applied for the current user, system and administrators.

[ianlancetaylor]

As far as I can see the compiler does pass nil for the security attributes when opening a file.

I think what is happening is more complicated. The go tool directs the compiler to create the file in a temporary directory. Then the Go tool moves the file into its final location. On Windows this move is done using MoveFileEx. Perhaps that winds up giving the file the wrong permissions.

The relevant code is Builder.moveOrCopyFile, which on tip is in cmd/go/internal/work/exec.go. That function already has some special cases where Unix systems require a copy rather than a rename. Perhaps we need to add a similar special case on Windows, or perhaps on Windows we must always copy rather than rename.

[steamonimo]

Under Windows the Windows Explorer will adjust the NTFS permissions if files are moved. But the fundamental API function MoveFileEx will just move the file to the destination folder while the NTFS permissions stay the same. In the temp folder the NTFS permissions are very strict. Therefore the executable will inherit these strict permissions and then it will just be moved to the output folder.

I share your conclusion that a special case would be useful. Your approach to Copy from temp to output folder and then delete in the temp folder will solve the issue.

[ianlancetaylor]

Is there a way that we can whether NTFS permissions apply to the destination directory, comparable to the test of ModeSetgid on Unix systems? Or do these permissions always apply, and we should always copy on Windows?

[steamonimo]

In the Windows world it is unusual to allow read but not read/execute for users/groups in the folder permissions. Without read/execute granted you can not even list the folder content. Of course from the standpoint of pure NTFS permissions it is possible to do that restriction. However this is a choice by the Admin of the folder and this should be respected. In my opinion it would be wrong to ignore the folder settings and repair the executable to read/execute on your own. Mainly because you would speculate which users you would allow these permissions. You would have to assume that the current user, the system and the admins should be able to execute the file. However you do not know for sure what the folder admin intended here. You could also read the users/groups from the parent folder, enable inheritance and add the NTFS permission read/execute to the executable for all these users/groups. However I would not recommend these NTFS modifications at all.

Better always copy the executable and put it in the responsibility of the NTFS permissions of the target folder to allow this file to be executed. In most cases the executable will run because it has inherited read and read/execute permissions (Standard case in Windows file system).

In the rare case it can not run due to permissions it was the intentional choice of the folder admin. By one adjustment to the folder NTFS permissions the folder admin can then allow read/execute for all files inheriting from this folder. So the fix in this rare case is quite obvious. To have a rare case that needs manual attention is better than disrespecting the existing NTFS permissions of the target folder. Admins often invest a lot of time into their folder structure and their associated NTFS permissions. Same is true for other ACL based file systems like ZFS on linux or solaris.

[ianlancetaylor]

Thanks. You didn't really answer my question, though. Perhaps the question doesn't make sense.

Is there a way that we can tell whether NTFS permissions apply to the destination directory, comparable to the test of ModeSetgid on Unix systems? Or do these permissions always apply, and we should always copy on Windows?

[steamonimo]

The copied file will always inherit from the parent that is for sure. However what it will inherit depends on NTFS permissions of the parent folder. An untouched folder will itself inherit from its parent and so forth. Of course the destination folder could also disable inheritance. It could also set permissions read/execute and forbidden read/execute at the same time. In this case the forbidden permissions will overrule the allowed permissions. It is also possible to remove all permissions and leave them empty. But this is all on purpose by the folder admin. If he wanted the permission list to be empty then the executable copied there will inherit this empty permission list. Always copy is at least doing what the owner/admin of the folder intended.

To proof this I did a test with Visual Studio 2017 and created a small console application. I have set the following permissions for the "debug" folder: my user read, write, list folder. In the compilation process Visual Studio warned about missing access rights to the output folder. Still the executable has been created and it inherited the rights from the "debug" folder. The read/execute was not added by the compiler and therefore the resulting executable is not executable. The warning of the compiler about the access rights just informs about these missing read/execute rights. Here Microsoft is serious about not modifying or repairing NTFS permissions. If you copy from temp to a folder with insufficient permissions you will produce the same result. From my standpoint the correct approach.

[gopherbot]

Change https://golang.org/cl/72910 mentions this issue: cmd/go: always copy files on Windows