os: File.Stat on Windows doesn't use file handle for directories

[rolandshoemaker]

On Windows File.Stat doesn't use the file handle to retrieve attributes when the file is a directory, due to the lack of a ...ByHandle Windows API which works on directories (GetFileInformationByHandle only works on files), using the path instead.

This means there may be a TOCTOU issue, as the results returned by Stat represent the attributes for the file at the path passed to Open, which may no longer be the file represented by the file handle if the file has been moved/replaced.

[gopherbot]

Change https://go.dev/cl/405275 mentions this issue: os,syscall: File.Stat to use file handle for directories on Windows

[qmuntal]

I've been investigating this issue lately, and I've found that GetFileInformationByHandle does support directory handles. The trick is to specify the FILE_FLAG_BACKUP_SEMANTICS flag when creating the handle via syscall.CreateFile. This are the relevant docs: CreateFile.

This change breaks directory traversal, as readdir is expecting a handle created via FindFirstFile, not the CreateFile one. It can be solved by creating a new traversal-enabled handle the first time readdir is called, which has the nice benefit of only allocating a dirInfo if it is really needed, instead of every time a directory is opened.

I've done a working prototype of this idea in CL 405275.
I'm wondering if changing syscall.Open to support directories (only in read mode) would be a breaking behavior change.

I'm also pretty sure that there is no need to use the path-based Find* API for traversing directories, and that we can instead use NtQueryDirectoryFile. These should also be far more efficient, e.g. git-for-windows/git@b69c08c#diff-4b6d4f2af4b31f0bc3ff43eac9a2a437.

[qmuntal]

It is likely that CL 405275 also fixes #36019 and #43322, at least it does in my local computer.

I'll reenable TestDirSeek and update File.Seek in a separate CL just in case we have to revert it.

[qmuntal]

Another side effect of CL 405275: The directory handle created by syscall.Open does not have the share mode FILE_SHARE_DELETE set (see #32088), but the handle created by syscall.FindFirstFile did.

This means it wouldn't be possible to move or remove an opened directory, just as it is already happening with normal opened files. @alexbrainman @mattn thoughts?

[mattn]

How does os.File.Fd() work? It should always return the handle of the file. Because the user passes that handle to the function that expect the file descriptor. I think this might possibly makes another side-effect with this change. To put it simply, I think File should return ErrNotSupported or whatever for listing files.

[qmuntal]

How does os.File.Fd() work? It should always return the handle of the file. Because the user passes that handle to the function that expect the file descriptor. I think this might possibly makes another side-effect with this change. To put it simply, I think File should return ErrNotSupported or whatever for listing files.

os.File.Fd() returns whatever handler is stored in file.pfd.Sysfd.

This is the current behavior:

• If the opened file is not a directory, the handle is created using syscall.Open, which internally calls syscall.CreateFile. This handle works in most win32 apis expecting file handles.
• If the opened file is a directory, the handle is created using syscall.FindFirstFile. This handle mostly only works in syscall.Find* apis, i.e., it doesn't work in syscall.GetFileInformationByHandle, which is used by os.File.Stat.

So one could say that the current behavior of os.File.Fd() for directories is only good if you need to traverse it, but not for the general case.

CL 405275 changes how directory handles are constructed, instead of using syscall.FindFirstFile it uses syscall.Open, unifying regular file handles with directory handles. This means os.File.Fd() will return the same type of handle regardless if it is pointing to a directory, but it won't be directly usable in syscall.FindNextFile.