Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: os.ModeSocket is not set for domain socket files in Windows #33357

Open
ddebroy opened this issue Jul 29, 2019 · 22 comments
Open

os: os.ModeSocket is not set for domain socket files in Windows #33357

ddebroy opened this issue Jul 29, 2019 · 22 comments

Comments

@ddebroy
Copy link

@ddebroy ddebroy commented Jul 29, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build497560436=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I created a Windows golang program to listen on a Windows domain socket. In another Windows golang program, on the file corresponding to the socket, I invoked the following code:

fi, err := os.Stat(filename)
if err != nil {
	return fmt.Errorf("stat file %s failed: %v", filename, err)
}

if !fi.IsDir() {
	if fi.Mode()&os.ModeSocket == 0 {
		klog.V(5).Infof("Ignoring non socket file: %s", fi.Name())
                 return nil
	}
	klog.V(5).Infof("socket file found: %s", fi.Name())
        return nil
}

What did you expect to see?

socket file found: filename (as in Linux)

What did you see instead?

Ignoring non socket file: filename

@liggitt
Copy link
Contributor

@liggitt liggitt commented Jul 30, 2019

is the go env output accurate? that seems to show running on linux (GOOS="linux")

@ddebroy
Copy link
Author

@ddebroy ddebroy commented Jul 30, 2019

The go env output is for the Linux environment where the Windows binary was compiled (using GOOS="windows"). The Windows Server 2019 environment where the Windows binary was copied over and executed does not have go installed.

@mattn
Copy link
Member

@mattn mattn commented Jul 30, 2019

As far as I looked this problem, FindFirstFile and GetFileAttributes does not provide bit mask to indicate socket files.

@mattn
Copy link
Member

@mattn mattn commented Jul 31, 2019

I tested a domain socket file. GetFileAttributes(f) return FILE_ATTRIBUTE_ARCHIVE(0x20).

@mattn
Copy link
Member

@mattn mattn commented Jul 31, 2019

This is a code I used. https://gist.github.com/mattn/2dee75a1731bd471db44fc966a607f62

package main

import (
	"fmt"
	"log"
	"os"
	"syscall"
)

func main() {
	fi, err := os.Stat("server.sock")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("%x\n", fi.Sys().(*syscall.Win32FileAttributeData).FileAttributes)
}

@katiehockman katiehockman changed the title os.ModeSocket is not set for domain socket files in Windows os: os.ModeSocket is not set for domain socket files in Windows Jul 31, 2019
@katiehockman
Copy link
Member

@katiehockman katiehockman commented Jul 31, 2019

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jul 31, 2019

@ianlancetaylor ianlancetaylor added this to the Go1.14 milestone Jul 31, 2019
@ddebroy
Copy link
Author

@ddebroy ddebroy commented Aug 1, 2019

We can check the ReparsePoint data associated with the file and match the ReparsePoint Tag with that of domain sockets:

> fsutil reparsepoint query C:\temp\go2.sock
Reparse Tag Value : 0x80000023
Tag value: Microsoft

Reparse Data Length: 0x00000000

While Microsoft documents some of the tags, the value 0x80000023 does not appear to show up in any of the published docs yet.

@0xbadfca11
Copy link

@0xbadfca11 0xbadfca11 commented Aug 1, 2019

As far as I know, until Windows 10 version 1809 (include Windows Server 2019) creates unix domain socket as a reparse point with IO_REPARSE_TAG_AF_UNIX tag.
Starting with Windows 10 version 1903 creates as a regular file.

@mattn
Copy link
Member

@mattn mattn commented Aug 1, 2019

I used fsutil. but another result.

C:\dev\c-sandbox\af_unix>fsutil reparsepoint query server.sock
Error:  The file or directory is not a reparse point.

@ddebroy
Copy link
Author

@ddebroy ddebroy commented Aug 12, 2019

I got confirmation from Microsoft people in the Kubernetes SIG-Windows community that the 1903 behavior where the reparse point is not being reported for domain sockets is a bug (being tracked to be fixed internally at Microsoft)

For now, I am able to have something like this:

const (
	FSCTL_GET_REPARSE_POINT          = 0x900A8
	MAXIMUM_REPARSE_DATA_BUFFER_SIZE = 16 * 1024
	IO_REPARSE_TAG_SOCKET            = 0x80000023
)

type REPARSE_DATA_BUFFER_HEADER struct {
	ReparseTag        uint32
	ReparseDataLength uint16
	Reserved          uint16
}

type REPARSE_DATA_BUFFER struct {
	header REPARSE_DATA_BUFFER_HEADER
	detail [MAXIMUM_REPARSE_DATA_BUFFER_SIZE]byte
}

func IsSocketFile(filePath string) (bool, error) {
	fd, err := windows.CreateFile(windows.StringToUTF16Ptr(filePath), windows.GENERIC_READ, 0, nil, windows.OPEN_EXISTING, windows.FILE_FLAG_OPEN_REPARSE_POINT|windows.FILE_FLAG_BACKUP_SEMANTICS, 0)
	if err != nil {
		return false, errors.Wrap(err, "CreateFile failed")
	}
	defer windows.CloseHandle(fd)

	rdbbuf := make([]byte, MAXIMUM_REPARSE_DATA_BUFFER_SIZE)
	var bytesReturned uint32
	if err = windows.DeviceIoControl(fd, windows.FSCTL_GET_REPARSE_POINT, nil, 0, &rdbbuf[0], uint32(len(rdbbuf)), &bytesReturned, nil); err != nil {
		return false, errors.Wrap(err, "FSCTL_GET_REPARSE_POINT failed")
	}
	rdb := (*REPARSE_DATA_BUFFER)(unsafe.Pointer(&rdbbuf[0]))
	if rdb.header.ReparseTag == IO_REPARSE_TAG_SOCKET {
		return true, nil
	}
	return false, nil
}

return me the data I want in Windows 2019 LTS (following the pattern in ReadLink)

@mattn
Copy link
Member

@mattn mattn commented Aug 13, 2019

@ddebroy Thanks for the confirmation.

I guess that GetFileAttributes will return correct value (FILE_ATTRIBUTE_REPARSE_POINT) when the bug of 1903 will be fixed. However, until this bug is really fixed, nothing can be done for Go standard package.

@alexbrainman
Copy link
Member

@alexbrainman alexbrainman commented Aug 18, 2019

fsutil reparsepoint query C:\temp\go2.sock
Reparse Tag Value : 0x80000023
Tag value: Microsoft

Reparse Data Length: 0x00000000

That is what I see here too. I am on Version 1809 of Windows 10.

While Microsoft documents some of the tags, the value 0x80000023 does not appear to show up in any of the published docs yet.

I agree. 0x80000023 is missing from that page.

I got confirmation from Microsoft people in the Kubernetes SIG-Windows community that the 1903 behavior where the reparse point is not being reported for domain sockets is a bug (being tracked to be fixed internally at Microsoft)

Do you know what is the bug? Because we could adjust current os.Stat to look for 0x80000023 in reparse point tag and translate it into os.ModeSocket.

I guess that GetFileAttributes will return correct value (FILE_ATTRIBUTE_REPARSE_POINT) when the bug of 1903 will be fixed. However, until this bug is really fixed, nothing can be done for Go standard package.

I can see GetFileAttributesExW with GetFileExInfoStandard returns FILE_ATTRIBUTE_ARCHIVE | FILE_ATTRIBUTE_REPARSE_POINT for WIN32_FILE_ATTRIBUTE_DATA.dwFileAttributes field. So, if we assume that sockets will return 0x80000023 reparse tag, that should be enough to identify sockets. No?

Alex

@mattn
Copy link
Member

@mattn mattn commented Aug 18, 2019

@alexbrainman Yes, 1809 return correct value. But 1903 does not return correct value. Currently, we can't know when this bug will be fixed.

@alexbrainman
Copy link
Member

@alexbrainman alexbrainman commented Aug 21, 2019

Yes, 1809 return correct value. But 1903 does not return correct value.

I don't have 1903 to test, so I would not know what the problem is there.

Regardless, we still need to do something here. I don't see how os.ModeSocket would suddenly appear in os.Stat after Microsoft fixes some bug.

Alex

@rsc rsc removed this from the Go1.14 milestone Oct 9, 2019
@rsc rsc added this to the Backlog milestone Oct 9, 2019
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 5, 2020
Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
right mode(socket) on windows. Hence deleting the file, without checking whether
its a socket, on windows.
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 6, 2020
- Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
  right mode(socket) on windows. Hence deleting the file, without checking whether
  its a socket, on windows.

- Place os specific logic into util_{linux, windows} files.
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 6, 2020
- Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
  right mode(socket) on windows. Hence deleting the file, without checking whether
  its a socket, on windows.

- Place os specific logic into util_{linux, windows} files.
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 6, 2020
- Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
  right mode(socket) on windows. Hence deleting the file, without checking whether
  its a socket, on windows.

- Place os specific logic into util_{linux, windows} files and move util under pkg.
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 11, 2020
- Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
  right mode(socket) on windows. Hence deleting the file, without checking whether
  its a socket, on windows.

- Place os specific logic into util_{linux, windows} files and move util under pkg.
kkmsft added a commit to kkmsft/node-driver-registrar that referenced this issue Feb 11, 2020
- Until the bug - golang/go#33357 is fixed, os.stat wouldn't return the
  right mode(socket) on windows. Hence deleting the file, without checking whether
  its a socket, on windows.

- Place os specific logic into util_{linux, windows} files and move util under pkg.
@djdv
Copy link
Contributor

@djdv djdv commented Mar 2, 2021

@mattn cc: @alexbrainman
I've run your C server, and the Go snippet here: #33357 (comment)
And am now encountering different results. os.Stat is returning a system error: ERROR_CANT_ACCESS_FILE errno 1920 (0x780) "The file cannot be accessed by the system."
Independent of this, I first saw this problem when using my own server written in Go which hosts the Unix socket.
In both cases, the socket's permission's allow all operations, and are valid to dial and use, but os.Stat returns an access error regardless.

This is on Go version 1.16, Windows version 19042.844.
I have not tested the same scenarios on 1809 or 1903 or older versions of Go, but according to the posts I assume it worked there previously in some capacity.
If you need, I can run whatever tests in these environments.

Edit:
Decided to test this in a VM and got the same results.
Screencast related https://www.youtube.com/watch?v=XaKrxpxrbtY

@networkimprov
Copy link

@networkimprov networkimprov commented Mar 2, 2021

cc @zx2c4

@zx2c4
Copy link
Contributor

@zx2c4 zx2c4 commented Mar 2, 2021

@mattn @ddebroy Is the suggestion here to augment os.Stat so that if GetFileAttributes returns something with FILE_ATTRIBUTE_REPARSE_POINT set, then we CreateFile(FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_OPEN_REPARSE_POINT), followed by a DeviceIoControl(FSCTL_GET_REPARSE_POINT), and then see what its tag info is?

If that series of steps works, it seems like that'd be feasible, since I assume we're already calling GetFileAttributes in os.Stat, so the additional syscalls would only incur the penalty on the unlikely chance that we're looking at a file with a reparse point.

@sschaap
Copy link

@sschaap sschaap commented Jun 5, 2021

@mattn @ddebroy Is the suggestion here to augment os.Stat so that if GetFileAttributes returns something with FILE_ATTRIBUTE_REPARSE_POINT set, then we CreateFile(FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_OPEN_REPARSE_POINT), followed by a DeviceIoControl(FSCTL_GET_REPARSE_POINT), and then see what its tag info is?

That approach would work, except that FILE_ATTRIBUTE_REPARSE_POINT is (incorrectly) not reported for UNIX sockets on some Windows 10 versions. As noted earlier, this affects at least version 1903 and later. It looks like all functions using the WIN32_FIND_DATA struct erroneously do not report this flag for UNIX sockets. This affects at least the GetFileAttributes, GetFileAttributesEx, and Find(First|Next)File functions.

That said, obtaining a file handle using CreateFile(FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_OPEN_REPARSE_POINT) followed by DeviceIoControl(FSCTL_GET_REPARSE_POINT) will either retrieve the reparse tag or (for non-reparse points) report ERROR_NOT_A_REPARSE_POINT using windows.Errno(4390) which can be tested for.

@mattn
Copy link
Member

@mattn mattn commented Jun 5, 2021

This change works well for me. But I wonder that Go have the way to create the file with AF_UNIX attribute.

diff --git a/src/os/stat_windows.go b/src/os/stat_windows.go
index da4c49090e..22c78b8380 100644
--- a/src/os/stat_windows.go
+++ b/src/os/stat_windows.go
@@ -42,6 +42,41 @@ func (file *File) Stat() (FileInfo, error) {
 	return fs, err
 }
 
+const (
+	_MAXIMUM_REPARSE_DATA_BUFFER_SIZE = 16 * 1024
+	_IO_REPARSE_TAG_SOCKET            = 0x80000023
+)
+
+type _REPARSE_DATA_BUFFER_HEADER struct {
+	ReparseTag        uint32
+	ReparseDataLength uint16
+	Reserved          uint16
+}
+
+type _REPARSE_DATA_BUFFER struct {
+	header _REPARSE_DATA_BUFFER_HEADER
+	detail [_MAXIMUM_REPARSE_DATA_BUFFER_SIZE]byte
+}
+
+func isSocketFile(name string) (bool, error) {
+	fd, err := syscall.CreateFile(syscall.StringToUTF16Ptr(name), syscall.GENERIC_READ, 0, nil, syscall.OPEN_EXISTING, syscall.FILE_FLAG_OPEN_REPARSE_POINT|syscall.FILE_FLAG_BACKUP_SEMANTICS, 0)
+	if err != nil {
+		return false, err
+	}
+	defer syscall.CloseHandle(fd)
+
+	rdbbuf := make([]byte, _MAXIMUM_REPARSE_DATA_BUFFER_SIZE)
+	var bytesReturned uint32
+	if err = syscall.DeviceIoControl(fd, syscall.FSCTL_GET_REPARSE_POINT, nil, 0, &rdbbuf[0], uint32(len(rdbbuf)), &bytesReturned, nil); err != nil {
+		return false, err
+	}
+	rdb := (*_REPARSE_DATA_BUFFER)(unsafe.Pointer(&rdbbuf[0]))
+	if rdb.header.ReparseTag == _IO_REPARSE_TAG_SOCKET {
+		return true, nil
+	}
+	return false, nil
+}
+
 // stat implements both Stat and Lstat of a file.
 func stat(funcname, name string, createFileAttrs uint32) (FileInfo, error) {
 	if len(name) == 0 {
@@ -59,20 +94,34 @@ func stat(funcname, name string, createFileAttrs uint32) (FileInfo, error) {
 	// See https://golang.org/issues/19922#issuecomment-300031421 for details.
 	var fa syscall.Win32FileAttributeData
 	err = syscall.GetFileAttributesEx(namep, syscall.GetFileExInfoStandard, (*byte)(unsafe.Pointer(&fa)))
-	if err == nil && fa.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT == 0 {
-		// Not a symlink.
-		fs := &fileStat{
-			FileAttributes: fa.FileAttributes,
-			CreationTime:   fa.CreationTime,
-			LastAccessTime: fa.LastAccessTime,
-			LastWriteTime:  fa.LastWriteTime,
-			FileSizeHigh:   fa.FileSizeHigh,
-			FileSizeLow:    fa.FileSizeLow,
+	if err == nil {
+		if fa.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT == 0 {
+			// Not a symlink.
+			fs := &fileStat{
+				FileAttributes: fa.FileAttributes,
+				CreationTime:   fa.CreationTime,
+				LastAccessTime: fa.LastAccessTime,
+				LastWriteTime:  fa.LastWriteTime,
+				FileSizeHigh:   fa.FileSizeHigh,
+				FileSizeLow:    fa.FileSizeLow,
+			}
+			if err := fs.saveInfoFromPath(name); err != nil {
+				return nil, err
+			}
+			return fs, nil
 		}
-		if err := fs.saveInfoFromPath(name); err != nil {
-			return nil, err
+
+		if ok, err := isSocketFile(name); err == nil && ok {
+			fs := &fileStat{
+				FileAttributes: fa.FileAttributes,
+				CreationTime:   fa.CreationTime,
+				LastAccessTime: fa.LastAccessTime,
+				LastWriteTime:  fa.LastWriteTime,
+				FileSizeHigh:   fa.FileSizeHigh,
+				FileSizeLow:    fa.FileSizeLow,
+			}
+			return fs, nil
 		}
-		return fs, nil
 	}
 	// GetFileAttributesEx fails with ERROR_SHARING_VIOLATION error for
 	// files, like c:\pagefile.sys. Use FindFirstFile for such files.
diff --git a/src/os/types_windows.go b/src/os/types_windows.go
index 59bf5ca381..15a920a7f6 100644
--- a/src/os/types_windows.go
+++ b/src/os/types_windows.go
@@ -121,6 +121,9 @@ func (fs *fileStat) Mode() (m FileMode) {
 	if fs.isSymlink() {
 		return m | ModeSymlink
 	}
+	if fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT != 0 {
+		return m | ModeSocket
+	}
 	if fs.FileAttributes&syscall.FILE_ATTRIBUTE_DIRECTORY != 0 {
 		m |= ModeDir | 0111
 	}

@djdv
Copy link
Contributor

@djdv djdv commented Jul 27, 2021

@mattn one of your related CL's was mentioned here:
microsoft/Windows-Containers#97 (comment)

Commenting so Github links them together.

But I wonder that Go have the way to create the file with AF_UNIX attribute.

Your patch is working for me with a socket created by net.Listen
Environment: Windows 11 (22000.100)

package main

import (
	"fmt"
	"io/fs"
	"log"
	"net"
	"os"
)

func main() {
	log.SetFlags(log.Lshortfile)

	socketFilePath := "uds"

	unixListener, err := net.Listen("unix", socketFilePath)
	if err != nil {
		log.Fatal(err)
	}
	defer unixListener.Close()

	stat, err := os.Stat(socketFilePath)
	if err != nil {
		log.Print(err)
		return
	}

	fileMode := stat.Mode()
	fmt.Println(fileMode)
	fmt.Println("Has socket bit?", fileMode&fs.ModeSocket != 0)
}
> go version
go version go1.16.6 windows/amd64
> go run .\main.go
main.go:24: CreateFile uds: The file cannot be accessed by the system.
> go-tip
> go version
go version devel go1.17-7cd10c1149 Tue Jul 27 22:01:54 2021 +0000 windows/amd64
> go run .\main.go
Srw-rw-rw-
Has socket bit? true

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 28, 2021

Change https://golang.org/cl/338069 mentions this issue: os,net: fix file mode in unix domain socket on Windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet