os: ReadDir on regular file returns ERROR_PATH_NOT_FOUND instead of ENOTDIR

[codefromthecrypt]

What version of Go are you using (go version)?

$ go version
go version go1.16.5 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\moo\AppData\Local\go-build
set GOENV=C:\Users\moo\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\moo\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\moo\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\moo\.gimme\versions\go1.16.5.windows.amd64
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\moo\.gimme\versions\go1.16.5.windows.amd64\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.16.5
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\moo\oss\getenvoy\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\moo\AppData\Local\Temp\go-build3391750208=/tmp/go-build -gno-record-gcc-switches

What did you do?

	files, err := os.ReadDir(filePath)

What did you expect to see?

err should be ENOTDIR

What did you see instead?

os.IsNotExist(err) matches because it was ERROR_PATH_NOT_FOUND

[codefromthecrypt]

the problem is here

go/src/os/dir_windows.go

Line 14 in 2ebe77a

if !file.isdir() {

the state check isn't valid for the call site

a lazy double-check like darwin is one way out, but up to you!

go/src/os/dir_darwin.go

Line 28 in 2ebe77a

if f.dirinfo == nil {

[codefromthecrypt]

specifically, in windows isdir is only valid if the file was opened as a directory, which isn't the case when there was a file there https://github.com/golang/go/blob/master/src/os/file_windows.go#L102

[networkimprov]

cc @alexbrainman @bcmills @zx2c4 @mattn

@dmitshur re incomplete triage

[alexbrainman]

What did you do?

	files, err := os.ReadDir(filePath)

That is not enough to reproduce this issue.

@codefromthecrypt can you, please, provide complete program with steps to reproduce?

Thank you.

Alex

[iwdgo]

Reading the line below indicates that ENOTDIR is already returned.

go/src/os/dir_windows.go

Lines 14 to 15 in 2ebe77a

	if !file.isdir() {
	return nil, nil, nil, &PathError{Op: "readdir", Path: file.name, Err: syscall.ENOTDIR}

This error could be checked with like recommended since #41122 was closed. documentation of os.IsNotExist(err) was improved. It recommends to use errors.Is(err, fs.ErrNotExist) on new code.

if errors.Is(err, syscall.ENOTDIR) {
    // Error handling
}

If walking the directory is the issue, then fs.Walkdir was added in go1.17.

It seems that there is nothing left to do for this issue.

[codefromthecrypt]

@iwdgo the comments you made rely on file.isdir working. Specifically, check the notes here, which are that file.isdir is the problem #46734 (comment)

[codefromthecrypt]

Here's the unit test, and on windows GitHub actions runner, it fails like this, but only on windows (doesn't fail on linux or macOS). Feel free too do whatever you want, just I owed a unit test from earlier. If you think this is ok, just close the issue.

--- FAIL: TestDirError (0.00s)
    dir_test.go:25: didn't want os.IsNotExist() on os.ReadDir(C:\Users\RUNNER~1\AppData\Local\Temp\TestDirError275160222\001/foo)

func TestDirError(t *testing.T) {
	dir := t.TempDir()
	fileNotDir := path.Join(dir, "foo")
	if os.WriteFile(fileNotDir, []byte{}, 0600) != nil {
		t.Fatalf("cannot write %s", fileNotDir)
	}
	_, err := os.ReadDir(fileNotDir)
	if err == nil {
		t.Fatalf("wanted an error on os.ReadDir(%s)", fileNotDir)
	}
	pErr, ok := err.(*os.PathError)
	if !ok {
		t.Fatalf("wanted an os.PathError on os.ReadDir(%s)", fileNotDir)
	}
	if os.IsNotExist(err) {
		t.Fatalf("didn't want os.IsNotExist() on os.ReadDir(%s)", fileNotDir)
	}
	if pErr.Err != syscall.ENOTDIR {
		t.Fatalf("want syscall.ENOTDIR, have %s, on os.ReadDir(%s)", pErr.Err, fileNotDir)
	}
}

[iwdgo]

@codefromthecrypt Thanks. The provided test fails too on windows 10 21H1.
The discussed isdir() is in line with the purpose.

go/src/os/file_windows.go

Line 102 in 91e7821

func (f *file) isdir() bool { return f != nil && f.dirinfo != nil }

Some directory data is found which lets readdir() occur with a handle that triggers a misleading error. IMHO, there is something to fix. It seems related to the use of a temporary directory following some investigation.
My first idea was erroneous as this part is compile early in the toolchain and some errors are not reported as later in the toolchain.

[iwdgo]

Using == for type error is not recommended. Using something like below provides details on what is happening:

// Testing ENOTDIR first provides the detailed cause of the failure.
if !errors.Is(pErr.Err, syscall.ENOTDIR) {
		t.Fatalf("want syscall.ENOTDIR, have %s, on os.ReadDir(%s)", pErr.Err, fileNotDir)
}
if os.IsNotExist(err) {
		t.Fatalf("didn't want os.IsNotExist() on os.ReadDir(%s)", fileNotDir)
}

IsNotExist(err) reports true as the directory is not found which is ERROR_PATH_NOT_FOUND on windows.
It seems that issue can be closed.

[codefromthecrypt]

I think the investigation so far feels like someone trying hard to prove the current code is correct, but wholly missing that the overall experience isn't consistent. I would suggest another person take a look even if they also think the experience is ok to sacrifice and that the result should be telling people one by one that they shouldn't call == on error (which is done incidentally in tests etc).