path/filepath: Glob fails and hangs on Windows when using character ranges

[neclepsio]

What version of Go are you using (go version)?

$ go version
go version go1.16.4 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\...\AppData\Local\go-build
set GOENV=C:\Users\...\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=c:\...\Go-Modules\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=c:\Ignazio\Personale\Go-Modules
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\...\Go\go1.16.4
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\...\Go\go1.16.4\pkg\tool\windows_amd64
set GOVERSION=go1.16.4
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\...\Go-Modules\src\...\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\...\AppData\Local\Temp\go-build3528342390=/tmp/go-build -gno-record-gcc-switches

What did you do?

I am trying to get case insensitive results for filepath.Glob() on Windows. To do so, I'm transforming each letter in a character range for uppercase and lowercase. See #5441.

What did you expect to see?

The same results as when using pattern exactly matching an existing file.

What did you see instead?

\\server\share\filename.ext and c:\dir\filename.ext are existing files.

filepath.Glob(`c:\dir\filename.ext`) correctly returns one result.

filepath.Glob(`[Cc]:\[Dd][Ii][Rr]\[fF][iI][lL][eE][nN][aA][mM][eE].[eE][xX][tT]`) returns no result, while it should return one result.

filepath.Glob(`C:\[Dd][Ii][Rr]\[fF][iI][lL][eE][nN][aA][mM][eE].[eE][xX][tT]`) correctly returns one result.

filepath.Glob(`\\server\share\filename.ext`) correctly returns one result.

filepath.Glob(`\\[sS][eE][rR][vV][eE][rR]\[sS][hH][aA][rR][eE]\[fF][iI][lL][eE][nN][aA][mM][eE].[eE][xX][tT]`) returns no result, while it should return one result.

filepath.Glob(`\\server\share\[fF][iI][lL][eE][nN][aA][mM][eE].[eE][xX][tT]`) correctly returns one result.

filepath.Glob(`\\server\[sS][hH][aA][rR][eE]\[fF][iI][lL][eE][nN][aA][mM][eE].[eE][xX][tT]`) hangs, and it should not.

[mknyszek]

CC @robpike @rsc via https://dev.golang.org/owners

[antong]

The idea with Glob is that when a path component has a pattern, Glob lists the containing directory, and checks whether entries match the pattern. (It does not try to first expand the pattern to all possible candidate paths, and then check whether these potential matches have corresponding files. That would be very inefficient or even impossible for the wildcard "*".) If listing a directory causes an error, that directory is skipped, the error is ignored, and no results are returned for that directory. If listing a directory "hangs", then Glob will also hang.

For instance Glob(`\\serv?r\*.*`) will try to list "\\", which gives an error. Glob(`\\server\share[123]\*.*`) will try to list "\\server\", which can take a very long time to complete or error out (hang). @neclepsio , I hope this helps to explain why the cases you say have incorrect results behave the way they do. And also, it could hint at a workaround that may work for you: As you say you are working on case-insensitive windows, you don't have to do case insensitive matching on intermediate directories where you want only one hit, and doing so only slows things down. So, instead of Glob(`[Ff][Oo][Oo]\*.[Tt][Xx][Tt]`), try Glob(`foo\*.[Tt][Xx][Tt]`). Instead of Glob(`\\[Ss][Rr][Vv]\[Ss][Hh][Aa][Rr][Ee]\*.go`), try Glob(`\\srv\share\*.[Gg][Oo]`).

Now I'm not really sure what to think about Glob(`[CD]:\*.*`), i.e., trying to match on the drive letter. This doesn't work today. Case sensitivity isn't the issue, you could always use capital drive letters. It could be possible to list Windows logical drives (I don't think Go has an interface for this currently) and match against a pattern. But right now Glob doesn't detect that the first path component can have a drive letter if it has meta characters (a pattern). I think this is such a niche special case that it may not be worth trying to do it.

[neclepsio]

Thank you. Now I understand why this happens, but it's a different behaviour from what's in documents, which state the syntax of patterns is the same as filepath.Match, and, for example, filepath.Match("[Cc]:\test.txt", "c:\test.txt") returns true. So one between code and docs should be fixed.

As for listing shares of the server in case of \\srv\[Ss][Hh][Aa][Rr][Ee], Explorer takes less than a second to show them all in my case, so I don't think it's the reason filepath.Glob hangs.