path/filepath: Glob matching does not work on Windows with long path workaround

[AudriusButkevicius]

If you run this:

package main

import(
    "fmt"
    "os"
    "io/ioutil"
    "path/filepath"
)

func main(){
    dir, err := ioutil.TempDir("", "")
    if err != nil {
        panic(err)
    }

    longdir := "\\\\?\\" + dir

    for _, file := range []string{"foo", "boo", "moo", "vex", "rex"} {
        fd, err := os.Create(filepath.Join(longdir, file))
        if err != nil {
            panic(err)
        }
        fd.Close()
    }

    matches, err := filepath.Glob(filepath.Join(dir, "?oo"))
    if err != nil {
        panic(err)
    }


    longmatches, err := filepath.Glob(filepath.Join(longdir, "?oo"))
    if err != nil {
        panic(err)
    }

    fmt.Println("Matches:", len(matches), "Long matches:", len(longmatches))
}

Expected output would be:
Matches: 3 Long matches: 3
Actual output is:
Matches: 3 Long matches: 0

The magical \\?\ prefix is a workaround for 252 character filepath limits on the ancient Win32 API.

Verified in go 1.4.2

[AudriusButkevicius]

This seems to be sufficient to fix it.
path/filepath/match.go

 // hasMeta returns true if path contains any of the magic characters
 // recognized by Match.
 func hasMeta(path string) bool {
+   // Strip off Windows long path prefix if it exists.
+   if runtime.GOOS == "windows" && strings.HasPrefix(path, "\\\\?\\") {
+       path = path[4:]
+   }
    // TODO(niemeyer): Should other magic characters be added here?
    return strings.IndexAny(path, "*?[") >= 0
 }

[alexbrainman]

I am bit worried about using long paths. Not all programs support them. As far as I remember simple shell commands fail with long paths. Why do you need long path support?

Alex

[AudriusButkevicius]

Well, the rest of the stdlib works with long paths, so I don't see why Glob shouldn't.

Especially because you are not using some win32/magic which doesn't support it, but have your own thing which does the job.

Furthermore, Glob has nothing todo with long shell commands, it's a convenience function for finding files, plus it's broken even for short path if you happen to have the prefix by accident/default.

[alexbrainman]

... the rest of the stdlib works with long paths, ...

I don't think it is true. There are many problems with long paths. That is why I think we should support them as little as possible. For example, why this test http://play.golang.org/p/uF7_491yCT fails:

C:\go\path\mine\src\foo>go test -v
=== RUN TestNormal
--- PASS: TestNormal (0.00s)
=== RUN TestLong
--- FAIL: TestLong (0.00s)
        main_test.go:24: open hello.txt: The system cannot find the path specified.
FAIL
exit status 1
FAIL    foo     0.109s

C:\go\path\mine\src\foo>

But, if you insist, send a change http://golang.org/doc/contribute.html Others might think differently.

Alex

[rsc]

Too late for Go 1.5, but Alex I would like to understand why your test program fails like that. It looks like Chdir succeeds but then trying to write to 'hello.txt' in the current directory fails? How is that possible? Are we using the wrong system call in OpenFile or something like that? Thanks.

[alexbrainman]

@rsc

I don't know much about this subject. See MSDN: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx - "Maximum Path Length Limitation" section in particular.

Also http://blogs.msdn.com/b/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx might explain this somewhat.

Alex

[gopherbot]

CL https://golang.org/cl/32451 mentions this issue.