Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "go get: unzip: malformed file path: double dot" #27299

Open
sirkon opened this issue Aug 28, 2018 · 8 comments

Comments

Projects
None yet
9 participants
@sirkon
Copy link

commented Aug 28, 2018

  • go version

    go version go1.11 linux/amd64
    

    downloaded yesterday (27 august)

  • $ go env
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/home/emacs/.cache/go-build"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOOS="linux"
    GOPATH="/home/emacs/go"
    GOPROXY=""
    GORACE=""
    GOROOT="/usr/local/go"
    GOTMPDIR=""
    GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    CC="gcc"
    CXX="g++"
    CGO_ENABLED="1"
    GOMOD="/home/emacs/Desktop/main/go.mod"
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build252846370=/tmp/go-build -gno-record-gcc-switches"
    
  • I am trying to use ANTLR4 runtime support via go modules, steps to reproduce:

    1. Choose some directory
    2. Create new Go package/module:
      mkdir newpack
      cd newpack
      go mod init main
      
    3. Try to get ANTLR4 runtime support module
      go get github.com/antlr/antlr4/runtime/Go/antlr
      
    4. Obviously, successful download was expected, got an error message instead:
      go: finding github.com/antlr/antlr4/runtime/Go/antlr latest
      go: finding github.com/antlr/antlr4/runtime/Go latest
      go: finding github.com/antlr/antlr4/runtime latest
      go: finding github.com/antlr/antlr4 latest
      go: extracting github.com/antlr/antlr4 v0.0.0-20180728001836-7d0787e29ca8
      -> unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot
      go get github.com/antlr/antlr4/runtime/Go/antlr: unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot
      

    I guess double dots in some unrelated file is hardly a reason to diagnose a error

@sirkon sirkon changed the title go modules related `go get` issue [go modules] go get error Aug 29, 2018

@FiloSottile FiloSottile changed the title [go modules] go get error cmd/go: "go get: unzip: malformed file path: double dot" Aug 30, 2018

@FiloSottile FiloSottile added this to the Go1.12 milestone Aug 30, 2018

@sjbodzo

This comment has been minimized.

Copy link

commented Sep 5, 2018

I am also facing this issue. Since DynamoDB uses ANTLR to parse expression grammar, this effectively means we cannot use the aws-sdk in Golang with go modules until there is a fix.

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 24, 2018

Consolidating into #28001.

@bcmills bcmills closed this Oct 24, 2018

@bcmills bcmills reopened this Dec 6, 2018

klingtnet added a commit to klingtnet/klingt.net that referenced this issue Dec 15, 2018

Update almost all services
- installation instruction of go-bindata has changed, see
golang/go#27299 for details
- using a item-loop for installing multiple apt packages is deprecated,
instead an array of package names should be used
- remove filebrowser plugin from caddy since it's broken: filebrowser/filebrowser#554
- hash user password
@odeke-em

This comment has been minimized.

Copy link
Member

commented Jan 20, 2019

Hey @bcmills @rsc, I realize that perhaps we plainly reject all double-dot-containing paths because we are trying to avoid trafficking/arbitrary/malicious filesystem access e.g. "foo/../secrets" where ".." contains the keys to the kingdom.
However, this seems broadly sweeping and I think we can make an exception to this rule like other filesystems to, by checking that the base of the path doesn't just consist of
dots(after checking if the entire path has "..") so basically something like this

        if strings.Contains(path, "..") {
                // Rejecting paths with ".." in them is a security consideration meant to
                // prevent arbitrary filesystem accesses. However, the exception to
                // this rule is that if the path's base consists of more
                // than just "."+ which would disambiguate between ".", ".." and valid
                // paths, then accept it.
                base := filepath.Base(path)
                if strings.Count(base, ".") == len(base) {
                        // Consists entirely of "."*
                        return fmt.Errorf("double dot")
                }
        }

which would match and allow say "foo..ps..mp4" as well as the bug report from the ANTLR parser generator.

I realize that we are almost releasing Go1.12 and perhaps could punt this to Go1.13 for safety but if this issue is deemed high priority perhaps let's discuss this -- also note that the ANTLR project replace the double dots that triggered this bug report, in August 2018(right after this issue was filed) as per antlr/antlr4#2341 but there might be other projects like this in the wild.

I'll also kindly page @mikesamuel for an evaluation of the security considerations of how I've proposed accepting ".." in the base.

@DanielJonesEB

This comment has been minimized.

Copy link

commented Mar 14, 2019

Is there a workaround for this issue? Hitting it trying to depend on https://github.com/SpectoLabs/hoverfly

go: extracting github.com/SpectoLabs/hoverfly v0.17.7
-> unzip /Users/deejay/go/pkg/mod/cache/download/github.com/!specto!labs/hoverfly/@v/v0.17.7.zip: malformed file path "core/handlers/v2/hoverfly_upstream_proxy_handler..go": double dot
@sirkon

This comment has been minimized.

Copy link
Author

commented Mar 14, 2019

@DanielJonesEB good workaround is to create an issue in the project

@DanielJonesEB

This comment has been minimized.

Copy link

commented Mar 15, 2019

good workaround is to create an issue in the project

Sadly I'm depending on an old version of the library, so we can't go back in time to fix it.

@keitwb

This comment has been minimized.

Copy link

commented Mar 17, 2019

Found this same problem in the latest version of the Telegraf repo. The use of double dots appears to be perfectly legitimate since they are testing that the config loader doesn't descend into directories that start with double dots. I guess the test could be changed to dynamically create the double dot directory to avoid it being committed to the repo, but would be nice not to have to and still work with modules.

@zouyee

This comment has been minimized.

Copy link

commented Mar 28, 2019

go get -u golang.org/x/tools/... found the same issue.

stefan-improbable added a commit to stefan-improbable/go-flagz that referenced this issue Apr 25, 2019

Add support for go modules.
- create go.sum
- do not use .. in filenames in this repo due to golang/go#27299

Testing strategy: I almost successfully ran `test_all.sh` and found a
data race, but I don't think it's related to this PR:

```
WARNING: DATA RACE
Read at 0x00c00016cac0 by goroutine 29:
  github.com/spf13/pflag.(*FlagSet).VisitAll()
      /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:277 +0x14a
  github.com/mwitkow/go-flagz.ChecksumFlagSet()
      /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8
  github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect()
      /home/stefan/Projects/go-flagz/monitoring/collector.go:57 +0x2a2
  github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
      /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb

Previous write at 0x00c00016cac0 by goroutine 28:
  github.com/spf13/pflag.sortFlags()
      /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:204 +0x2f2
  github.com/spf13/pflag.(*FlagSet).VisitAll()
      /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:270 +0x1b0
  github.com/mwitkow/go-flagz.ChecksumFlagSet()
      /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8
  github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect()
      /home/stefan/Projects/go-flagz/monitoring/collector.go:55 +0xe9
  github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
      /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb
```

umk added a commit to umk/go-dymessage that referenced this issue May 21, 2019

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.