Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: malformed file path: invalid char ':' #28001

Closed
vitarb opened this issue Oct 4, 2018 · 20 comments

Comments

Projects
None yet
@vitarb
Copy link

commented Oct 4, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.11.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/vit/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/vit/go-code"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/vit/go-code/devexp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6_/htkppg753px1jglygs0c297h0000gn/T/go-build852474991=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Try consuming a repo which has non-go files that contain ":" in their filenames.

What did you expect to see?

Repo can be imported as a dependency.

What did you see instead?

Error that says:
malformed file path ... invalid char ':'

Notes:

This issue should be similar to the #27642

Would it be possible to relax some of these naming restrictions?
Having to work with third-party package owners for each dependency that might contain these special symbols in filenames sounds like a lot of pain.

@vitarb vitarb changed the title go mod vendor: malfored file path: invalid char ':' go mod vendor: malformed file path: invalid char ':' Oct 4, 2018

@mvdan

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

Somewhat related to #26912, but that was about a replace directive and this is a bout a filename.

/cc @bcmills

@mvdan mvdan changed the title go mod vendor: malformed file path: invalid char ':' cmd/go: malformed file path: invalid char ':' Oct 4, 2018

@myitcv

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

Sounds like (away from keyboard) a dup of #26672

@mvdan

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

You're right. But perhaps the workaround there isn't satisfactory here - let's see what @vitarb has to say.

@bcmills bcmills added this to the Unplanned milestone Oct 4, 2018

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

Would it be possible to relax some of these naming restrictions?

Modules are intended to be portable across platforms, and the : character is reserved on Windows.

@DeedleFake

This comment has been minimized.

Copy link

commented Oct 4, 2018

He's saying that it's non-Go files in the repo that are causing it, though, not the module path itself, or even files that the toolchain is going to care about directly. It looks like Git for Windows just ignores the files, which means that they'll wind up deleted if you commit and you're not paying attention. Since these are apparently files that aren't an issue in terms of importing the repository and the module is presumably being cloned into the cache where commits are fairly unlikely to happen, maybe the tools could complain only if it's a Go, C, or assembly file that's illegal?

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

If they are non-Go files needed for testing some Go package, then go test all will erroneously report test failures if the package is used.

If they are non-Go files that are also not needed for testing any Go package, why can't they be pruned out of the Go module in the first place (e.g. by adding another go.mod)?

@vitarb

This comment has been minimized.

Copy link
Author

commented Oct 4, 2018

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 24, 2018

if you think that community will figure this out and will clean up code over time then I'm okay with it.

We've only had a few related reports so far, so it doesn't seem to be a major pain point yet (and should only improve over time). Let's keep the validation as-is, and we can revisit if we find that there are a lot more repos that can't be converted to modules.

@bcmills bcmills closed this Oct 24, 2018

@glasser

This comment has been minimized.

Copy link
Contributor

commented Oct 24, 2018

Note that #27299 had a very specific use case: Antlr.

@mewmew

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2018

I was hit by this today as well, when trying to get https://github.com/scottshotgg/express_old working with Go modules.

Instead of failing with an unzip malformed file path error, would it be possible to fall back to using Git instead to download the module instead of relying on ZIP? It seem quite likely that projects such as Antlr may take some time to update, since Go is not their main language (Java is in the case of Antlr).

[u@x1 express_old]$ go install ./...
go: finding github.com/scottshotgg/Express/parse latest
go: finding github.com/scottshotgg/Express/lex latest
go: finding github.com/scottshotgg/Express/token latest
go: finding github.com/scottshotgg/Express/program latest
go: finding github.com/scottshotgg/Express/c latest
go: finding github.com/scottshotgg/Express latest
go: extracting github.com/scottshotgg/Express v0.0.0-20180928040624-b4a2644154a7
-> unzip /home/u/goget/pkg/mod/cache/download/github.com/scottshotgg/!express/@v/v0.0.0-20180928040624-b4a2644154a7.zip: malformed file path "build/dist/linux/386/ecc_386_24.08.18-20:25.exe": invalid char ':'
can't load package: package github.com/scottshotgg/express: C++ source files not allowed when not using cgo or SWIG: main.expr.cpp operatorOverload.cpp
main.go:7:2: unknown import path "github.com/scottshotgg/Express/c": cannot find module providing package github.com/scottshotgg/Express/c
main.go:8:2: unknown import path "github.com/scottshotgg/Express/lex": cannot find module providing package github.com/scottshotgg/Express/lex
main.go:9:2: unknown import path "github.com/scottshotgg/Express/parse": cannot find module providing package github.com/scottshotgg/Express/parse
main.go:10:2: unknown import path "github.com/scottshotgg/Express/program": cannot find module providing package github.com/scottshotgg/Express/program
main.go:11:2: unknown import path "github.com/scottshotgg/Express/token": cannot find module providing package github.com/scottshotgg/Express/token
@joerodriguez

This comment has been minimized.

Copy link

commented Nov 30, 2018

Running into the same issue trying to go get github.com/influxdata/telegraf:

± jr |master U:2 ✗| → go get github.com/influxdata/telegraf@master
go: finding github.com/influxdata/telegraf/filter master
go: finding github.com/influxdata/telegraf master
go: extracting github.com/influxdata/telegraf v0.10.2-0.20181130000756-1d6db08dc859
-> unzip /Users/josephrodriguez/.gvm/pkgsets/go1.11/global/pkg/mod/cache/download/github.com/influxdata/telegraf/@v/v0.10.2-0.20181130000756-1d6db08dc859.zip: malformed file path "internal/config/testdata/subconfig/..4984_10_04_08_28_06.119/invalid-config.conf": double dot
go get github.com/influxdata/telegraf/filter@master: unzip /Users/josephrodriguez/.gvm/pkgsets/go1.11/global/pkg/mod/cache/download/github.com/influxdata/telegraf/@v/v0.10.2-0.20181130000756-1d6db08dc859.zip: malformed file path "internal/config/testdata/subconfig/..4984_10_04_08_28_06.119/invalid-config.conf": double dot
@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 6, 2018

Reopened #27299. Per previous discussion with @rsc we might be able to allow names that contain double-dots along with non-dot characters.

We still can't allow :, though: that makes the filename invalid on Windows.

@guysmoilov

This comment has been minimized.

Copy link

commented Dec 14, 2018

Any chance someone could explain to me why this happens with go get github.com/jteeuwen/go-bindata/...? I can't find any double dots or : anywhere in that project.

go version go1.11.3 linux/amd64
package github.com/jteeuwen/go-bindata/...: github.com/jteeuwen/go-bindata/...: invalid import path: malformed import path "github.com/jteeuwen/go-bindata/...": double dot

I should point out that this only started happening to me since go 1.11.3, 1.11.2 worked fine.

@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 14, 2018

@guysmoilov That's a mostly-unrelated issue (#29247).

@guysmoilov

This comment has been minimized.

Copy link

commented Dec 14, 2018

@bcmills Thanks. So I take it that will be fixed soon? On which version?

@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 14, 2018

That will be in the 1.11.4 release, which I believe is in final testing now. (Should be out later today.)

@tekkamanendless

This comment has been minimized.

Copy link

commented Dec 31, 2018

I'm running into this with text files that happen to be in the repo's "docs" folder when I run go mod tidy:
unzip /home/xxx/go/pkg/mod/cache/download/github.com/vjeantet/goldap/@v/v0.0.0-20160521203625-ea702ca12a40.zip: malformed file path "doc/RFC 4511 - LDAP: The Protocol.txt": invalid char ':'

Is this merely cosmetic at this point? The exit value from go mod tidy is 0.

@tekkamanendless

This comment has been minimized.

Copy link

commented Dec 31, 2018

Hmm; maybe not. On my build server, running go install -v ./... gave me this output and a nonzero exit code:

go: downloading github.com/vjeantet/goldap v0.0.0-20160521203625-ea702ca12a40
-> unzip /go/pkg/mod/cache/download/github.com/vjeantet/goldap/@v/v0.0.0-20160521203625-ea702ca12a40.zip: malformed file path "doc/RFC 4511 - LDAP: The Protocol.txt": invalid char ':'
go: downloading github.com/influxdata/platform v0.0.0-20181228230219-3429e8d0c675
/go/pkg/mod/github.com/vjeantet/ldapserver@v0.0.0-20170919170217-479fece7c5f1/client.go:9:2: unknown import path "github.com/vjeantet/goldap/message": cannot find module providing package github.com/vjeantet/goldap/message
@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 9, 2019

Is this merely cosmetic at this point? The exit value from go mod tidy is 0.

See #27063.

keitwb added a commit to signalfx/telegraf that referenced this issue Mar 17, 2019

Rename testdata dir with double dots
Unfortunately due to a security check in Go modules (as of the latest
release), the existance of '..' in a file path in a repository causes
the module to be rejected when adding it with `go get`:
golang/go#28001 (comment).

This renames the testdata directory with the double dots and augments
the test code to rename it on the fly back to having two dots to get the
appropriate test behavior. This means the tests cannot run on a
read-only filesystem but hopefully that is not a requirement.

@keitwb keitwb referenced this issue Mar 17, 2019

Closed

Rename testdata dir with double dots #5596

3 of 3 tasks complete
@Prativa20

This comment has been minimized.

Copy link

commented Mar 19, 2019

encountering this error while using go get in one of the files in the docs folder

unzip malformed file path invalid char '™'
go version go1.11.5 darwin/amd64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.