Go 1.12b1 doesn't work due to files with utf8 chars

[steeve]

Go 1.12b1 doesn't work due to the presence of utf8 chars in certain files:

go_download_sdk(
    name = "go_sdk",
    sdks = {
        "darwin_amd64": ("go1.12beta1.darwin-amd64.tar.gz", "e49bf83ae10b2232d2efa918f0e9df1d76f93a0c6b0ea18c11edd9ef9defa505"),
        "linux_amd64": ("go1.12beta1.linux-amd64.tar.gz", "65bfd4a99925f1f85d712f4c1109977aa24ee4c6e198162bf8e819fdde19e875"),
    },
)

Bazel fails with:

ERROR: error loading package '': Encountered error while reading extension file 'go/def.bzl': no such package '@XXXX//go': no such package '@bazel_gazelle_go_repository_tools//': no such package '@go_sdk//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/io_bazel_rules_go/go/private/sdk.bzl", line 51
		_remote_sdk(ctx, [url.format(filename) for url...], <2 more arguments>)
	File "/private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/io_bazel_rules_go/go/private/sdk.bzl", line 113, in _remote_sdk
		ctx.download_and_extract(url = urls, stripPrefix = strip_pr..., ...)
java.io.FileNotFoundException: /private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/go_sdk/test/fixedbugs/issue27836.dir/A?foo.go (No such file or directory)
ERROR: error loading package '': Encountered error while reading extension file 'go/def.bzl': no such package '@XXXX//go': no such package '@bazel_gazelle_go_repository_tools//': no such package '@go_sdk//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/io_bazel_rules_go/go/private/sdk.bzl", line 51
		_remote_sdk(ctx, [url.format(filename) for url...], <2 more arguments>)
	File "/private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/io_bazel_rules_go/go/private/sdk.bzl", line 113, in _remote_sdk
		ctx.download_and_extract(url = urls, stripPrefix = strip_pr..., ...)
java.io.FileNotFoundException: /private/var/tmp/_bazel_steeve/309ac22c92b055a3e3f78825fe63406b/external/go_sdk/test/fixedbugs/issue27836.dir/A?foo.go (No such file or directory)

Note that this is a Bazel bug, which didn't get much traction, so I'm opening it here in the hopes it does.

Related issues are:

• download_and_extract: strange characters when unzip  bazel#6855
• Allow any characters in filenames / labels bazel#374

The offending directory is there: https://github.com/golang/go/tree/master/test/fixedbugs/issue27836.dir

[jayconrod]

Thanks for reporting this. Sorry it's taken me so long to triage.

This is a bug in Bazel, specifically on download_and_extract. I've opened a new issue as bazelbuild/bazel#7055. Honestly, I think it's a dup of bazelbuild/bazel#6855 because that one mentions go_repository, not a specific Bazel API. bazelbuild/bazel#374 is pretty general.

I'll see if I can come up with a workaround. I think we should be able to download the SDK with ctx.download then extract it with a shell command. Hopefully that will still hit the cache. I'll close this issue when the workaround is in place. The workaround will be cherry-picked to release branches ahead of 1.12 support.

[steeve]

We got the issue with another project on another dependency, downloaded via http_archive. We could fix it by applying a patch (via patches attr) that removed the offending files.

I think you may be fine if you remove that directory right after download_and_extract, since I believe patches are applied after it.

[jayconrod]

@steeve I think the error is coming from download_and_extract itself: Bazel is showing a Starlark stack on that line. My workaround replaces the call to download_and_extract with a simple download (which I think should use the same cache), then extracts it using the system tar. I'm only enabling this for macOS; Linux and Windows don't seem affected, and I'm not sure if Windows will always have tar.

The problem you described in your last comment seems like bazelbuild/bazel#374. I hope that's also fixed soon, but I don't think it will block people from using Go 1.12, since the "bad" file is not part of the build.

[steeve]

Good call @jayconrod, I think the workaround will do just fine in the mean time.
Although I cannot explain why http_archive.patches did work on our other project which had the same issue. Oh well.

[bttk]

It's not only macOS. I have a similar issue on Linux: bazelbuild/bazel#7055 (comment)