always produce .a files at the beginning of a build #3385

matloob · 2022-12-06T22:22:21Z

This change updates rules_go in anticipation for Go 1.20, which will ship without .a files for the standard library. Now, instead of conditionally building the standard library if the configuration differs from the default, the standard library will always be built. In addition, the builder will be built using 'go build' instead of individual invocations of go tool compile/link to avoid needing to bootstrap the standard library before producing the builder.

What type of PR is this?

Uncomment one line below and remove others.

Other

What does this PR do? Why is it needed?

It updates rules_go to run a go install to produce stdlib .a files (which won't be shipped with 1.20). It's necessary to get rules_go working with 1.20.

Which issues(s) does this PR fix?

Fixes #

Other notes for review

This change updates rules_go in anticipation for Go 1.20, which will ship without .a files for the standard library. Now, instead of conditionally building the standard library if the configuration differs from the default, the standard library will always be built. In addition, the builder will be built using 'go build' instead of individual invocations of go tool compile/link to avoid needing to bootstrap the standard library before producing the builder.

achew22 · 2022-12-07T05:05:37Z

In no way objecting to this change since it or something like it is necessary, but out of curiosity, what is the performance implication (or benefit) or switching from "{go} tool compile ..." to {go} build ..."?

fmeum · 2022-12-07T07:51:34Z

go/private/rules/binary.bzl

    return [DefaultInfo(
        files = depset([out]),
        executable = out,
    )]

 go_tool_binary = rule(
    implementation = _go_tool_binary_impl,
+    cfg = go_tool_transition,  # For fmeum: Is this correct? do we need go_reset_target?


non_go_tool_transition may perform better here (the bootstrap rules don't use nogo, so the setting shouldn't matter), but better verify that with cquery.

They seem to be the same at 3 configurations produced either way, but i'll stick with the non go tool transition.

With non_go_tool_transition the three configs seem to be

a plain config

an opt config with the java and c++ opt options

the opt config with an additional //go/private:request_nogo = true

The recent fix brought it down to just target and exec variants. I don't know where the target variant comes from when building all of rules_go and cquery doesn't show me the path. It doesn't seem to be built with a normal go_binary, so I'm inclined to say it's not an issue.

I also checked the stdlib rebuilds and they all look reasonable.

This allows the builder to be built in the same configuration as other targets from //tests/intagration/reproducibility:reproducibility_tests, so the bazel-bin directory doesn't get deleted as it would in a mulit-config build.

matloob · 2022-12-07T21:09:31Z

In no way objecting to this change since it or something like it is necessary, but out of curiosity, what is the performance implication (or benefit) or switching from "{go} tool compile ..." to {go} build ..."?

On Go 1.20+ the go build will build the necessary parts of the standard library needed to build the builder program which the go tool compile commands didn't need to do because the stdlib .a files existed in the distribution. On Go 1.19 and earlier there shouldn't be much of a performance difference over the go command analysis overhead, which is minimal, because the standard library will be found in the pkg/ directories.

The "annoying" but hard to avoid thing is that on 1.20+ we will build part of the standard library when go building the builder, then throwing away that standard library because the GOCACHE is temporary, then rebuilding the whole standard library in the //:stdlib rule. But transferring the cache over will be difficult to do.

fmeum

Thanks for figuring out the Windows build, the change looks good.

We shouldn't merge this yet though as it is only required with Go 1.20 and regresses build times with currently available versions of Bazel.

@matloob Could you add a temporary commit that uses Go 1.20 in CI?

matloob · 2022-12-08T16:55:00Z

Thanks for figuring out the Windows build, the change looks good.

We shouldn't merge this yet though as it is only required with Go 1.20 and regresses build times with currently available versions of Bazel.

When do you think we'll be able to get this in? I think it would be nice to make sure that people can easily test with the 1.20 release candidate (which was released yesterday),

@matloob Could you add a temporary commit that uses Go 1.20 in CI?

Could you give me a pointer to what to modify to do that?

fmeum · 2022-12-08T17:15:25Z

When do you think we'll be able to get this in? I think it would be nice to make sure that people can easily test with the 1.20 release candidate (which was released yesterday),

That's a good point. At the cost of making this change more complex, we could probably make the new logic conditional on the SDK version. We accept this version here and would need to wire this through all the way into the sdk properties available at analysis time. @linzhp @achew22 Do you think we should do this?

@matloob Could you add a temporary commit that uses Go 1.20 in CI?

Could you give me a pointer to what to modify to do that?

Are there binaries for the release candidate? If so, you could add go_download_sdk to the WORKSPACE file before the go_register_toolchain call.

linzhp · 2022-12-08T18:01:13Z

yeah, I think we should add that conditional logic on SDK versions. It may take a while for big repos to fix all issues and upgrade to Go 1.20.

fmeum · 2022-12-08T18:41:19Z

@matloob Do you want to take a stab at this conditional logic? We can also pair on it next week.

matloob · 2022-12-08T21:22:43Z

@matloob Do you want to take a stab at this conditional logic? We can also pair on it next week.

I'll give it a try

Are there binaries for the release candidate? If so, you could add go_download_sdk to the WORKSPACE file before the go_register_toolchain call.

Got it, I'll do that.

matloob · 2022-12-13T01:33:09Z

Added conditional logic to only build a new SDK for Go 1.20+.

Set the version to 1.20rc1 in the WORKSPACE file for CI purposes.

fmeum

~~Looks good to me with one comment, we should be able to merge this right away.~~

~~@linzhp Could you test at Uber once the comment has been resolved?~~

Missed the coverage test failure, will also look into it.

fmeum · 2022-12-13T06:56:54Z

go/private/actions/stdlib.bzl

+        minor = minor[:rc]
+    beta = minor.find("beta")
+    if beta != -1:
+        minor = minor[:beta]


Assuming there could be other strings (e.g. "alpha"), and we are already at three (".", "rc", "beta"), could we instead cut off the first non-numeric character? You could use isdigit.

~~Looks good to me with one comment, we should be able to merge this right away.~~

~~@linzhp Could you test at Uber once the comment has been resolved?~~

Missed the coverage test failure, will also look into it.

I think the coverage test is the same issue on the boringcrypto cl: that we need to make the path to "Tool" relative on 1.19+.

I ran it locally with extra output and it complained about -test.coverprofile not being supported when built without coverage. Since the CI is passing for the other PR, I rather suspect something changed with respect to coverage instrumentation in Go 1.20.

Ah, got it, that will require a closer look...

Ok so that error was due to the flag-flip setting the GOEXPERIMENT coverageredesign by default. We'll need to eventually update the way coverage works in rules_go, but for now we can set GOEXPERIMENT=nocoverageredesign. I'll try to find a way to set that to get this change moving forward. It might be easiest to plumb something through the same way we do in the boringcrypto cl

fmeum

I will do one more cquery pass tomorrow to make sure that we are generating the minimal possible number of configured targets.

go/private/actions/compilepkg.bzl

matloob · 2022-12-15T18:29:02Z

I updated the change to add the nocoverageredesign experiment earlier, when we write the SDK BUILD file setting the list of experiements on the GoSDK, so we don't need to plumb that information later on. And we can rely on the check of whether experiments are set when deciding to use the stdlib to decide whether to set that for 1.20+.

go/private/sdk.bzl

fmeum · 2022-12-16T07:26:33Z

WORKSPACE

-go_register_toolchains(
-    version = "1.19.4",
-)
+go_register_toolchains(version = "1.19.4")


This is back to 1.19.4. Given that 1.20 isn't out yet, maybe we should test it in go_bazel_test instead?

Added a simple test doing a build in 1.20.

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>

fmeum · 2022-12-17T06:37:49Z

Looks good to me, thanks for going through all the iterations!

Since I lean towards not forcing -c opt given that we haven't done so in the past and it may still be different from what Go used for release builds, I would can 6 this done.

@linzhp Could you take a final look and/or test it?

linzhp · 2022-12-17T17:03:59Z

OK. I will test it out and let you know

sywhang · 2022-12-17T21:46:56Z

Hey folks, I work at Uber with @linzhp and I'm in charge of upgrading Go versions at Uber.

I just tested this patch with our testing branch for Go 1.20 rc1, and we're still noticing issues with it. Specifically I'm seeing the stdlib archives are not being picked up by rules_go.

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
load packages: error building pkg "<some package name>": some/package/file.go:4:2: could not import bytes (opening archive file: open external/go_sdk/pkg/linux_amd64/bytes.a: no such file or directory)

load packages: error building pkg "<another package name>": another/package/file.go:4:2: could not import context (opening archive file: open external/go_sdk/pkg/linux_amd64/context.a: no such file or directory)

I'm noticing that rules_go builds the stdlib archive files:

[sungyoon@sungyoon]❯ ls -l bazel-bin/external/io_bazel_rules_go/stdlib_/pkg/linux_amd64                                                                                                                go-code/git/arcpatch-D8841493
total 30592
dr-xr-xr-x 2 sungyoon user    4096 Dec 17 21:40 archive
-r-xr-xr-x 1 sungyoon user  368040 Dec 17 21:40 bufio.a
-r-xr-xr-x 1 sungyoon user  419322 Dec 17 21:40 bytes.a
dr-xr-xr-x 2 sungyoon user    4096 Dec 17 21:40 compress
dr-xr-xr-x 2 sungyoon user    4096 Dec 17 21:40 container
-r-xr-xr-x 1 sungyoon user  230702 Dec 17 21:40 context.a
dr-xr-xr-x 4 sungyoon user    4096 Dec 17 21:40 crypto

but it seems like rules_go is unable to pick these up and populate them as dependencies.

fmeum · 2022-12-17T22:03:01Z

@sywhang Do you know why CI can't reproduce this when running with 1.20rc1? I am also unable to reproduce this locally. Which version of Bazel are you using?

matloob · 2022-12-19T17:52:19Z

@sywhang What do the lines in your WORKSPACE file that declare the toolchain/sdk look like?

matloob · 2022-12-19T19:25:24Z

@sywhang Could you also provide details about what action you're running that's failing? Those error messages don't look familiar to me and I'm wondering if something needs to be updated in your repo apart from rules_go to be compatible with Go 1.20. (Just a theory of course)

sywhang · 2022-12-19T22:03:31Z

I think this might have to do with the way we're extracting the binary since we're not using go_download_sdk rule. I also have found a workaround for this, so I'm ok with this PR merging as-is. If there are any issues we can just follow-up with a fix afterwards.

matloob · 2022-12-19T22:57:38Z

Ok, thanks @sywhang! @fmeum, Is this CL okay to merge?

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [io_bazel_rules_go](https://togithub.com/bazelbuild/rules_go) | http_archive | minor | `v0.37.0` -> `v0.38.0` | --- ### Release Notes <details> <summary>bazelbuild/rules_go</summary> ### [`v0.38.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.38.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.37.0...v0.38.0) #### Breaking changes Starting this release, rules_go requires Bazel 5.3 and golang/x/tools at least 0.3.0. #### Major new features This is the first release to support Go 1.20. When on this version of Go, it is highly recommended to use Bazel 6 and set `--experimental_output_directory_naming_scheme=diff_against_baseline` to prevent certain unnecessary rebuilds. #### What's Changed - preserve intermediate files if `-work` is specified by [@motiejus](https://togithub.com/motiejus) in [bazelbuild/rules_go#3389 - Adapt `lcov_coverage_test` to `exec.Command` change by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3392 - explicitly build relative path for Tool command in lcov_coverage_test by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3399 - Add support for building with boringcrypto by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3398 - Generalize `boringcrypto` to `GOEXPERIMENT` support by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3401 - Replace `cfg = "host"` with `cfg = "exec"` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3404 - always produce .a files at the beginning of a build by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3385 - Mark `nogo` targets as `manual` to keep top-level symlinks by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3410 - Refactoring the extld related logic by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3400 - Fix incompatible flags not applied to integration tests by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3415 - chore: remove experimental warning from bzlmod module by [@alexeagle](https://togithub.com/alexeagle) in [bazelbuild/rules_go#3418 - Add a clarifying error about dependency cycle found for internal tests by [@yushan26](https://togithub.com/yushan26) in [bazelbuild/rules_go#3422 - Do not use exec paths for rundir by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3414 - Upgrade org_golang_x_tools to 0.5.0 by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3356 - fix: Relative path of embedsrc by [@bakjos](https://togithub.com/bakjos) in [bazelbuild/rules_go#3407 - Add support for env_inherit by [@ttpathan](https://togithub.com/ttpathan) in [bazelbuild/rules_go#3256 - Update stardoc to 0.5.3 by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3427 - Add `env` attribute to `go_binary` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3428 #### New Contributors - [@matloob](https://togithub.com/matloob) made their first contribution in [bazelbuild/rules_go#3399 - [@yushan26](https://togithub.com/yushan26) made their first contribution in [bazelbuild/rules_go#3422 - [@bakjos](https://togithub.com/bakjos) made their first contribution in [bazelbuild/rules_go#3407 - [@ttpathan](https://togithub.com/ttpathan) made their first contribution in [bazelbuild/rules_go#3256 **Full Changelog**: bazelbuild/rules_go@v0.37.0...v0.38.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/cgrindel/bazel-starlib).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [io_bazel_rules_go](https://togithub.com/bazelbuild/rules_go) | http_archive | minor | `v0.35.0` -> `v0.38.1` | --- ### Release Notes <details> <summary>bazelbuild/rules_go</summary> ### [`v0.38.1`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.38.1) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.38.0...v0.38.1) #### Bug fixes The go_googleapis upgrade in 0.38.0 is too disruptive. It changes the Go import paths of the proto packages, causing build failures in repos trying to upgrade to rules_go 0.38.0. Reverting the go_googleapis to the master of 2022-12-05 ([#3432](https://togithub.com/bazelbuild/rules_go/issues/3432)) for now to make rules_go upgrades easier. #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "dd926a88a564a9246713a9c00b35315f54cbd46b31a26d5d8fb264c07045f05d", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.19.5") **Full Changelog**: bazelbuild/rules_go@v0.38.0...v0.38.1 ### [`v0.38.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.38.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.37.0...v0.38.0) #### Breaking changes Starting this release, rules_go requires Bazel 5.3 and golang/x/tools at least 0.3.0. If you are using [stardoc](https://togithub.com/bazelbuild/stardoc), you also need to upgrade it to 0.5.3 to recognize `RunEnvironmentInfo`, which is used in this release. #### Major new features This is the first release to support Go 1.20. When on this version of Go, it is highly recommended to use Bazel 6 and set `--experimental_output_directory_naming_scheme=diff_against_baseline` to prevent certain unnecessary rebuilds. #### What's Changed - preserve intermediate files if `-work` is specified by [@motiejus](https://togithub.com/motiejus) in [bazelbuild/rules_go#3389 - Adapt `lcov_coverage_test` to `exec.Command` change by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3392 - explicitly build relative path for Tool command in lcov_coverage_test by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3399 - Add support for building with boringcrypto by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3398 - Generalize `boringcrypto` to `GOEXPERIMENT` support by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3401 - Replace `cfg = "host"` with `cfg = "exec"` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3404 - always produce .a files at the beginning of a build by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3385 - Mark `nogo` targets as `manual` to keep top-level symlinks by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3410 - Refactoring the extld related logic by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3400 - Fix incompatible flags not applied to integration tests by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3415 - chore: remove experimental warning from bzlmod module by [@alexeagle](https://togithub.com/alexeagle) in [bazelbuild/rules_go#3418 - Add a clarifying error about dependency cycle found for internal tests by [@yushan26](https://togithub.com/yushan26) in [bazelbuild/rules_go#3422 - Do not use exec paths for rundir by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3414 - Upgrade org_golang_x_tools to 0.5.0 by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3356 - fix: Relative path of embedsrc by [@bakjos](https://togithub.com/bakjos) in [bazelbuild/rules_go#3407 - Add support for env_inherit by [@ttpathan](https://togithub.com/ttpathan) in [bazelbuild/rules_go#3256 - Update stardoc to 0.5.3 by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3427 - Add `env` attribute to `go_binary` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3428 #### New Contributors - [@matloob](https://togithub.com/matloob) made their first contribution in [bazelbuild/rules_go#3399 - [@yushan26](https://togithub.com/yushan26) made their first contribution in [bazelbuild/rules_go#3422 - [@bakjos](https://togithub.com/bakjos) made their first contribution in [bazelbuild/rules_go#3407 - [@ttpathan](https://togithub.com/ttpathan) made their first contribution in [bazelbuild/rules_go#3256 **Full Changelog**: bazelbuild/rules_go@v0.37.0...v0.38.0 ### [`v0.37.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.37.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.36.0...v0.37.0) #### Major New Features - Support fetching packages for generated code in the Go Packages Driver #### What's Changed - bzlmod: Add missing `strip_prefix` field to `source.template.json` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3359 - Declare toolchains in a separate repository by [@jfirebaugh](https://togithub.com/jfirebaugh) in [bazelbuild/rules_go#3348 - Delete legacy actions API by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3173 - go_path: support go:embed of generated files by [@S-Chan](https://togithub.com/S-Chan) in [bazelbuild/rules_go#3285 - Properly deprecate `bindata`, `go_embed_data`, and `go_embed_data_deps` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3362 - link.bzl: ignore duplicate dep on coverdata by [@robfig](https://togithub.com/robfig) in [bazelbuild/rules_go#3032 - feat(pkg-drv): add support for generated files by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3354 - Remove unused variables in link action by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3367 - Reduce number of declared files in `emit_stdlib` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3366 - Update docs regarding vendored proto files by [@garymm](https://togithub.com/garymm) in [bazelbuild/rules_go#3360 - go link: use external linker when in race mode by [@motiejus](https://togithub.com/motiejus) in [bazelbuild/rules_go#3370 - Adding first example by [@chrislovecnm](https://togithub.com/chrislovecnm) in [bazelbuild/rules_go#3317 - fix(packagesdriver): bazelFlags should prefix the command by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3371 - chore(gpd): export aspect utils for reusability by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3373 - nogo: Add a \_base key to be a default config for all Analyzers. by [@DolceTriade](https://togithub.com/DolceTriade) in [bazelbuild/rules_go#3351 - Document that `Rlocation` can return relative paths by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3377 - Fix normalization check for `Rlocation` path by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3378 - fix(gpd): Write large target patterns to file by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3372 - Make Go runfiles library repo mapping aware by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3347 #### New Contributors - [@jfirebaugh](https://togithub.com/jfirebaugh) made their first contribution in [bazelbuild/rules_go#3348 - [@S-Chan](https://togithub.com/S-Chan) made their first contribution in [bazelbuild/rules_go#3285 - [@garymm](https://togithub.com/garymm) made their first contribution in [bazelbuild/rules_go#3360 - [@motiejus](https://togithub.com/motiejus) made their first contribution in [bazelbuild/rules_go#3370 - [@chrislovecnm](https://togithub.com/chrislovecnm) made their first contribution in [bazelbuild/rules_go#3317 - [@DolceTriade](https://togithub.com/DolceTriade) made their first contribution in [bazelbuild/rules_go#3351 **Full Changelog**: bazelbuild/rules_go@v0.36.0...v0.37.0 #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "56d8c5a5c91e1af73eca71a6fab2ced959b67c86d12ba37feedb0a2dfea441a6", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.37.0/rules_go-v0.37.0.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.37.0/rules_go-v0.37.0.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.19.3") ### [`v0.36.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.36.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.35.0...v0.36.0) #### Major new features - The Go runfiles library developed by [@phst](https://togithub.com/phst) at https://github.com/phst/runfiles is now available as `github.com/bazelbuild/rules_go/go/runfiles` (target `@io_bazel_rules_go//go/runfiles`) and is now the official way to look up runfiles from Go. It will be made compatible with Bzlmod in a future release. **Note:** For improved consistency with runfiles libraries in other languages, the `Path` function present in https://github.com/phst/runfiles is called `Rlocation` in rules_go. - Adding two new command line flags `--@io_bazel_rules_go//go/config:gc_goopts` and `--@io_bazel_rules_go//go/config:gc_linkopts` to accept additional compiler and linker flags. #### What's Changed - bzlmod: Use register_toolchains by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3290 - Remove unused env_execute by [@kmicklas](https://togithub.com/kmicklas) in [bazelbuild/rules_go#3276 - Correctly forward InstrumentedFilesInfo from dependencies by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3294 - Parse Go SDK version out of `go version` instead of `VERSION` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3296 - also write test stderr to json (xml) output file by [@rickystewart](https://togithub.com/rickystewart) in [bazelbuild/rules_go#3300 - Bzlmod: Pin Bazel version in CI by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3301 - Allow to select Go SDK version from declared SDKs by [@ash2k](https://togithub.com/ash2k) in [bazelbuild/rules_go#3303 - Use top level VERSION file when it exists by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3309 - synchronize when writing to the JSON converter by [@rickystewart](https://togithub.com/rickystewart) in [bazelbuild/rules_go#3311 - update version manifest URL by [@benjaminp](https://togithub.com/benjaminp) in [bazelbuild/rules_go#3314 - Symlink "misc" into repository for local_sdk. by [@adam-azarchs](https://togithub.com/adam-azarchs) in [bazelbuild/rules_go#3313 - Ensure compatibility with incompatible flags close to flipping by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3319 - Add additional bzlmod requirements to allow grpc protobufs to work by [@shs96c](https://togithub.com/shs96c) in [bazelbuild/rules_go#3320 - bzlmod: Update pinned Bazel version by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3321 - bzlmod: Set up automated BCR releases by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3324 - bzlmod: Add support for gogo proto compiler by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3323 - gopackagesdriver: move non-Go files to OtherFiles (Fixes [#3326](https://togithub.com/bazelbuild/rules_go/issues/3326)) by [@michaelarusso](https://togithub.com/michaelarusso) in [bazelbuild/rules_go#3327 - bzlmod: Allow yanked versions in CI by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3329 - feat(pkg-drv): allow setting custom aspect and kinds by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3328 - fix(packages-driver): allow defining additional aspects instead by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3330 - Add support for parsing Go development versions by [@zakcutner](https://togithub.com/zakcutner) in [bazelbuild/rules_go#3333 - Match GOARCH ppc64 to @platforms//cpu:ppc as well. by [@mahiuchun](https://togithub.com/mahiuchun) in [bazelbuild/rules_go#3336 - bzlmod: Update Publish to BCR app config by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3337 - gopackagesdriver: default to NotHandled:true for package queries by [@ian-h-chamberlain](https://togithub.com/ian-h-chamberlain) in [bazelbuild/rules_go#3338 - fix(packagesdrv): resolve third party go packages by [@JamyDev](https://togithub.com/JamyDev) in [bazelbuild/rules_go#3332 - runfiles: port phst/runfiles to rules_go by [@sluongng](https://togithub.com/sluongng) in [bazelbuild/rules_go#3205 - Adding gc_linopts to command line flag by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3342 - Move new runfiles library to `//go/runfiles` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3341 - Rename `Path` to `Rlocation` in runfiles library by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3346 - Add 'gc_goopts' flag to command line to allow gc_goopts to propagate by [@LWarrens](https://togithub.com/LWarrens) in [bazelbuild/rules_go#3340 #### New Contributors - [@kmicklas](https://togithub.com/kmicklas) made their first contribution in [bazelbuild/rules_go#3276 - [@adam-azarchs](https://togithub.com/adam-azarchs) made their first contribution in [bazelbuild/rules_go#3313 - [@shs96c](https://togithub.com/shs96c) made their first contribution in [bazelbuild/rules_go#3320 - [@michaelarusso](https://togithub.com/michaelarusso) made their first contribution in [bazelbuild/rules_go#3327 - [@zakcutner](https://togithub.com/zakcutner) made their first contribution in [bazelbuild/rules_go#3333 - [@mahiuchun](https://togithub.com/mahiuchun) made their first contribution in [bazelbuild/rules_go#3336 - [@LWarrens](https://togithub.com/LWarrens) made their first contribution in [bazelbuild/rules_go#3340 **Full Changelog**: bazelbuild/rules_go@v0.35.0...v0.36.0 #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "ae013bf35bd23234d1dea46b079f1e05ba74ac0321423830119d3e787ec73483", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.36.0/rules_go-v0.36.0.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.36.0/rules_go-v0.36.0.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.19.3") </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/kreempuff/rules_unreal_engine).

* always produce .a files at the beginning of a build This change updates rules_go in anticipation for Go 1.20, which will ship without .a files for the standard library. Now, instead of conditionally building the standard library if the configuration differs from the default, the standard library will always be built. In addition, the builder will be built using 'go build' instead of individual invocations of go tool compile/link to avoid needing to bootstrap the standard library before producing the builder. * fix a bad cleanup in stdlib.bzl * redefine transition in terms of what's kept * use go_reset_target for the builder This allows the builder to be built in the same configuration as other targets from //tests/intagration/reproducibility:reproducibility_tests, so the bazel-bin directory doesn't get deleted as it would in a mulit-config build. * remove previosly deleted code that came back in a merge` * fix buildifier issue * try to use tmpdir on WINDOWS * try another way of getting gocache dirs for windows * add output * debug * debug2 * debug 3 * debug 4 * debug 5 * debug 6 * only compile stdlib .a files for Go 1.20+ * set version to 1.20rc1 for CI testing purposes * fix libs attribute of Go sdk to allow zero libs files * fix coverage test * get only digits of the part before the next dot * allow empty * set GOEXPERIMENT=nocoverageredesign * conditionally turn off goexperiment based on versioun * rename variable * fix flag * add lcov fix * address comments, and remove one config of builder * decide to compile .a files based on whether they exist * fix add call * add nocoverageredesign to experiments earlier * another small fix * fix double * Update go/private/sdk.bzl Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im> * add a simple test doing a build on 1.20rc1 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/bazelbuild/buildtools](https://togithub.com/bazelbuild/buildtools) | require | digest | `180a94a` -> `bf454e7` | | [github.com/bazelbuild/rules_go](https://togithub.com/bazelbuild/rules_go) | require | minor | `v0.37.0` -> `v0.38.0` | | [github.com/onsi/gomega](https://togithub.com/onsi/gomega) | require | minor | `v1.25.0` -> `v1.26.0` | | [golang.org/x/exp](https://togithub.com/golang/exp) | require | digest | `a68e582` -> `b3c2aaf` | | [google.golang.org/genproto](https://togithub.com/googleapis/go-genproto) | require | digest | `9d59e20` -> `dcaf20b` | | [google.golang.org/grpc](https://togithub.com/grpc/grpc-go) | require | patch | `v1.52.0` -> `v1.52.3` | --- ### ⚠ Dependency Lookup Warnings ⚠ Warnings were logged while processing this repo. Please check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>bazelbuild/rules_go</summary> ### [`v0.38.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.38.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.37.0...v0.38.0) #### Breaking changes Starting this release, rules_go requires Bazel 5.3 and golang/x/tools at least 0.3.0. If you are using [stardoc](https://togithub.com/bazelbuild/stardoc), you also need to upgrade it to 0.5.3 to recognize `RunEnvironmentInfo`, which is used in this release. #### Major new features This is the first release to support Go 1.20. When on this version of Go, it is highly recommended to use Bazel 6 and set `--experimental_output_directory_naming_scheme=diff_against_baseline` to prevent certain unnecessary rebuilds. #### What's Changed - preserve intermediate files if `-work` is specified by [@motiejus](https://togithub.com/motiejus) in [bazelbuild/rules_go#3389 - Adapt `lcov_coverage_test` to `exec.Command` change by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3392 - explicitly build relative path for Tool command in lcov_coverage_test by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3399 - Add support for building with boringcrypto by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3398 - Generalize `boringcrypto` to `GOEXPERIMENT` support by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3401 - Replace `cfg = "host"` with `cfg = "exec"` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3404 - always produce .a files at the beginning of a build by [@matloob](https://togithub.com/matloob) in [bazelbuild/rules_go#3385 - Mark `nogo` targets as `manual` to keep top-level symlinks by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3410 - Refactoring the extld related logic by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3400 - Fix incompatible flags not applied to integration tests by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3415 - chore: remove experimental warning from bzlmod module by [@alexeagle](https://togithub.com/alexeagle) in [bazelbuild/rules_go#3418 - Add a clarifying error about dependency cycle found for internal tests by [@yushan26](https://togithub.com/yushan26) in [bazelbuild/rules_go#3422 - Do not use exec paths for rundir by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3414 - Upgrade org_golang_x_tools to 0.5.0 by [@linzhp](https://togithub.com/linzhp) in [bazelbuild/rules_go#3356 - fix: Relative path of embedsrc by [@bakjos](https://togithub.com/bakjos) in [bazelbuild/rules_go#3407 - Add support for env_inherit by [@ttpathan](https://togithub.com/ttpathan) in [bazelbuild/rules_go#3256 - Update stardoc to 0.5.3 by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3427 - Add `env` attribute to `go_binary` by [@fmeum](https://togithub.com/fmeum) in [bazelbuild/rules_go#3428 #### New Contributors - [@matloob](https://togithub.com/matloob) made their first contribution in [bazelbuild/rules_go#3399 - [@yushan26](https://togithub.com/yushan26) made their first contribution in [bazelbuild/rules_go#3422 - [@bakjos](https://togithub.com/bakjos) made their first contribution in [bazelbuild/rules_go#3407 - [@ttpathan](https://togithub.com/ttpathan) made their first contribution in [bazelbuild/rules_go#3256 **Full Changelog**: bazelbuild/rules_go@v0.37.0...v0.38.0 </details> <details> <summary>onsi/gomega</summary> ### [`v1.26.0`](https://togithub.com/onsi/gomega/releases/tag/v1.26.0) [Compare Source](https://togithub.com/onsi/gomega/compare/v1.25.0...v1.26.0) #### 1.26.0 ##### Features - When a polled function returns an error, keep track of the actual and report on the matcher state of the last non-errored actual \[[`21f3090`](https://togithub.com/onsi/gomega/commit/21f3090)] - improve eventually failure message output \[[`c530fb3`](https://togithub.com/onsi/gomega/commit/c530fb3)] ##### Fixes - fix several documentation spelling issues \[[`e2eff1f`](https://togithub.com/onsi/gomega/commit/e2eff1f)] </details> <details> <summary>grpc/grpc-go</summary> ### [`v1.52.3`](https://togithub.com/grpc/grpc-go/releases/tag/v1.52.3): Release 1.52.3 [Compare Source](https://togithub.com/grpc/grpc-go/compare/v1.52.1...v1.52.3) ### Bug Fixes - Fix user-agent version ### [`v1.52.1`](https://togithub.com/grpc/grpc-go/releases/tag/v1.52.1): Release 1.52.1 [Compare Source](https://togithub.com/grpc/grpc-go/compare/v1.52.0...v1.52.1) ### Bug Fixes - grpclb: rename grpclbstate package back to state ([#5963](https://togithub.com/grpc/grpc-go/issues/5963)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/aspect-build/silo).  Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

sluongng · 2023-11-01T20:33:19Z

It seems like golang/go#55953 is aiming to remove nocoverageredesign in 1.22.

Anybody have context on why did we introduce that experiment here and what would it take to adapt to the new design?

cc: @fmeum @matloob

fmeum · 2023-11-03T09:06:20Z

We introduced it because Go changed the way it handles coverage by default. Supporting the new mode is tracked in #3513 and it looks like we might need to work on this sooner rather than later. It is not an easy task though.

matloob added 3 commits December 6, 2022 17:11

fix a bad cleanup in stdlib.bzl

784c1c0

redefine transition in terms of what's kept

dc93dfb

fmeum reviewed Dec 7, 2022

View reviewed changes

matloob added 3 commits December 7, 2022 15:33

Merge branch 'bazelbuild:master' into go

36c5314

use go_reset_target for the builder

04c752c

This allows the builder to be built in the same configuration as other targets from //tests/intagration/reproducibility:reproducibility_tests, so the bazel-bin directory doesn't get deleted as it would in a mulit-config build.

remove previosly deleted code that came back in a merge`

b81c415

matloob added 10 commits December 7, 2022 16:17

fix buildifier issue

8421750

try to use tmpdir on WINDOWS

c45ec9c

try another way of getting gocache dirs for windows

106e275

add output

c3b34ad

debug

7a79e59

debug2

de594a0

debug 3

78a8e2e

debug 4

2b05009

debug 5

0d37f96

debug 6

f0e29ba

fmeum approved these changes Dec 8, 2022

View reviewed changes

matloob added 2 commits December 12, 2022 20:29

only compile stdlib .a files for Go 1.20+

3a35f5b

set version to 1.20rc1 for CI testing purposes

f757ef4

fix libs attribute of Go sdk to allow zero libs files

24e601f

fmeum approved these changes Dec 13, 2022

View reviewed changes

fmeum reviewed Dec 14, 2022

View reviewed changes

go/private/actions/compilepkg.bzl Outdated Show resolved Hide resolved

matloob added 6 commits December 14, 2022 16:47

Merge branch 'master' of https://github.com/bazelbuild/rules_go into go

b3dfe4b

Merge branch 'master' of https://github.com/bazelbuild/rules_go into go

de75424

fix add call

80235ea

add nocoverageredesign to experiments earlier

929b55c

another small fix

87f47ef

fix double

c12a10b

fmeum reviewed Dec 16, 2022

View reviewed changes

matloob and others added 2 commits December 16, 2022 14:03

Update go/private/sdk.bzl

108915e

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>

add a simple test doing a build on 1.20rc1

bc8eb25

fmeum approved these changes Dec 17, 2022

View reviewed changes

fmeum merged commit 8d309d5 into bazelbuild:master Dec 20, 2022

folliehiyuki mentioned this pull request Jun 1, 2023

Bootstrap fails on AlpineLinux edge thought-machine/please#2783

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

always produce .a files at the beginning of a build #3385

always produce .a files at the beginning of a build #3385

matloob commented Dec 6, 2022

achew22 commented Dec 7, 2022

fmeum Dec 7, 2022

matloob Dec 7, 2022

fmeum Dec 15, 2022

matloob commented Dec 7, 2022

fmeum left a comment

matloob commented Dec 8, 2022 •

edited

fmeum commented Dec 8, 2022

linzhp commented Dec 8, 2022

fmeum commented Dec 8, 2022

matloob commented Dec 8, 2022

matloob commented Dec 13, 2022

fmeum left a comment •

edited

fmeum Dec 13, 2022

matloob Dec 13, 2022

fmeum Dec 13, 2022

matloob Dec 13, 2022

matloob Dec 13, 2022

fmeum left a comment

matloob commented Dec 15, 2022

fmeum Dec 16, 2022

matloob Dec 16, 2022

fmeum commented Dec 17, 2022

linzhp commented Dec 17, 2022

sywhang commented Dec 17, 2022

fmeum commented Dec 17, 2022

matloob commented Dec 19, 2022 •

edited

matloob commented Dec 19, 2022

sywhang commented Dec 19, 2022

matloob commented Dec 19, 2022

sluongng commented Nov 1, 2023

fmeum commented Nov 3, 2023

always produce .a files at the beginning of a build #3385

always produce .a files at the beginning of a build #3385

Conversation

matloob commented Dec 6, 2022

achew22 commented Dec 7, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matloob commented Dec 7, 2022

fmeum left a comment

Choose a reason for hiding this comment

matloob commented Dec 8, 2022 • edited

fmeum commented Dec 8, 2022

linzhp commented Dec 8, 2022

fmeum commented Dec 8, 2022

matloob commented Dec 8, 2022

matloob commented Dec 13, 2022

fmeum left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fmeum left a comment

Choose a reason for hiding this comment

matloob commented Dec 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fmeum commented Dec 17, 2022

linzhp commented Dec 17, 2022

sywhang commented Dec 17, 2022

fmeum commented Dec 17, 2022

matloob commented Dec 19, 2022 • edited

matloob commented Dec 19, 2022

sywhang commented Dec 19, 2022

matloob commented Dec 19, 2022

sluongng commented Nov 1, 2023

fmeum commented Nov 3, 2023

matloob commented Dec 8, 2022 •

edited

fmeum left a comment •

edited

matloob commented Dec 19, 2022 •

edited