Automatically register SDKs for common execution platforms

[fmeum]

What type of PR is this?

Feature

What does this PR do? Why is it needed?

The go_sdk module extension now registers unnamed SDKs for all major execution platforms in addition to the host platform. This ensures that Go builds should "just work" if the exec platform is on this list, even if it differs from the host platform.

Which issues(s) does this PR fix?

Fixes #3007
Fixes #3402

Other notes for review

[fmeum]

@sluongng I would be very interested in your feedback on this, in particular the selection of common execution platforms.

[sluongng]

Since our repo is not net on bzlmod, we need to make similar modifications to sdk.bzl for it to work.

There was an error on our end on the initial attempt:

Error in fail: gazelle found more than one suitable Go SDK (go_sdk, go_sdk_darwin_amd64). Specify which one to use with gazelle_dependencies(go_sdk = "go_sdk").

Work around is to add the if clause to skip on duplicated declaration (or to change gazelle's declaration).

diff --git a/go/private/sdk.bzl b/go/private/sdk.bzl
index 08c6e74c..7cea13bc 100644
--- a/go/private/sdk.bzl
+++ b/go/private/sdk.bzl
@@ -303,6 +303,18 @@ def _go_toolchains(name, sdk_repo, sdk_type, sdk_version = None, goos = None, go
         sdk_versions = [sdk_version or ""],
     )

+# A list of (goos, goarch) pairs that are commonly used for remote executors in cross-platform
+# builds (where host != exec platform). By default, we register toolchains for all of these
+# platforms in addition to the host platform.
+_COMMON_EXEC_PLATFORMS = [
+    ("darwin", "amd64"),
+    ("darwin", "arm64"),
+    ("linux", "amd64"),
+    ("linux", "arm64"),
+    ("windows", "amd64"),
+    ("windows", "arm64"),
+]
+
 def go_download_sdk(name, register_toolchains = True, **kwargs):
     go_download_sdk_rule(name = name, **kwargs)
     _go_toolchains(
@@ -316,6 +328,28 @@ def go_download_sdk(name, register_toolchains = True, **kwargs):
     if register_toolchains:
         _register_toolchains(name)

+    for goos, goarch in _COMMON_EXEC_PLATFORMS:
+        if goos == kwargs.get("goos") and goarch == kwargs.get("goarch"):
+            continue
+
+        n = "{}_{}_{}".format(name, goos, goarch)
+        go_download_sdk_rule(
+            name = n,
+            goos = goos,
+            goarch = goarch,
+            **kwargs
+        )
+        _go_toolchains(
+            name = n + "_toolchains",
+            sdk_repo = n,
+            sdk_type = "remote",
+            sdk_version = kwargs.get("version"),
+            goos = goos,
+            goarch = goarch,
+        )
+        if register_toolchains:
+            _register_toolchains(n)
+
 def _go_local_sdk_impl(ctx):
     goroot = ctx.attr.path
     platform = _detect_sdk_platform(ctx, goroot)

Could you please include the sdk.bzl change in this PR as well? (please feel free to modify my monkey patch above)

[darkrift]

@fmeum I was just hit by this problem this week, any way we can get this move forward ?

[fmeum]

Given the limited time all of us get to spend on maintaining this ruleset, I would like to restrict this to Bzlmod. It's less likely to break anything there with better way for us to contain any problems that may arise (WORKSPACE macros offer limited options here).

[sluongng]

Agree with the reasoning above.

[tyler-french]

Tested this, looks good!