Document mixed/heterogeneous builds across different operating systems

[lukts30]

Is it possible to have a BUILD step automatically run on a remote host if the local OS is not target_compatible without having to use --fake-host?

E.g I want to make something like this to work:

genrule(
    name = "hello_windows",
    out = "hello_windows.txt",
    cmd = "echo Hello from Windows >> $OUT",
    target_compatible_with = ["config//os:windows"],
)

genrule(
    name = "hello_linux",
    out = "hello_linux.txt",
    cmd = "sha1sum $(location :hello_windows) >> $OUT",
    target_compatible_with = ["config//os:linux"],
)

This is similar to the following bazel issues:

• Remote execution from windows host to linux executor bazelbuild/bazel#19587
• Running a test on a Windows executor from a Linux host is not possible bazelbuild/bazel#19209
• Please allow remote execution to have its own Platform, separate from host and target. bazelbuild/bazel#5309

[lukts30]

buck2 build root//:from_linux --out=- --prefer-local does now work with the following example but I hoped that buck2 would fallback to local execution if the remote_execution_properties could not be satisfied.
For now I am required to use --prefer-local.

buck2 build root//:from_linux --out=- --prefer-local
File changed: root//BUCK
Build ID: f7875f28-00da-483e-82e9-1d5e1e5bc5b8
Network: (GRPC-SESSION-ID)
Jobs completed: 14. Time elapsed: 0.2s.
Cache hits: 0%. Commands: 2 (cached: 0, remote: 1, local: 1)
BUILD SUCCEEDED
Hello from Windows!!!  

Microsoft Windows [Version 10.0.22000.2538]
Linux 6.6.1-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 08 Nov 2023 16:05:38 +0000

genrule(
    name = "hello_from_windows",
    out = "hello_from_windows.txt",
    cmd = "echo Hello from Windows!!! >> $OUT & ver >> $OUT",
    exec_compatible_with = ["config//os:windows"],
)

genrule(
    name = "from_linux",
    out = "hello_linux.txt",
    cmd = "cat $(location :hello_from_windows) >> $OUT; uname -svr >> $OUT;",
    exec_compatible_with = ["config//os:linux"],
)

def _platforms(ctx: AnalysisContext) -> list[Provider]:
    name = ctx.label.raw_target()

    constraints_windows = dict()
    constraints_windows.update(ctx.attrs.os_configuration_windows[ConfigurationInfo].constraints)



    platform_windows = ExecutionPlatformInfo(
        label = name,
        configuration = ConfigurationInfo(
            constraints = constraints_windows,
            values = {},
        ),
        executor_config = CommandExecutorConfig(
            local_enabled = host_info().os.is_windows,
            remote_enabled = True,
            use_limited_hybrid = True,
            remote_execution_properties = {
                "OSFamily": "Windows",
            },
            remote_execution_use_case = "buck2-default",
            use_windows_path_separators = True,
        ),
    )


    constraints2 = dict()
    constraints2.update(ctx.attrs.os_configuration_linux[ConfigurationInfo].constraints)
    platform1 = ExecutionPlatformInfo(
        label = name,
        configuration = ConfigurationInfo(
            constraints = constraints2,
            values = {},
        ),
        executor_config = CommandExecutorConfig(
            local_enabled = host_info().os.is_linux,
            remote_enabled = True,
            use_limited_hybrid = True,
            remote_execution_properties = {
                "OSFamily": "Linux",
            },
            remote_execution_use_case = "buck2-default",
            use_windows_path_separators = False,
        ),
    )

    return [
        DefaultInfo(),
        ExecutionPlatformRegistrationInfo(platforms = [platform_windows, platform1]),
    ]


platforms = rule(
    impl = _platforms,
    attrs = {
        "use_windows_path_separators": attrs.bool(default = False),
        "os_configuration_windows": attrs.dep(
            providers = [ConfigurationInfo],
            default = "config//os:windows",
        ),
        "os_configuration_linux": attrs.dep(
            providers = [ConfigurationInfo],
            default = "config//os:linux",
        ),
    },
)

[lukts30]

So this does work and it is a feature that bazel does not have today.

I am keeping this issue open for now because this feature deserves a bit more documentation / examples:

• exec_compatible_with
• ExecutionPlatformRegistrationInfo with more that one platform

[ndmitchell]

CC @lmvasquezg who is looking at documentation