Confusion on the use of 'extra_target_triples' in rust_repository_set

[photex]

Howdy!

I'm not sure if this is related to #101 or not but I had expected this to work:

rust_repository_set(
    name = "rust_linux_x86_64",
    exec_triple = "x86_64-unknown-linux-gnu",
    extra_target_triples = [
        "aarch64-unknown-linux-gnu"
    ],
    version = "1.27.2",
)

The error I get:

ERROR: /home/kopernikus/.cache/bazel/_bazel_kopernikus/3b8e07831dccba8888e630aef5b461f7/external/rust_linux_x86_64_toolchains/BUILD:11:1: no such target '@bazel_tools//platforms:aarch64': target 'aarch64' not declared in package 'platforms' defined by /home/kopernikus/.cache/bazel/_bazel_kopernikus/3b8e07831dccba8888e630aef5b461f7/external/bazel_tools/platforms/BUILD and referenced by '@rust_linux_x86_64_toolchains//:toolchain_for_aarch64-unknown-linux-gnu'
ERROR: While resolving toolchains for target //components/services/kpns-bus/rust:kpns-bus: invalid registered toolchain '@rust_linux_x86_64_toolchains//:toolchain_for_aarch64-unknown-linux-gnu': Analysis failed
ERROR: Analysis of target '//components/services/kpns-bus/rust:kpns-bus' failed; build aborted: invalid registered toolchain '@rust_linux_x86_64_toolchains//:toolchain_for_aarch64-unknown-linux-gnu': Analysis failed
INFO: Elapsed time: 0.751s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (3 packages loaded)

What "extra_target_triples" seemed to be was the ability to have this pull down appropriate versions of the stdlib for alternate targets in the style of rustup target add aarch64-unknown-linux-gnu.

[acmcarther]

Two parts:

First, regd:

target 'aarch64' not declared in package 'platforms'

This is a normal bug that I need to fix. Will investigate soon

Second, more generally around the cross compilation story:

This is a very new feature that isn't completely implemented and I apologize for the confusion. Currently the extra_target_triples field downloads the requisite stdlibs for those alternative targets, but it doesn't configure the additional host-to-target toolchains, or expose a way for you to provide the cross-linker that you'll need.

I'll keep this issue updated with the implementation progress.

[photex]

No need to apologize. Thanks for explaining. Please let me know if I can be of any assistance.

[photex]

Howdy again!

So after having added my aarch64 toolchain I no longer get this error. I still have to mess around with this now and attempt to get cross-compilation working.

[Noia]

Any update on this? I am trying to compile some rust for a Raspberry Pi and I'm running into similar issues:

rust_repository_set(
    name = "rust_linux_arm",
    exec_triple = "arm-unknown-linux-gnueabihf",
    extra_target_triples = [
    ],
    version = RUST_VERSION,
)

$ blaze query '"@rust_linux_arm//..."'
INFO: Invocation ID: d7ed739f-ba95-4f9c-b258-c457105cd215
@rust_linux_arm//:toolchain_for_arm-unknown-linux-gnueabihf_impl
@rust_linux_arm//:rustdoc
@rust_linux_arm//:rustc_lib
@rust_linux_arm//:rustc
@rust_linux_arm//:rust_lib-arm-unknown-linux-gnueabihf

None of these targets appear to export a PlatformInfo, when I try to use @rust_linux_arm//:toolchain_for_arm-unknown-linux-gnueabihf_impl I get a different error however.

$ blaze build --platforms=@rust_linux_arm//:toolchain_for_arm-unknown-linux-gnueabihf_impl //04:hello --sandbox_debug 
INFO: Invocation ID: 734f9c77-2714-46ca-b4a2-eba88b61f35e
ERROR: /home/emis/.cache/bazel/_bazel_emis/e492159d29688c624934eedb7f1a4c0b/external/rust_linux_arm/BUILD:36:1: in rust_toolchain rule @rust_linux_arm//:toolchain_for_arm-unknown-linux-gnueabihf_impl: cycle in dependency graph:
    //04:hello
.-> @rust_linux_arm//:toolchain_for_arm-unknown-linux-gnueabihf_impl [self-edge]
`--
This cycle occurred because of a configuration option
ERROR: Analysis of target '//04:hello' failed; build aborted
INFO: Elapsed time: 0.039s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

Am I missing something? How am I supposed to invoke this? I get basically the same error when using --extra_toolchains. Could you post an example of how you expect this to be done?

[acmcarther]

I don't have an update on this feature. The cross compilation functionality itself is still soft-blocked on adding a toolchain configuration option for specifying a per target crosslinker[*], though if you're willing to wait long enough that problem will eliminate itself when lld becomes the default linker (which ostensibly can crosslink to any supported target).

Crosslink-aside, I don't understand the circular dependency you're seeing there. I can't promise that it'll be enlightening, but can you add the result of:

$ blaze query --output=build //04:hello

---

[*] This isn't actually "hard" to do, its just something I was supposed to do and haven't done. Don't worry, this isn't the only feature I've been neglecting.

[Noia]

That's just the hello world I'm compiling, noting exciting in there:

$ blaze query --output=build //04:hello
INFO: Invocation ID: 138cd105-fff1-4ce5-bbbc-63c1e50fcf50
# /home/emis/Documents/git/stuff/04/BUILD:20:1
rust_binary(
  name = "hello",
  srcs = ["//04:hello.rs"],
)

[Noia]

Out of curiosity, what is the command for crosscompiling to arm meant to look like. Also out of curiosity, why is armv7 not supported when it's apparently supported by rustc? Just wondering if I'm missing something

[mfarrugi]

The couple of blog posts on the topic make it look like cargo passes

rules_rust/rust/private/rustc.bzl

Line 224 in 23ab9d5

args.add("--codegen=linker=" + ld)

to the cross-linker.. other than that I would prod cargo to see what command lines it produces.

@acmcarther can you qualify "not hard to do" with an outline of what needs to be done? Is lld usable, or should we do something else?

[acmcarther]

Sorry for the lack of responsiveness. I'm actually working to implement this now, but I didn't want to say anything and then not actually do it. I've basically garbage collected my knowledge of build configuration attributes, so I'm reacquiring the knowledge.

After some brief exploration with my last working state, I can tell you that the semantics of config_setting have changed in some way to induce this circular reference purely based on the fact that I'm no longer getting as far into the build process (i.e. to the part where I attempt to link with the wrong linker), though I'm not clear on the specific issue as I'm still loading the knowledge of platform, select, constraint_setting, constraint_value, and config_setting back into my brain. That is to say that I have no idea what was or is going on with the platform decls and I need to actually really sit down for a while and remember.

That aside, I can give you a general update on what needs to happen w.r.t. the linker configuration and lld, though the former knowledge is stale after #133 went in:

---

On providing a crosslinker
Our primary toolchain fetching repository is like this:

    rust_repository_set(
        name = "rust_linux_x86_64"
        exec_triple = "x86_64-unknown-linux-gnu",
        extra_target_triples = [
            "arm-unknown_linux-gnueabi",
        ],
        version = RUST_VERSION,
    )

What this actually means is:

"Get me a compiler that runs on x86_64-unknown-linux-gnu", has the precompiled stdlib for x86_64-unknown-linux-gnu", but also has the precompiled stdlib for arm-unknown_linux-gnueabi. Additionally, declare a toolchain and a rust_toolchain from x86_64-unknown-linux-gnu to both x86_64-unknown-linux-gnu and arm-unknown_linux-gnueabi"

Here's what they actually look like.

---

Now take that knowledge and store it somewhere, and lets talk about how the linker is selected right now.

Currently, rust_toolchain still requires the _crosstool argument. This is plumbed all the way to the link section of rustc_compile_action and accessed via find_cpp_toolchain:

    # Link!
    rpaths = _compute_rpaths(toolchain, output_dir, dep_info)
    ld, link_args = _get_linker_and_args(ctx, rpaths)
    args.add("--codegen=linker=" + ld)
    args.add_joined("--codegen", link_args, join_with = " ", format_joined = "link-args=%s")

(might be helpful to see _get_linker_and_args)

def _get_linker_and_args(ctx, rpaths):
    if (len(BAZEL_VERSION) == 0 or
        versions.is_at_least("0.18.0", BAZEL_VERSION)):
        user_link_flags = ctx.fragments.cpp.linkopts
    else:
        user_link_flags = depset(ctx.fragments.cpp.linkopts)

    cc_toolchain = find_cpp_toolchain(ctx)
    feature_configuration = cc_common.configure_features(
        cc_toolchain = cc_toolchain,
        requested_features = ctx.features,
        unsupported_features = ctx.disabled_features,
    )
    link_variables = cc_common.create_link_variables(
        feature_configuration = feature_configuration,
        cc_toolchain = cc_toolchain,
        is_linking_dynamic_library = False,
        runtime_library_search_directories = rpaths,
        user_link_flags = user_link_flags,
    )
    link_args = cc_common.get_memory_inefficient_command_line(
        feature_configuration = feature_configuration,
        action_name = CPP_LINK_EXECUTABLE_ACTION_NAME,
        variables = link_variables,
    )
    ld = cc_common.get_tool_for_action(
        feature_configuration = feature_configuration,
        action_name = CPP_LINK_EXECUTABLE_ACTION_NAME,
    )

    return ld, link_args

^^^ All that stuff up there is stuff I basically know nothing about. However, I can tell you that one avenue of potential crosslinking goodness is updating the cc linker to lld. I have no immediate knowledge of how that should be accomplished. it should be noted also that the churn in the CC toolchain is counterproductive to this exploration. @hlopko may be able to clue us in on what the state of LLD is in C++ land.

Now, back to "what needs to happen to get a crosslinker"

For lack of any specialized knowledge in this, but having a general familiarity with installing a crosslinker from the japaric/rust-cross repo, I figured we could just add a new optional attr to rust_toolchain like "linker_bin". The logic above can just check if a linker_bin is available and prefer that to whatever cc_common.get_tool_for_action decides to spit out from the default cc toolchain.

Moving backward now to the rust_repository_set decl -- we'd need to add a new attr to that macro configurable per-extra_target_triple for finding the linker, and then plumb that through rust/repositories.bzl:_load_rust_stdlib into the rust_toolchain entry yielded in the call to BUILD_for_rust_toolchain (if provided such a linker for the associated extra_target_triple is provided.

---

On Rust-LLD

Now, finding a custom crosslinker for the host to each target and including it into the workspace, is a hassle. Luckily, LLD solves this by just automatically supporting linking to every supported target on every host. Unfortunately, I wasn't able to find an authoritative source on exactly which targets are supported, though I know Cargo prefers LLD for some obscure targets and we could conceivably do so as well.

The LLD linker is provided in the rustc package, and we could add a filegroup BUILD rule into the rust/repositories.bzl:BUILD_for_compiler to expose it. It's available at lib/rustlib/${HOST_TRIPLE}/bin/lld.

[m3rcuriel]

I've had success currently setting up the cross-linker using typical Bazel cc toolchain setup, so I'm not sure it needs to be exposed through rules_rust specifically since it's arguably a cc link step, or more fairly it's on the onus of upstream (or more rules?) to set up some "LLVM" link step since even in our case it isn't a "rust specific" link since the cc deps would be linked in at that point.

Actually, I ran into an issue with rules_go where they weren't properly respecting the existing cc toolchain, so I imagine going out of the way to let existing structures exist might be ideal.

[TheAifam5]

I encounter similar problem when setting extra_target_triples as x86_64-unknown-uefi. Any workaround available?