toolchains attribute requires toolchain_type target to expose provider which it can't

[illicitonion]

Description of the problem / feature request:

As far as I can tell, there's no way for a starlark-defined toolchain to be consumed for make variable expansion.

I wanted to use perl from a Perl toolchain from rules_perl in a genrule:

genrule(
    name = "run_perl",
    cmd = "$(PERL) --help > $@",
    outs = ["out"],
    toolchains = [
        "@rules_perl//:toolchain_type",
    ],
)

Unfortunately, this doesn't quite work for two reasons.

1. "in toolchains attribute of genrule rule //:run_perl: '@rules_perl//:toolchain_type' does not have mandatory providers: 'TemplateVariableInfo'"

As far as I can tell, there's no way for a toolchain_type to expose extra providers. I can substitute in the concrete toolchain target: "@perl_darwin_amd64//:toolchain_impl", but this feels like it's subverting toolchain selection. Shouldn't the toolchain_type target proxy its providers through?

2. There are files required from the toolchain, and these aren't exposed. I need to manually add: tools = ["@perl_darwin_amd64//:runtime"] to the target, and then the files are present.

So this target actually works:

genrule(
    name = "run_perl",
    cmd = "$(PERL) --help > $@",
    outs = ["out"],
    toolchains = [
        "@perl_darwin_amd64//:toolchain_impl",
    ],
    tools = [
        "@perl_darwin_amd64//:runtime",
    ],
)

What operating system are you running Bazel on?

macOS

What's the output of bazel info release?

release 4.2.1

Here's my WORKSPACE file to repro:

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
    name = "rules_perl",
    remote = "https://github.com/illicitonion/rules_perl.git",
    commit = "9c3f37dfb08085cbba0f2566b3dc10d9e0c73ab8",
)

load("@rules_perl//perl:deps.bzl", "perl_register_toolchains", "perl_rules_dependencies")

perl_rules_dependencies()
perl_register_toolchains()

[illicitonion]

/cc @katre @gregestren

[katre]

This is an unfortunate wart of Bazel. The toolchains attribute on most rules does not want a toolchain_type: it wants something else that exposes TemplateVariableInfo providers. This can be several other concrete toolchains (I know CC toolchains and Java toolchains provide this, I don't know about Perl toolchains).

This was named separately from the toolchain_type and related rules, and is very annoying (I didn't name it! I promise).

In an ideal world, toolchain resolution would also consult the toolchains attribute and add those types to the set of toolchain types needed for resolution, as you note. Unfortunately, we haven't implemented this yet.

One workaround is to add a simple rule which does declare the toolchain type of @rules_perl//:toolchain_type, and then forwards the TemplateVariableInfo provider from that toolchain, and use the toolchain-forwarding rule in your target that wants Perl toolchain variables. This is actually how the Java rules handle all toolchain dependencies, see https://cs.opensource.google/bazel/bazel/+/master:tools/jdk/java_toolchain_alias.bzl?ss=bazel&q=java_toolchain_alias for the implementation of java_toolchain_alias.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[illicitonion]

@bazelbuild/triage - this is still relevant

[katre]

The problem is that the genrule.toolchains attribute isn't looking for a toolchain_type, due to inconsistent naming. You should not be passing a toolchain type here: it wants an actual toolchain implementation.

Ideally we'd fix this to also take a toolchain type, perform toolchain resolution, and then get the make variables from the toolchain, but this is actually pretty complex (right now the set of toolchain types for a rule is static, this would make it dynamic).