Unify bazel configuration

[Colecf]

Description of the feature request:

While working on configuration for builds in Roboleaf (building android with bazel), we noticed that bazel configuration seems to be in a half-finished state and there's some improvements that could be made to make it much clearer and more flexible.

Notably, there is currently no way to transition on constraint settings/values, other than transitioning on --platforms or some other command line flags like --cpu. However, transitioning on command line flags is supposed to be deprecated in the future. Transitioning on the target architecture is probably the most common transition to do, in order to build fat apks and similar file formats like Apple's universal apps, so it's surprising that it's only possible to do this via deprecated behavior in bazel.

It sounds like the goal is to deprecate constraint settings/values and replace them with build settings. This would solve the transition issue, but would likely require some extra features to platforms and toolchains in order to make them work with build settings instead of constraint settings.

Am I accurate in assuming that bazel is migrating to build settings? Is there a timeline / roadmap for it?

Android is building a significant amount of tech debt trying to work around the issue. One decision we made was to store all our non-constraint setting configurability inside of a rule instead of using build settings, because when using build settings there's no way for the user to specify a certain default group of settings like they can with the --platforms flag. (See this google-internal doc for more)

What underlying problem are you trying to solve with this feature?

Allowing easily changing configuration (in our case, which android product you're building for) both on the command line and via transitions.

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[jin]

cc @lizkammer @lberki

[lberki]

cc @gregestren

Do you consider wrapping the product configuration variables in a rule tech debt?

I thought that it was a pretty neat abstraction because one custom build setting is better than hundreds of them. If we wanted to change single product variables in configuration transitions, we'd have a problem, I guess, but to the best of my knowledge, that's not necessary.

You can both set label_flag() flags on the command line as demonstrated in the example here:

https://github.com/bazelbuild/examples/tree/HEAD/configurations/label_typed_build_setting

and change its value in a configuration transition. Configuration transitions on custom flags are demonstrated here:

https://github.com/bazelbuild/examples/tree/9e57222e580aa33c7e01ff3e78b7759c87d8709b/configurations/attaching_transitions_to_rules

It's a string_flag not a label_flag but it does work with label_flag, too.

[gregestren]

Hi @Colecf

To be clear, we intend to further normalize constraint settings and constraint values. Those are the components that make up platforms, which is an API we continue to roll out to rules and make ubiquitous.

I don't consider the configuration APi to be incomplete in that regard. Although I understand that you have an interest in merging platform and flag properties in a way that's novel for this API.

As I understand you're interested in having a grab bag of settings that sets both platform settings (which we'd naturally model as constraint settings) and build settings that maybe aren't part of the platform. Or alternatively: build settings that are part of the platform but you want to be individually user-overrideable.

I like the label_flag idea. But it's true, if I'm reading it right, that it plays awkwardly with the platforms and flags APIs by arbitrarily mixing them. I'd have trouble commenting further without understanding in better detail the kinds of "products" you want to model.

[gregestren]

I think you have workarounds for your specific needs now. Rather than try to address this bug specifically I'll add it as a reference to some wider brainstorming we're doing about improving & consolidating the configuration APIs. This is good motivating feedback for that.

[Colecf]

I think what I'm most hung up about is how most of configuration seems to resolve around constraint_values/settings, (toolchain resolution, --platforms and --host_platforms on the command line), but they are less powerful than build settings (which have transitions, unbounded values, and the ability to be set directly on the command line).

As I understand you're interested in having a grab bag of settings that sets both platform settings (which we'd naturally model as constraint settings) and build settings that maybe aren't part of the platform. Or alternatively: build settings that are part of the platform but you want to be individually user-overridable.

I don't understand this part. It seems like you're trying to imply constraint settings and build settings have different idealistic use cases, but as far as I understand it, they are basically the same thing but have slightly different capabilities/restrictions.

That's why I'm basically proposing we deprecate constraint settings/values and migrate just to build settings, in an attempt to clean up bazel configuration in the process. The biggest blocker to that seems to be toolchain resolution. Is there anything stopping us from making toolchain resolution also work based off of build settings?

Platforms could also set build settings as you're proposing in "Centralized exec configurations", but you could also go the other way as well, and standardize on using transitions instead of platforms if everything was a build setting.

[gregestren]

Toolchain resolution was the original motivation behind constraint settings. Precisely to move away from relying on flags. And precisely to avoid having to set multiple flags at the command line, which historically made cross-language cross-platform builds bug-prone and unwieldy. We're still in the process of migrating away from flag-based builds, and trying to reduce the number of flags overall.

There are legitimate use cases to want to mix flags and constraints together. We're exploring APIs to mix them better now. We don't believe eliminating one can practically work.

[gregestren]

I think we're somehow still talking around each other. I'd love us to get on the same page ultimately, especially if that empowers us to make changes that make your work more effective.

[lberki]

I agree with the general sentiment that it's weird to have two "kinds of things" that describe the current configuration: build settings (very flexible, cannot affect toolchain resolution) and platform constraints (very limited, can affect toolchain resolution).

If my memory serves well, the reason why platforms were designed this way was to make the resolution algorithm simple. I could imagine extending it in some way that can take the full configuration into account, but it's by no means trivial and before we set out on the long journey of designing and implementing such a thing, I'd like to see it laid out clearly why that thing is necessary. Doing so kind of appeals my sense of beauty, but let's make sure that what we do has a good return on investment.

In particular, I haven't seen an answer to my last comment #16758 (comment) ; my best understanding was that the system @Colecf is planning to implement that will represent Android product variables by a provider in a rule doesn't require any extra features in toolchain resolution.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.