Autotools examples

[jsharpe]

Add targets to build autotools with the aim of eventually moving these to use the built_tools framework and put into a toolchain that the configure_make rule can use to ensure hermetic use of autotools during the build.

Fixes an issue with putting the bin directory from built tools onto the path when using tools_deps

[jsharpe]

This is green now but with the caveat that only m4 actually produces a working tool. automake isn't being built as autoconf generates scripts with hardcoded paths that match the install prefix and hence contains sandbox paths. Some can be worked around using environment variables but not all config unfortunately.

Anyone have any smart ideas on how to produce a reproducible and relocatable autoconf install?

[UebelAndre]

automake isn't being built as autoconf generates scripts with hardcoded paths that match the install prefix and hence contains sandbox paths.

would something like replace_in_files help?
https://github.com/bazelbuild/rules_foreign_cc/blob/f01fd353ee2adcd55aab899c12fa2733223228a1/foreign_cc/private/framework/toolchains/linux_commands.bzl#L62-L83

What is the specific issue with the hard coded paths?

[jsharpe]

automake isn't being built as autoconf generates scripts with hardcoded paths that match the install prefix and hence contains sandbox paths.

would something like replace_in_files help?

https://github.com/bazelbuild/rules_foreign_cc/blob/f01fd353ee2adcd55aab899c12fa2733223228a1/foreign_cc/private/framework/toolchains/linux_commands.bzl#L62-L83

What is the specific issue with the hard coded paths?

It's the generated autom4te script and its config files; they include the path specified as the install prefix in them and so this results in files not being found. Some can be worked around by setting environment variables to override such paths but this doesn't work for the config files where absolute paths are required (although I've not looked to see if it would expand an environment variable at time of use)

[djmarcin]

I actually just had this exact problem with autoconf — but my “solution” was to tag the targets as ‘no-sandbox’ and ‘no-cache’ so that the embedded paths would be the persistent execroot path and not an ephemeral sandbox path. More of a workaround than a solution, really. It requires #765 because otherwise rules_foreign_cc throws away these tags.

[djmarcin]

Curious if @jsharpe has had any further ideas here. I've been thinking about this a lot because we are trying to enable remote exec for our build and the no-sandbox solution doesn't work there. I've also tried a repository rule that builds autotools, which also only works locally -- once a task on a remote machine tries to use autoconf it fails because it tries to read the execroot path from my local machine, which obviously doesn't exist remotely.

It's unsatisfying, but without significant patches for autotools to be arbitrarily relocatable, I don't see how it can be successfully vendored inside bazel. I think I'm going to solve our need by just manually installing the versions we need into well known locations on all our build machines and referencing it directly. I think there are a few other tools in this non-relocatable bucket that need the same treatment (and by extension, can't effectively be built with rules_foreign_cc)

[jsharpe]

@UebelAndre PTAL.
I've not worked out a way to make the autoconf build relocatable and so automake won't build either. I've not turned on the build for either of these for now but this at least fixes the bison build by building m4 from source (although it doesn't work on RBE due to the timestamping issues 😢 ).
I could remove the autoconf and automake build files from this PR if you'd prefer not to add the broken build files?

[UebelAndre]

I think this is mostly fine since it's just adding examples. Is this a stepping stone to building autotools for the foreign_cc toolchains? I'd otherwise say maybe there wouldn't be much value in the examples since they're gonna be ripped out for something better maintained.

But I leave the decision up to you. One question but otherwise looks good!

[jsharpe]

I think this is mostly fine since it's just adding examples. Is this a stepping stone to building autotools for the foreign_cc toolchains? I'd otherwise say maybe there wouldn't be much value in the examples since they're gonna be ripped out for something better maintained.

But I leave the decision up to you. One question but otherwise looks good!

Yes; my idea was to get them building in examples, then add toolchain definitions for each of the components then migrate the examples into built_tools to create a hermetic toolchain setup for autotools et al.

[jsharpe]

Note that native rules for building some of these tools exist e.g. m4: https://github.com/hdl/bazel_rules_hdl/tree/main/dependency_support/org_gnu_m4

[UebelAndre]

By the way: Buildifier needs to be run 😅