Replace tera template with serde_starlark

[dtolnay]

This PR is a proposal to move crate_universe away from using tera / jinja2 templates for BUILD files, to using serde + a serde data format for serializing Rust data structures directly to well-formatted Starlark. This approach was developed originally for use in https://github.com/facebookincubator/reindeer and has been used there happily for >3 years.

In this PR, I've started with changing only the generation of BUILD.bazel (i.e. not yet touching BUILD.$crate-$version.bazel). If, based on this PR the approach seems like something the maintainers would be on board with, then I can commit to updating BUILD.$crate-$version.bazel to be generated using serde also, in a following PR.

Overview

To illustrate the basic idea, generating something like the following target:

alias(
    name = "serde_starlark",
    actual = "@cui__serde_starlark-0.1.5//:serde_starlark",
    tags = ["manual"],
)

involves defining a Rust data structure that represents the content of the Starlark call:

#[derive(Serialize)]
#[serde(rename = "alias")]
pub struct Alias {
    pub name: String,
    pub actual: String,
    pub tags: Set<String>,
}

instantiating it:

let alias = Alias {
    name: "serde_starlark".to_owned(),
    actual: "@cui__serde_starlark-0.1.5//:serde_starlark".to_owned(),
    tags: Set::from(["manual".to_owned()]),
};

and then just serializing it:

let starlark = serde_starlark::to_string(&alias)?; // String

Motivation

1. Just write plain old Rust code. Comparing the implementation before and after this PR, consider this part that deals with creating targets for all the bins:

   {%- set crate = context.crates | get(key=id) %}
   {%- for rule in crate.targets %}
   {%- for rule_type, target in rule %}
   {%- if rule_type in ["Binary"] %}
   alias(
   ...
   {%- endif %}
   {%- endfor %}
   {%- endfor %}

   This isn't Rust code; it's a separate DSL that someone needs to learn. In it, all the conveniences of working with the underlying data structure natively in Rust are gone, like unwrap_or or iterator combinators. Enums and pattern matching / if let are gone. Indentation doesn't work; we can't use it because indenting the body of the loops would end up visible in the output and we don't want that. IDE support is gone because everything is just dynamically typed. In general all compile-time type safety is gone: you need to run the template to find out whether there are basic mistakes like mistyped variables or invalid jinja2 template syntax.

   The equivalent new code:

   let krate = &context.crates[crate_id];
   for rule in &krate.targets {
       if let Rule::Binary(bin) = rule {
           binaries.push(Alias {
               ...
           });
       }
   }

   It's basic obvious Rust control flow that anyone could write.

2. Generate good-looking, easily reviewable output. Look at crate_universe/3rdparty/crates/BUILD.syn-1.0.105.bazel. The tera-based renderer's output is full of meaningless noise like:

   proc_macro_deps = [
   ] + select({
       "//conditions:default": [
       ],
   }),
   rustc_env = {
   },
   rustc_env_files = select_with_or({
       "//conditions:default": [
       ],
   }),

   because it's simply too onerous to do something friendlier given the limited programming environment of these templates.

   Reindeer's output is much cleaner and compact. Simple things like #[serde(skip_serializing_if = "Foo::is_empty")] go a long way. Simple things like "don't include a # Binaries comment if there are no binaries targets being generated" are so easy in Rust and pretty much hopeless in the templates.

[UebelAndre]

Amazing crate! You're a legend!

Would serde_starlark be capable of replacing all uses of tera? Any blockers in doing so?

[UebelAndre]

All the content here seems to be simpler versions of things in crate::utils::starlark. Any reason not to move it to the same module?

[dtolnay]

Done.

[dtolnay]

Would serde_starlark be capable of replacing all uses of tera?

Yes.

[UebelAndre]

Thank you so much! Hopefully we can migrate everything to this crate in the near future 😄