Shared Formatter infrastructure

[MichaReiser]

I've been extracting the language-agnostic formatter infrastructure into rome-formatter until I hit a problem that Rust (for good reasons), doesn't allow to implement of a trait if both, the trait and the type are foreign to the current crate.

I hit this problem when I moved the Format trait to rome-formatter because:

• rome-formatter defines the Format trait
• rome-js-syntax defines the AstNode* types

Thus, neither Format nor the AstNode types are part of rome-js-formatter.

I'll go through the different options that I considered and explain why I decided for the last. I'll start with the options that I ruled out. My proposed option comes last.

Option 1

One option is to extract the shared formatter logic to rome-formatter and implement the formatters inside rome-js-syntax. This lifts Rust's constraint because rome-js-syntax defines the AstNodes.

The reason I ruled out this option is that it makes rome-js-syntax and all its dependencies depend on the formatting logic, including the parser. This will most likely result in cyclic dependencies and a significant slowdown in development speed because of the increased compilation times (rome-js-formatter and rome-js-syntax already the slowest crates when it comes to compilation time and combining the two will only make it worse).

Option 2

The second option is to forgo with the goal to have different rome-<language>-formatter crates for every language. This lifts Rust's constraint because rome-formatter defines the Format trait.

This option probably works well for now where we only have very few languages but I don't see it scale to multiple languages. Again, rome-js-formatter is one of the slowest crate when it comes to compilation time because it contains a lot of generated code. Adding more languages will make this worse, not just by the absolute time it takes to compile but also because recompilation becomes necessary if:

• the syntax of any supported language changes
• the formatting of any supported language changes

It also negatively impacts downstream dependencies because it becomes impossible to express that a crate only requires JS formatting, but doesn't depend on CSS formatting. That means, every time rome_formatter gets recompiled, all downstream dependencies must so as well. This will significantly slow down compilation time and our iteration speed.

Option 3

Since moving the formatting to the Format trait's crate or moving the formatting to the syntax crates are valid options, the only option to lift Rust's constraint is by introducing newtype wrappers and implementing Format for the new-type wrapper.

The way I think about this is the same as the Display struct for Path (the reasons why this struct is necessary differs for Path but the idea is the same).

The core idea is that each node gets a format() method that returns &Format<Node>. The Format<Node> struct is a new-type wrapper that wraps the node and implements Format.

**rome_formatter*`

trait Format {
  fn format(&self, formatter: &Formatter) -> FormatElement;
}

pub trait AsFormat<'a> {
    type Format: Format;
    
    fn as_format(&'a self) -> Self::Format;
    
    // To make format_elements![if.format(formatter)]` work
    fn format(&'a self, formatter: &Formatter) -> FormatResult {
        self.as_format().format(formatter)
    }
}

rome_js_formatter

struct FormatJsIfStatement<'node> {
  node: &'node JsIfStatement
}

impl Format for FormatJsIfStatement<'_> { ... }

impl<'a> AsFormat<'a> for JsIfStatement {
  type Format = FormatJsIfStatement<'a>;

  fn as_format(&'a self) -> Self::Format {
   FormatJsIfStatement { node: self }
  }
}

The new-type wrappers solve Rust's constraint because the wrapper types are defined in the rome-<language>-formatter traits, and thus, are local to the crate.

The main downsides of this approach are:

• Less discoverable because it requires importing the AsFormat trait
• Increase of generated code in the formatter crate

However, in my opinion the downsides are outweighed by the following upsides

• Keep the formatting out of the syntax crate
• it allows us to have separate formatting crates per language
• Both of these result in significant lower compilation time

What are your thoughts?

Future

I hope to soon replace format_elements with a macro that instead behaves more like format_args![formatter, node1, node2]. The motivation of doing this is that it's currently a bit of a hazzle because there are two formatting concepts:

• Format Something that can be formatted
• FormatElement: Some formatted output

and we often want to combine the two. Only having one concept (Format) and treating FormatElement as pure output of that simplifies the combination of different Formatable types.

[MichaReiser]

Lol, I guess I just fell for the same trap again. Implementing AsFormat for JsIfStatement won't be possible for the very same reason it isn't possible to implement Format. This only works if AsFormat is local to rome-js-formatter. I think that's a good enough compromise.

[MichaReiser]

Related Work

• Serde Remote Data

[NicholasLYang]

Ah yes, the orphan rule. Probably the top gripe I have with Rust.

I know this might be a ugly choice from a call-site perspective, but could we forgo traits entirely and just have a bunch of formatter methods? Like formatter.format_call_expression.

With AsFormat, it seems like we'd just be creating a new trait per crate. That's a valid option imo.

[MichaReiser]

format_call_expression that's something I'm currently exploring as well. These should probably become standalone functions to make the formatter language agnostic or part of an extension trait.

Yes, AsFormat would need to be re-implemented per crate. This makes it harder to share logic between languages (that operate on nodes) but we can work around that by either have wrapper structs that can wrap a formattable node or re-use on a lower level (check for suppression comment, formatting leading/trailing trivia).

The main challenge I see right now is that anything operating on lists format_nodes, format_delimited becomes more annoying and aren't even sure how to implement if AsFormat takes a reference to the node rather than cloning the node.

[NicholasLYang]

What I'm envisioning for format_call_expression is:

impl JsFormatter {
  fn format_call_expression(expr: JsCallExpression) {}
}

As for reusing logic, having it on a lower level makes sense to me. We could treat the language specific crates as essentially compiler front ends to a consistent back end.

Lists are tricky. Could we make format_delimited, format_nodes macros? In which case they can use whichever trait is implemented in their call site.

[ematipico]

I think that every logic that we have that we try to share should not be shared.

I've come to the conclusion that the main goal here is to try to create a shared infrastructure where any data structure (rome_rowan for our particular case) should be able to create a FormatElement. What happens in the middle is an implementation detail that should live exclusively inside the language itself.

Now, this has the big downside that most of the code we've created so far can't be reused for other languages (maybe the bare minimum about reusing text slices and comments), but from one point of view it kind of makes sense. Trivias might work differently in another language, lists too, etc.

Option 3 seems to be the best solution, but I'm not very fond of it.

[NicholasLYang]

I think that's a fair choice. I'm a fan of duplicating logic in the beginning and only deduplicating once we've gotten a first implementation going.