feature(rome_js_formatter): Inline comment formatting

[MichaReiser]

This PR improves the formatting of inline comments (/* comment */) so that it doesn't insert any space after a group start token (any of (, [, {) nor before a group end token (any of ), }, ], ,, ;, EOF, .).

The implementation introduces new helpers to format tokens:

• format_inserted: To format a token that isn't present in the source code. For example, when adding parentheses around an expression. Use this over token
• format_replaced: To replace the content of a token with some other content
• format_removed: To remove a token but still format the tokens trivia
• format_only_if_breaks: A combination of format_replaced and format_removed. Used in the cases where a token should be removed if its enclosing group doesn't break but otherwise gets formatted as is (or with slightly different content). Commonly used when formatting the trailing separator of a list.

The PR introduces two new formatter states:

• The kind of the last formatted token. Necessary to know whether a leading space must be inserted before the first leading inline comment or not.
• Whether the last token's last trivia is an inline comment in which case the next token must insert a leading space ONLY IF it is not a group end token. This is necessary to avoid a token looking ahead to guess what next token (which wouldn't work in case of inserted tokens)

This PR moves the comment formatting to rome_formatter because there's now nothing JS-specific about formatting comments.

Closes #2447

Differences to Prettier's formatting'

I decided to diverge from Prettier for the following case:

// Input
function name/*comment*/   () {
}

// Prettier
function name /*comment*/() {
}

// Rome
function name /*comment*/ () {
}

Prettier omits whitespace for inline comments before a ( token but it doesn't do so for [ or { in which case prettier prints a single whitespace too. In my view, inline comments before start group tokens should either all be separated from the token by a whitespace or the whitespace is omitted for all of them. That's why I didn't align Rome with Prettier's behaviour in this case.

Test Plan

cargo test.

Reviewed the changed snapshot, added a few new tests

Increases compatibility by ~0.6% (from 72.65% to 73.25%)

[cloudflare-pages]

Deploying with    Cloudflare Pages

Latest commit:	61bc52f
Status:	✅  Deploy successful!
Preview URL:	https://21f3b613.tools-8rn.pages.dev
Branch Preview URL:	https://feature-inline-comment-spaci.tools-8rn.pages.dev

View logs

[MichaReiser]

We should use format_removed instead

[MichaReiser]

The space before the , is because of the array hole

[MichaReiser]

I renamed this to avoid naming conflicts with the newly introduced Comment struct that stores comments.

[MichaReiser]

I wanted to avoid the formatter needing a L type parameter. I believe this should be safe even in cases where we mix different languages because it then always inserts a leading space before the next token (same as for skipped token trivia)

[MichaReiser]

Ideally, it would be possible that rome_formatter defines a new Language trait that extends rome_rowan::Language and adds these methods to it. Unfortunately, this isn't possible without setup because rome_js_formatter isn't able to implement that trait because of the orphan rule: Both the trait and the JsLanguage struct are foreign to the crate.

[github-actions]

Playground for commit d1e5a14

[ematipico]

Would it be possible to add new tests to show us the new changes? This is more needed, I think, especially because we decided to diverge.

[ematipico]

If I understood correctly, here we check if the next token has comments?

EDIT: actually, no. The next comment says if previous token has comments (my initial thoughts), but the description of the API says has_trailing_inline_comment says:

Returns `true` if there's a trailing inline comment before the next token or the next token's comments.

Maybe we should revisit the documentation or align the comments, so there's no confusion. Or call the API with a different name? previous_token_has_trailing_inline_comment? Just a suggestion

[MichaReiser]

This is good feedback and you're right, it's the previous token. The logic behind this is non-trivial. What it tracks is whether the last written comment requires a space before whatever content gets written next (that can be a comment, skipped token, or a regular token). Let's see if I can improve the documentation somehow.

[MichaReiser]

Would it be possible to add new tests to show us the new changes? This is more needed, I think, especially because we decided to diverge.

I added a few more tests to the comments test but good call out. Let me add some tests that explicitly verify the cases where we diverge from prettier's formatting.