feat(assist): import organizer revamping

[Conaclos]

Summary

This long awaited feature is finally here! It is one of the most difficult features I have implemented. It took me one failed attempt and many hours. I am very happy with the result.

This PR implements #3177 with some divergences:

• Import merging is not implemented yet.
  This is left to a future PR.
• Import groups support glob exceptions.
  This drastically simplifies import matching and makes it more powerful.

This new import organizer provides the following features:

• It orders named specifiers using a natural order

  - import { A10, A9 } from "";
  + import { A9, A10 } from "";
  
  - export { A10, A9 } from "";
  + export { A9, A10 } from "";

• It orders import attributes according to their keys using a natural order

  - import {} from "" with { "k10": 0, "k9": 0 };
  + import {} from "" with { "k9": 0, "k10": 0 };
  
  - export {} from "" with { "k10": 0, "k9": 0 };
  + export {} from "" with { "k9": 0, "k10": 0 };

• Imports and exports are separated into chunks before to be sorted.
  A new chunk starts every time we met a side effect (bare) import, a distinct statement kind, or a detached comment.
  A detached comment is separated by a blank line of the import/export.

  // Chunk 1
  import { A } from "a";
  
  import { B } from "b";
  // attached comment
  import { C } from "c";
  import "side-ffect"
  // Chunk 2
  import { D } from "d";
  
  // Chunk 3 (detached comment)
  
  import { E } from "e";
  
  // Chunk 4
  export { F } from "f";
  export { G } from "g";
  
  function f() {}
  
  // Chunk 5
  export { FH } from "h";

• Chunk of imports or exports are first sorted according to the group they belong to.
  By default, there is a single group: the implicit group.
  Users can define their own groups and define an order between them.
  The implicit group always comes last.
  Groups are either predefined (like :BUN: and :NODE:) or globs with exceptions.

• Imports of a group are sorted according to other parameters, notably their import source.
  See the RFC and the implementation for more details.

• The implementation takes care of removing and adding newlines where required.
  Chunks are separated by a blank line except for side effect imports when other imports surround them.
  Comments are either attached or detached to an import/export.
  A detached comment (separated by a blank line of the import/export) creates a new chunk and is kept above the chunk.
  I added an exception for the header comment in a file: this comment is never attached. This ensures that Copyright notice are never moved.

Limitation / Possible improvements

• We could improve our glob handling:
  • @scope/lib doesn't match the glob @scope/lib/**.
    This could create some frictions.
  • It is currently not possible to write globs like node:**
    This could be easily fixed.
    As a workaround, users have to match against both node:* and node:*/**
• Trivia handling when sorting named specifiers and attributes should be improved. (Done)
• Some data structures could be shared for sorting distinct named specifiers and import attributes. (Not worth the complexity?)

Implementation strategy

The run procedure checks if everything is organized and register any "issues" that it founds.
An issue can report an unorganized import/export or an unsorted chunk (more precisely unsorted prefix of a chunk).
The action procedure is responsible for fixing the found issues.

This is different of the previous implementation where the entire sorting logic was in run.
This new approach pursuits two objectives:

1. Make the happy path (The imports and exports are already organized) as fast as possible.
2. Try to reduce the work to do for sorting.
   Notably all the chunk logic is in run.
   action doesn't bother about it.

Also, by dividing chunk sorting and individual import/export organizer, we are able to avoid a costly chunk sort when only named specifiers of an import are unsorted.

I think there is room for improving the code. However, it is already a big PR and I would like to ship it before we release Biome 2.0 beta.

Test Plan

I added some tests.

[github-actions]

Parser conformance results on

js/262

Test result	main count	This PR count	Difference
Total	50431	50431	0
Passed	49116	49116	0
Failed	1315	1315	0
Panics	0	0	0
Coverage	97.39%	97.39%	0.00%

jsx/babel

Test result	main count	This PR count	Difference
Total	40	40	0
Passed	37	37	0
Failed	3	3	0
Panics	0	0	0
Coverage	92.50%	92.50%	0.00%

symbols/microsoft

Test result	main count	This PR count	Difference
Total	6645	6645	0
Passed	2229	2229	0
Failed	4416	4416	0
Panics	0	0	0
Coverage	33.54%	33.54%	0.00%

ts/babel

Test result	main count	This PR count	Difference
Total	798	798	0
Passed	706	706	0
Failed	92	92	0
Panics	0	0	0
Coverage	88.47%	88.47%	0.00%

ts/microsoft

Test result	main count	This PR count	Difference
Total	18664	18664	0
Passed	14314	14314	0
Failed	4350	4350	0
Panics	0	0	0
Coverage	76.69%	76.69%	0.00%

[codspeed-hq]

CodSpeed Performance Report

Merging #5364 will not alter performance

_{Comparing conaclos/import-organizer-revamping-2 (ea6b399) with main (2b3a577)}

Summary

✅ 95 untouched benchmarks

[dyc3]

Not a full review, looks great at a glance!

[arendjr]

Nice work! Do you still want to include this into the beta?

[Conaclos]

Nice work! Do you still want to include this into the beta?

Yes. I can add something to the blog post.

[ematipico]

The documentation must be updated to reflect the new options. Which means that:

• we need to document legacy and how it works (probably not needed but it needs to be highlighted)
• we need to document importGroups and provide enough examples to explain the new behaviour

I'm not going to block the PR, but this is something we must do before the beta, if we plan to release it for the beta

[arendjr]

I think we’re getting there, right? 😊

[Conaclos]

I think we’re getting there, right? 😊

I have just committed the blank-line between groups feature.
I think we have enough features for the beta.

I have to document the new system before merging the PR.

[Conaclos]

The documentation is committed. Feel free to review.

[ematipico]

This is an outstanding feature; thank you, @Conaclos! I'm sure a lot of users will be pleased. I hope you're proud of what you just did.

I left various comments around the docs. Many are grammar fixes. Here are some high-level pointers:

• consider a glossary to explain specific terms to place at the beginning
• Let's first discuss our defaults without mentioning custom groups. Then, once all the important information is listed, we can have "custom groups" at the very end. A user needs to know about default groups, blank lines, comments, etc., to create groups.
• Before showing the example, could you explain it and describe its outcome, too? I know the codegen shows a diagnostic, but it's important for accessibility too.

[ematipico]

I know that I am reviewing only the docs now, but I wonder if I should narrow down the expectation only to multiline comments. What do you think?

[Conaclos]

What do you mean?

[arendjr]

Suggested change

	/// Conversely, `@any/lib/subpath` matches `@any/lib/**`, but not `@any/lib`.
	/// Thus you have to specify the two patterns if you want to accept all imports/exports that starts with `@any/lib`.
	/// Conversely, `@any/lib/subpath` is matched by `@any/lib/**`, but not `@any/lib`.
	/// So you have to specify both patterns if you want to accept all imports/exports that start with `@any/lib`.

[Conaclos]

is matched by

Is it not the reverse?
A string matches a glob pattern?

[arendjr]

Well, there's a subject doing the matching, and an object being matched. To be really pedantic, I think you should even say there's an algorithm using a glob to match a string (because a glob on its own cannot perform an action), meaning the string is matched by the algorithm. But more commonly I think it's acceptable to say a glob is doing the matching, which is also how you're using it almost everywhere else in the documentation, including the sentence above. But in a few places, like here, you swap the subject and the object, which is confusing because it leads the reader to believe @any/lib/subpath is the glob.

When I read this, I literally stopped to wonder what part of @any/lib/subpath was the pattern.

[arendjr]

Left one more round of suggestions/typo fixes. They should all be directly applicable from the comment ;)

Thanks for your immense work on this!

[arendjr]

Suggested change

	/// You have to understand the structure of a source to understand which source match a glob.
	/// You have to understand the structure of a source to understand how it can be matched by a glob pattern.

[arendjr]

Suggested change

	/// `file.js` matches `*.js`.
	/// Conversely, `src/file.js` doesn't match `*.js`
	/// `*.js` matches `file.js`, but does not match `src/file.js`.

[arendjr]

Suggested change

	/// `file.js` and `src/file.js` match `**` and `**/*.js`
	/// Conversely, `README.txt` doesn't match `**/*.js` because the source ends with `.txt`.
	/// `**` and `**/*.js` match both `file.js` and `src/file.js`.
	/// Conversely, **/*.js` doesn't match `README.txt` because the source ends with `.txt`.

[arendjr]

Suggested change

	/// `file.js` matches `!*.test.js`.
	/// `src/file.js` matches `!*.js` because the source contains several segments.
	/// `!*.test.js` matches `file.js` because it doesn't end with `.test.js`.
	/// `!*.js` matches `src/file.js` because the source contains several segments.