Support async parser for embedded languages (#13211)

Co-authored-by: fisker Cheung <lionkay@gmail.com>

changelog_unreleased/api/12574.md

@@ -1,7 +1,8 @@
-#### [BREAKING] Change public apis to asynchronous (#12574, #12788, #12790 by @fisker)
+#### [BREAKING] Change public APIs to asynchronous (#12574, #12788, #12790 by @fisker)
 
 - `prettier.format()` returns `Promise<string>`
 - `prettier.formatWithCursor()` returns `Promise<{formatted: string, cursorOffset: number}>`
+- `prettier.formatAST()` returns `Promise<string>`
 - `prettier.check()` returns `Promise<boolean>`
 - `prettier.getSupportInfo()` returns `Promise`
 - `prettier.resolveConfig.sync` is removed

changelog_unreleased/api/12748.md

@@ -1,7 +1,7 @@
-#### [HIGHLIGHT] Support plugin with async parse function (#12748 by @fisker)
+#### [HIGHLIGHT] Support plugins with async parsers (#12748 by @fisker, #13211 by @thorn0 and @fisker)
 
-[`parse` function](https://prettier.io/docs/en/plugins.html#parsers) in plugin can return a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) now.
+[`parse` function](https://prettier.io/docs/en/plugins.html#parsers) in a plugin can return a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) now.
 
-<!-- TODO: Discuss this before v3 release -->
+In order to support async parsers for embedded languages, we had to introduce a breaking change to the plugin API. Namely, the `embed` method of a printer has now to match a completely new signature, incompatible with previous versions. If you're a plugin author and your plugins don't define `embed`, you have nothing to worry about, otherwise see the [docs](/docs/plugins.html#optional-embed) for details.
 
-Note: Since [`embed` functions in plugin](https://prettier.io/docs/en/plugins.html#optional-embed) are still sync, plugins with async parser can't be used to format embed code.
+Also, the `preprocess` method of a printer can return a promise now.

docs/plugins.md

@@ -183,11 +183,14 @@ import { doc } from "prettier";
 const { join, line, ifBreak, group } = doc.builders;
 ```
 
-The printing process works as follows:
+The printing process consists of the following steps:
 
-1. `preprocess(ast: AST, options: object): AST`, if available, is called. It is passed the AST from the _parser_. The AST returned by `preprocess` will be used by Prettier. If `preprocess` is not defined, the AST returned from the _parser_ will be used.
-2. Comments are attached to the AST (see _Handling comments in a printer_ for details).
-3. A Doc is recursively constructed from the AST. i) `embed(path: AstPath, print, textToDoc, options: object): Doc | null` is called on each AST node. If `embed` returns a Doc, that Doc is used. ii) If `embed` is undefined or returns a falsy value, `print(path: AstPath, options: object, print): Doc` is called on each AST node.
+1. **AST preprocessing** (optional). See [`preprocess`](#optional-preprocess).
+2. **Comment attachment** (optional). See [Handling comments in a printer](#handling-comments-in-a-printer).
+3. **Processing embedded languages** (optional). The [`embed`](#optional-embed) method, if defined, is called for each node, depth-first. While, for performance reasons, the recursion itself is synchronous, `embed` may return asynchronous functions that can call other parsers and printers to compose docs for embedded syntaxes like CSS-in-JS. These returned functions are queued up and sequentially executed before the next step.
+4. **Recursive printing**. A doc is recursively constructed from the AST. Starting from the root node:
+   - If, from the step 3, there is an embedded language doc associated with the current node, this doc is used.
+   - Otherwise, the `print(path, options, print): Doc` method is called. It composes a doc for the current node, often by printing child nodes using the `print` callback.
 
 #### `print`
 
@@ -212,9 +215,9 @@ The `print` function is passed the following parameters:
 Here’s a simplified example to give an idea of what a typical implementation of `print` looks like:
 
 ```js
-const {
-  builders: { group, indent, join, line, softline },
-} = require("prettier").doc;
+import { doc } from "prettier";
+
+const { group, indent, join, line, softline } = doc.builders;
 
 function print(path, options, print) {
   const node = path.getValue();
@@ -248,42 +251,65 @@ Check out [prettier-python's printer](https://github.com/prettier/prettier-pytho
 
 #### (optional) `embed`
 
-The `embed` function is called when the plugin needs to print one language inside another. Examples of this are printing CSS-in-JS or fenced code blocks in Markdown. Its signature is:
+A printer can have the `embed` method to print one language inside another. Examples of this are printing CSS-in-JS or fenced code blocks in Markdown. The signature is:
 
 ```ts
 function embed(
   // Path to the current AST node
   path: AstPath,
-  // Print a node with the current printer
-  print: (selector?: string | number | Array<string | number> | AstPath) => Doc,
-  // Parse and print some text using a different parser.
-  // You should set `options.parser` to specify which parser to use.
-  textToDoc: (text: string, options: object) => Doc,
   // Current options
-  options: object
-): Doc | null;
+  options: Options
+):
+  | ((
+      // Parses and prints the passed text using a different parser.
+      // You should set `options.parser` to specify which parser to use.
+      textToDoc: (text: string, options: Options) => Promise<Doc>,
+      // Prints the current node or its descendant node with the current printer
+      print: (
+        selector?: string | number | Array<string | number> | AstPath
+      ) => Doc,
+      // The following two arguments are passed for convenience.
+      // They're the same `path` and `options` that are passed to `embed`.
+      path: AstPath,
+      options: Options
+    ) => Promise<Doc | undefined> | Doc | undefined)
+  | Doc
+  | undefined;
 ```
 
-The `embed` function acts like the `print` function, except that it is passed an additional `textToDoc` function, which can be used to render a doc using a different plugin. The `embed` function returns a Doc or a falsy value. If a falsy value is returned, the `print` function is called with the current `path`. If a Doc is returned, that Doc is used in printing and the `print` function is not called.
+The `embed` method is similar to the `print` method in that it maps AST nodes to docs, but unlike `print`, it has power to do async work by returning an async function. That function's first parameter, the `textToDoc` async function, can be used to render a doc using a different plugin.
+
+If a function returned from `embed` returns a doc or a promise that resolves to a doc, that doc will be used in printing, and the `print` method won’t be called for this node. It's also possible and, in rare situations, might be convenient to return a doc synchronously directly from `embed`, however `textToDoc` and the `print` callback aren’t available at that case. Return a function to get them.
 
-For example, a plugin that had nodes with embedded JavaScript might have the following `embed` function:
+If `embed` returns `undefined`, or if a function it returned returns `undefined` or a promise that resolves to `undefined`, the node will be printed normally with the `print` method. Same will happen if a returned function throws an error or returns a promise that rejects (e.g., if a parsing error has happened). Set the `PRETTIER_DEBUG` environment variable to a non-empty value if you want Prettier to rethrow these errors.
+
+For example, a plugin that has nodes with embedded JavaScript might have the following `embed` method:
 
 ```js
-function embed(path, print, textToDoc, options) {
+function embed(path, options) {
   const node = path.getValue();
   if (node.type === "javascript") {
-    return textToDoc(node.javaScriptText, { ...options, parser: "babel" });
+    return async (textToDoc) => {
+      return [
+        "<script>",
+        hardline,
+        await textToDoc(node.javaScriptCode, { parser: "babel" }),
+        hardline,
+        "</script>",
+      ];
+    };
   }
-  return false;
 }
 ```
 
+If the [`--embedded-language-formatting`](options.md#embedded-language-formatting) option is set to `off`, the embedding step is entirely skipped, `embed` isn’t called, and all nodes are printed with the `print` method.
+
 #### (optional) `preprocess`
 
-The preprocess function can process the AST from parser before passing into `print` function.
+The `preprocess` method can process the AST from the parser before passing it into the `print` method.
 
 ```ts
-function preprocess(ast: AST, options: object): AST;
+function preprocess(ast: AST, options: Options): AST | Promise<AST>;
 ```
 
 #### (optional) `insertPragma`
@@ -455,9 +481,9 @@ function isPreviousLineEmpty<N>(text: string, node: N, locStart: (node: N) => nu
 Since plugins can be resolved using relative paths, when working on one you can do:
 
 ```js
-const prettier = require("prettier");
+import * as prettier from "prettier";
 const code = "(add 1 2)";
-prettier.format(code, {
+await prettier.format(code, {
   parser: "lisp",
   plugins: ["."],
 });

src/language-css/embed.js

@@ -1,12 +1,14 @@
 import { hardline } from "../document/builders.js";
 import printFrontMatter from "../utils/front-matter/print.js";
 
-function embed(path, print, textToDoc /*, options */) {
+function embed(path) {
   const node = path.getValue();
 
   if (node.type === "front-matter") {
-    const doc = printFrontMatter(node, textToDoc);
-    return doc ? [doc, hardline] : "";
+    return async (textToDoc) => {
+      const doc = await printFrontMatter(node, textToDoc);
+      return doc ? [doc, hardline] : undefined;
+    };
   }
 }