What would be the ideal transpilation story?

[benjamn]

Since it will take some time for native top-level await to find its way into every JavaScript environment, it's important to think about how this functionality can be provided by transpilers in the meantime.

As I see it, the underlying goal of this proposal is to enable authoring modules whose evaluation is asynchronous rather than synchronous, without requiring other modules to be aware of that distinction. While top-level await may be an ergonomic way to achieve that goal, it's worth considering how asynchronous modules could be written without using the top-level await syntax.

As an analogous historical precedent, the adoption of async/await benefitted from the prior standardization of generator functions and Promises, since they provided a convenient compilation target for async functions. In general, JavaScript benefits when advanced features can be explained in terms of more fundamental features. It's a good thing that any async function can be implemented as a function that just happens to return a Promise object, because it means native async functions can be introduced in the future without breaking existing APIs.

If there was a way to define a module with asynchronous evaluation using the existing syntax of ECMAScript modules, then transpilers like Babel could compile top-level await to valid ECMAScript module syntax, rather than abandoning ESM syntax in favor of CommonJS.

While it's tempting to use the export function then trick, that only works for dynamic import(), and doesn't work (yet) for static import declarations, though we might consider making static import consistent with dynamic import() in this regard.

A less clever strategy would be to allow modules to export a specially-named function that returns a Promise. For example, consider the following source code using top-level await:

import { x, getFoo } from "module";
export const foo = await getFoo(x);

If we had a more fundamental way of authoring asynchronous modules (such as exporting a function named __evaluate__), then the code above could be transpiled to something like

import { x, getFoo } from "module";
export let foo;
export async function __evaluate__() {
  foo = await getFoo(x);
}

This is essentially the strategy of exporting an async function, as mentioned in the Motivation section, except that the language would take care of invoking __evaluate__ and awaiting its Promise, so that importing modules would not have to modify their code. Ideally __evaluate__ could be any function that returns a Promise, not necessarily a native async function (useful if you're targeting older browsers).

If both static import and dynamic import() were made to respect the __evaluate__ function (🚨this requires spec changes 🚨), then the original top-level await code could be used in browsers long before top-level await achieved native support, and our transpilers would have an easier time simulating top-level await in older JS engines, and we could begin exploring the consequences of asynchronous module evaluation without waiting for top-level await to achieve consensus.

[bmeck]

is this use case strictly required? various platform features are not able to be transpiled.

[benjamn]

My point is that being hard to transpile is a significant barrier to pre-native adoption and usage in libraries intended for broad consumption, and leads to the ecosystem adopting compilation strategies that are difficult to deprecate later (such as Babel compiling ESM down to synchronous CommonJS).

This could be a companion proposal, though I believe it would speed standardization and early adoption of the top-level await proposal, which is why I think it's worth considering in this context.

[ljharb]

For what it's worth, if we adopted the restriction that TLA couldn't coexist in a module with export, I think that it could be trivially transpiled to CJS?

[littledan]

I agree with @benjamn that a transpilation story is important here, if it's possible. And with the semantics in this proposal, I believe transpilation is possible. TLA isn't part of CJS semantics, so a new convention would be needed.

@xtuc, @loganfsmyth and I were discussing this convention the other day: Maybe the convention could be, the module.exports of a module with TLA or which imports a module with TLA would be a Promise, and otherwise, it'd be the ordinary exports object.

One strategy would be for transpilers like Babel to refuse to process TLA and leave this for bundlers, which would adopt the convention internally, which would always know which modules to await and which to treat as synchronous objects. Another strategy would be to encourage this to be released into the ecosystem as transpiler output, with dynamic checks at each module usage to see whether, dynamically, it's a Promise. I'm leaning towards the first strategy, personally, to avoid complexity.

cc @lukastaegert @Sorka

[ljharb]

Babel is used for non-web targets as well (ie, non-bundler targets), so it's not tenable for babel to simply "refuse" to process a language feature. If it can't be handled without a bundler, then it hasn't been handled.

[littledan]

@ljharb Bundlers are used for non-web targets as well! Anyway, I believe this could be handled on a file-by-file basis, and described a strategy above. It's up to the Babel maintainers (of which I am not one) which they want to do here.

[ljharb]

Certainly, but not nearly as often, and it shouldn't be a requirement.

A file-by-file solution seems pretty important.

[littledan]

Do you have thoughts about the file-by-file approach I described above?

[ljharb]

The Promise thing? i don’t think that’s sufficient, since module.exports = Promise.resolve()isexport default Promise.resolve()` in most interops, so if a module using TLA requires different semantics a promise can’t be used to convey that.

[littledan]

Well, this gets a bit tautological. CJS doesn't have a way for modules to require awaiting a module, so something about its conventions would have to change. If this were a fatal problem, we should not have advanced to Stage 2. What I thought was nice about it was that it's a natural way to use these modules manually from CJS--just await or .then the return value of require, when it's async.

[ljharb]

I do agree we shouldn't have advanced to stage 2 at the time we did; unfortunately this interaction didn't become clear until just after the advancement.

Certainly having them export a Promise is much less bad than not having interop at all! but I think we should keep looking for a better interop story than that.

[nicolo-ribaudo]

What about something like Babel's exports.__esModule = true?

For example:

module.exports = async function () {
  // ...
}();

module.exports.__topLevelAwait = true;

[ljharb]

That seems fine as a transpiler-specific choice, but as a result __esModule has pervaded the ecosystem to a degree where in retrospect, it wasn't actually fine :-/ I'd prefer to avoid repeating that process.

[benjamn]

@nicolo-ribaudo In my original description, I think I suggested something very similar, except that the __topLevelAwait metadata would be expressed via the name of the exported async function (__evaluate__), which means it would work with pure ESM import and export syntax. I prefer this small restriction (targeting ESM rather than CommonJS) because it leaves room for any ESM compilation strategy, CommonJS or otherwise.

I'm sure we could come up with some convention for modules containing top-level await to export/expose this information to other modules, ideally using ESM import/export syntax rather than CommonJS… but unfortunately that's only half of the story.

I honestly have no idea what the transpilation story would be on the importing side. Like, for every single import declaration, you also import the namespace object, so you can dynamically test whether that module contains TLA (because it exported an __evaluate__ function). But then, in order to await that result, the importing module would also logically have to use top-level await (or whatever we transpile it to), so (after transpilation) virtually every module would have to be wrapped with an export function __evaluate__() { ... await ... }, with its exported declarations all hoisted to the outer scope, etc. etc.

Although I suppose this could work, it no longer feels like a useful way to explain the intended mental model of TLA using simpler concepts.

If there's no compelling, relatively simple TLA transpilation story, then I would be happy to close this issue, and we can keep working with maintainers of major JS module systems/compilers (as I know @littledan has been doing) to make sure they have some reasonable path forward, even if their implementations of TLA are highly specific to their module systems.

[codehag]

Closing this as per @benjamn's last comment. Please feel free to reopen if there are further concerns.