Top-level await (Stage 3)

[hzoo]

Champion: @MylesBorins https://twitter.com/MylesBorins/status/999005736914087936

Presented as Stage 1 at Jan 2018
Moved to Stage 2 at May 2018

Spec repo: https://github.com/tc39/proposal-top-level-await

[MylesBorins]

Let me know if I can help with this at all.

I'm not entirely sure how this could be implemented, wrapping the body in an async function would enable TLA, but you couldn't export any symbols. Perhaps exported symbols could be proxies that will block when trying to get a value that is not available?

[loganfsmyth]

I'm on the fence about whether it would be worth Babel trying to support a transform for this. I don't see a way to compile this while not also compiling ES6 module syntax to CommonJS, since there's no way to block execution of the module graph. If we are converting to CJS, we could set module.exports to a promise with a custom flag like we do for __esModule I think, but that has huge issues too. We can't know until we call require if something is async or not, meaning Babel's compiled output would always have to expect to be async. To handle that, it means Babel's CJS output wouldn't be able to have synchronous results.

[nicolo-ribaudo]

I think that this should be handled by tools like webpack.

[loganfsmyth]

Yeah totally, I'm mostly thinking for cases where people still use Babel for modules, like Node. Not sure how esm would handle this either.

[hzoo]

I think that this should be handled by tools like webpack.

That sounds reasonable too

We already have the parser option allowAwaitOutsideFunction for this case, done by @jdalton actually 😛for REPL and I'm assuming esm babel/babel#7637

[guybedford]

I've just implemented some basic top-level await support in SystemJS 0.21.4 so that the system module format output can potentially support this via the execute property being an async function or returning a promise:

console.log('sync exec');
await x();
console.log('async exec');

->

System.register([], function (_export) {
  return {
    setters: [],
    execute: function () {
      console.log('sync exec');
      return Promise.resolve(x())
      .then(function () {
        console.log('async exec');
      });
    }
  }
});

The simplest transformation to support this would be to just have execute: async function () {} then containing the exact top-level await module body, although a transformation like the above for non-async support might be nice too.

I've put together a basic implementation for Rollup at rollup/rollup#2235, would be happy to help further here if I can.

[loganfsmyth]

@guybedford I'm curious, have the option of using generators to flatten the structure of register come up? I know in the past we've had trouble transforming SystemJS because it is hard to hoist function declarations things outside of execute while allowing then to access things lexically scoped to the module body, without simply dropping support for lexical declarations. It seems like using a generator would allow both of those issues to be resolved. For instance, something like

export * from "foo";

console.log(val);

let val = 4;

export function foo() {
  console.log(val); 
}

should in theory end up as

System.register(["foo"], function (_export, _context) {
  "use strict";

  let val;
  function foo() {
    console.log(val);
  }

  _export("foo", foo);

  return {
    setters: [function (_foo) {
      var _exportObj = {};

      for (var _key in _foo) {
        if (_key !== "default" && _key !== "__esModule") _exportObj[_key] = _foo[_key];
      }

      _export(_exportObj);
    }],
    execute: function () {

      console.log(val);

      val = 4;
    }
  };
});

but then the TDZ behavior has to be lost.

You could imagine something like

System.register(["foo"], function* (_export, _context) {
  "use strict";

  _export(0, function (_foo) {
    var _exportObj = {};

    for (var _key in _foo) {
      if (_key !== "default" && _key !== "__esModule") _exportObj[_key] = _foo[_key];
    }

    _export(_exportObj);
  });
  _export("foo", foo);

  console.log(val);

  let val = 4;
  function foo() {
    console.log(val);
  }
});

allowing preservation of module semantics much more accurately, which also allowing for easy yielding of promises for top-level await handling.

[arv]

Using a generator is a great idea. At least if you output generators. If you transform generators to ES5 code then the generated code would be pretty ugly.

[guybedford]

@loganfsmyth that is a really great idea, I've created systemjs/systemjs#1826 to track this further. Let's do it!

[guybedford]

There is one issue here though, and that is that the async function approach then becomes tricky for supporting top-level await... as we have the hoisting problem for the async part of execution without async generators being a thing...

[guybedford]

From a TDZ perspective also note that the format isn't designed to catch all invalid runtime scenarios (eg you can import named exports that don't exist). Rather it is designed to support the valid working runtime scenarios that work in ES modules, as a backwards compatibility workflow. So TDZ throwing being out of scope has kind of always been the assumption.

[benjamn]

The vast majority of Babel users who use ES module syntax currently compile it down to CommonJS. If there's no reasonable compilation strategy for top-level await that targets CommonJS (and I don't see how there could be, given that CommonJS require implicitly mandates synchronous module execution), I strongly believe Babel should not bother compiling TLA syntax.

Bundling tools (which ultimately decide how the runtime module system is implemented, and whether it can handle async modules) can handle top-level await expressions however they choose. Babel should leave top-level await alone.

The very fact that you're talking about implementing TLA in a SystemJS-specific way highlights the folly of handling top-level await in Babel. In order to get this right, you need to know how your runtime module system works, and that has never been something that Babel could assume.

To put it another way, what if you're targeting Node? Does the generator function strategy make any sense in that context? If it only makes sense for SystemJS… well, then that's all we've accomplished. Babel as a whole should not advertise support for top-level await unless it has a strategy for every environment.

We don't even know whether top-level await will be restricted to modules without exports (the optional constraint) yet. Do we really want developers to start writing code that depends on an unfinished specification? Is that really a constraint on future TLA design that we're willing to accept?

Perhaps exported symbols could be proxies that will block when trying to get a value that is not available?

Proxies are another notoriously impossible-to-transpile ECMAScript feature, so I don't think this strategy makes sense for a transpiler like Babel.

[benjamn]

I opened this issue back in January in hopes of sparking discussion about transpilation strategies for top-level await: tc39/proposal-top-level-await#2

In short, if we could agree on a general mechanism for defining asynchronous modules in ESM, without relying on top-level await as a primitive, then it would be dramatically easier for tools like Babel to compile top-level await down to something that works in all the major module runtimes.

[jamiebuilds]

I don't think under any circumstances that Babel should implement top-level await. It's a great feature, but not one that Babel should attempt to support because of the interoperability implications.

[loganfsmyth]

I think supporting it SystemJS could be feasible since it's a bit more flexible format-wise, but otherwise I totally agree. I don't see how we could hope to handle top-level await with Babel's per-file approach to compilation, and I think it's reasonable to leave that responsibility to other tooling.

[jamiebuilds]

Some speculation:

• The biggest use case (imo) for TLA is Node scripts and CLIs. (await import() is a distant second)
• Node will probably ship TLA within the next year (@MylesBorins can correct me if I'm wrong)
• Lots of people are already using async-await directly in Node, using it with Babel is actually kinda rare due to how difficult it is to setup today.
• Most people will just use versions of Node that support TLAs if they want it.

As such, I don't really feel a big need for this feature to be supported. At least not enough that justifies complicating things a lot.

[littledan]

@jamiebuilds I think it's worth it to get these things in Babel for early prototyping and feedback before Node would ship it, even if it might not be used so much in production. If you want to discuss the motivation of the feature, I'd recommend doing so in the proposal's repository.

[jdalton]

I wouldn't get too comfortable with TLA as it is in stage 2 (variant B alone).
Several folks have issues with it. As is, and if left unchanged, it will likely not advance.

[littledan]

Stage 2 proposals should not be interpreted as stable. Decorators has changed significantly while at Stage 2, and other Stage 2 proposals have been dropped entirely (e.g., Object.observe). In my opinion, it's still valuable for Babel to implement early-stage proposals, to get feedback from JS developers and guide the proposals' evolution. I'd recommend that developers avoid proposals which are Stage 2 and below for code which needs to be maintained over time.

[jdalton]

Yep. That said though, Babel is an open source project with limited resources. Since this proposal currently has significant complications, and will likely require changing, their resources may be better spent on other proposals is all.

[MylesBorins]

Can you speak to the specific complications? None of those were brought up at the last meeting We are continuing to work on the spec text and examining how inplementation would work. Of people have concerns they should be opening issues on the proposal.

…

On Sun, Jun 24, 2018, 12:10 PM John-David Dalton ***@***.***> wrote: Yep. That said though, Babel is an open source project with limited resources. Since this proposal currently has significant complications, and will likely require changing, their resources may be better spent on other proposals is all. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#44 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecV0AFAreBu_s9qU6h5vWu5TSNb871ks5t_7nzgaJpZM4UJRjP> .

[littledan]

@jdalton I don't think resources are zero-sum like that. New proposals often motivate people to get involved.

[jdalton]

@MylesBorins

None of those were brought up at the last meeting

I believe some were unclear at the last meeting on exactly which parts were making it to stage 2 and how those parts affect modules and the ecosystem as a whole. Since the meeting, while working with Babel and the Node Module WG, concerns have been raised which will need to be addressed.

Of people have concerns they should be opening issues on the
proposal.

Yep, probably.

[jamiebuilds]

@littledan

I think it's worth it to get these things in Babel for early prototyping and feedback before Node would ship it, even if it might not be used so much in production.

I agree, but we have in the past decided not to support features in Babel because of the problems they can cause.

Example: Proxies could have been supported in Babel, but it would've required wrapping every get/set operation in your entire program including in code that might not be compiled by Babel or by the same version of Babel.

TLA is similar in that it requires wrapping every module in your program with something systemjs-y, even modules that aren't compiled by Babel or by the same version of Babel.

If you want to discuss the motivation of the feature, I'd recommend doing so in the proposal's repository.

I think the proposal is already well motivated, I'm very keen to use it. Just not via a Babel transform.

[jamiebuilds]

it requires wrapping every module in your program

I should clarify that I mean "module" outside of it's meaning in 262 to include commonjs modules and such.

[jridgewell]

Full app proposals like this are outside of Babel's wheelhouse.

I talked with @MylesBorins about this at JSConf, or maybe the last meeting. This would require large refactoring to do well. That, or Babel outputs an export async function then(), which is what the committee is trying to avoid.

[Kingwl]

stage-3 now🙋🏻‍♂️