src: add support for TLA

PR-URL: nodejs#30370
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>

doc/api/errors.md

@@ -889,6 +889,11 @@ provided.
 Encoding provided to `TextDecoder()` API was not one of the
 [WHATWG Supported Encodings][].
 
+<a id="ERR_EVAL_ESM_CANNOT_PRINT"></a>
+### `ERR_EVAL_ESM_CANNOT_PRINT`
+
+`--print` cannot be used with ESM input.
+
 <a id="ERR_EXECUTION_ENVIRONMENT_NOT_AVAILABLE"></a>
 ### `ERR_EXECUTION_ENVIRONMENT_NOT_AVAILABLE`
 

doc/api/esm.md

@@ -36,8 +36,8 @@ initial input, or when referenced by `import` statements within ES module code:
 * Files ending in `.js` when the nearest parent `package.json` file contains a
   top-level field `"type"` with a value of `"module"`.
 
-* Strings passed in as an argument to `--eval` or `--print`, or piped to
-  `node` via `STDIN`, with the flag `--input-type=module`.
+* Strings passed in as an argument to `--eval`, or piped to `node` via `STDIN`,
+  with the flag `--input-type=module`.
 
 Node.js will treat as CommonJS all other forms of input, such as `.js` files
 where the nearest parent `package.json` file contains no top-level `"type"`
@@ -52,8 +52,8 @@ or when referenced by `import` statements within ES module code:
 * Files ending in `.js` when the nearest parent `package.json` file contains a
   top-level field `"type"` with a value of `"commonjs"`.
 
-* Strings passed in as an argument to `--eval` or `--print`, or piped to
-  `node` via `STDIN`, with the flag `--input-type=commonjs`.
+* Strings passed in as an argument to `--eval` or `--print`, or piped to `node`
+  via `STDIN`, with the flag `--input-type=commonjs`.
 
 ### `package.json` `"type"` field
 
@@ -159,9 +159,9 @@ package scope:
 
 ### `--input-type` flag
 
-Strings passed in as an argument to `--eval` or `--print` (or `-e` or `-p`), or
-piped to `node` via `STDIN`, will be treated as ES modules when the
-`--input-type=module` flag is set.
+Strings passed in as an argument to `--eval` (or `-e`), or piped to `node` via
+`STDIN`, will be treated as ES modules when the `--input-type=module` flag is
+set.
 
 ```sh
 node --input-type=module --eval "import { sep } from 'path'; console.log(sep);"
@@ -1076,6 +1076,32 @@ node --experimental-wasm-modules index.mjs
 
 would provide the exports interface for the instantiation of `module.wasm`.
 
+## Experimental Top-Level `await`
+
+When the `--experimental-top-level-await` flag is provided, `await` may be used
+in the top level (outside of async functions) within modules. This implements
+the [ECMAScript Top-Level `await` proposal][].
+
+Assuming an `a.mjs` with
+
+<!-- eslint-skip -->
+```js
+export const five = await Promise.resolve(5);
+```
+
+And a `b.mjs` with
+
+```js
+import { five } from './a.mjs';
+
+console.log(five); // Logs `5`
+```
+
+```bash
+node b.mjs # fails
+node --experimental-top-level-await b.mjs # works
+```
+
 ## Experimental Loaders
 
 **Note: This API is currently being redesigned and will still change.**
@@ -1779,6 +1805,7 @@ success!
 [Conditional Exports]: #esm_conditional_exports
 [Dynamic `import()`]: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports
 [ECMAScript-modules implementation]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md
+[ECMAScript Top-Level `await` proposal]: https://github.com/tc39/proposal-top-level-await/
 [ES Module Integration Proposal for Web Assembly]: https://github.com/webassembly/esm-integration
 [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md
 [Terminology]: #esm_terminology

doc/api/vm.md

@@ -402,7 +402,10 @@ support is planned.
 ```js
 const vm = require('vm');
 
-const contextifiedObject = vm.createContext({ secret: 42 });
+const contextifiedObject = vm.createContext({
+  secret: 42,
+  print: console.log,
+});
 
 (async () => {
   // Step 1
@@ -418,6 +421,7 @@ const contextifiedObject = vm.createContext({ secret: 42 });
   const bar = new vm.SourceTextModule(`
     import s from 'foo';
     s;
+    print(s);
   `, { context: contextifiedObject });
 
   // Step 2
@@ -460,16 +464,11 @@ const contextifiedObject = vm.createContext({ secret: 42 });
 
   // Step 3
   //
-  // Evaluate the Module. The evaluate() method returns a Promise with a single
-  // property "result" that contains the result of the very last statement
-  // executed in the Module. In the case of `bar`, it is `s;`, which refers to
-  // the default export of the `foo` module, the `secret` we set in the
-  // beginning to 42.
+  // Evaluate the Module. The evaluate() method returns a promise which will
+  // resolve after the module has finished evaluating.
 
-  const { result } = await bar.evaluate();
-
-  console.log(result);
   // Prints 42.
+  await bar.evaluate();
 })();
 ```
 
@@ -512,17 +511,14 @@ in the ECMAScript specification.
 
 Evaluate the module.
 
-This must be called after the module has been linked; otherwise it will
-throw an error. It could be called also when the module has already been
-evaluated, in which case it will do one of the following two things:
-
-* return `undefined` if the initial evaluation ended in success (`module.status`
-  is `'evaluated'`)
-* rethrow the same exception the initial evaluation threw if the initial
-  evaluation ended in an error (`module.status` is `'errored'`)
+This must be called after the module has been linked; otherwise it will reject.
+It could be called also when the module has already been evaluated, in which
+case it will either do nothing if the initial evaluation ended in success
+(`module.status` is `'evaluated'`) or it will re-throw the exception that the
+initial evaluation resulted in (`module.status` is `'errored'`).
 
 This method cannot be called while the module is being evaluated
-(`module.status` is `'evaluating'`) to prevent infinite recursion.
+(`module.status` is `'evaluating'`).
 
 Corresponds to the [Evaluate() concrete method][] field of [Cyclic Module
 Record][]s in the ECMAScript specification.

lib/internal/errors.js

@@ -805,6 +805,7 @@ E('ERR_ENCODING_INVALID_ENCODED_DATA', function(encoding, ret) {
 }, TypeError);
 E('ERR_ENCODING_NOT_SUPPORTED', 'The "%s" encoding is not supported',
   RangeError);
+E('ERR_EVAL_ESM_CANNOT_PRINT', '--print cannot be used with ESM input', Error);
 E('ERR_FALSY_VALUE_REJECTION', function(reason) {
   this.reason = reason;
   return 'Promise was rejected with falsy value';

lib/internal/modules/esm/loader.js

@@ -167,10 +167,9 @@ class Loader {
     };
     const job = new ModuleJob(this, url, evalInstance, false, false);
     this.moduleMap.set(url, job);
-    const { module, result } = await job.run();
+    const { module } = await job.run();
     return {
       namespace: module.getNamespace(),
-      result
     };
   }
 

lib/internal/modules/esm/module_job.js

@@ -31,21 +31,26 @@ class ModuleJob {
     this.isMain = isMain;
     this.inspectBrk = inspectBrk;
 
-    // This is a Promise<{ module, reflect }>, whose fields will be copied
-    // onto `this` by `link()` below once it has been resolved.
-    this.modulePromise = moduleProvider.call(loader, url, isMain);
     this.module = undefined;
+    // Expose the promise to the ModuleWrap directly for linking below.
+    // `this.module` is also filled in below.
+    this.modulePromise = moduleProvider.call(loader, url, isMain);
 
     // Wait for the ModuleWrap instance being linked with all dependencies.
     const link = async () => {
       this.module = await this.modulePromise;
       assert(this.module instanceof ModuleWrap);
 
+      // Explicitly keeping track of dependency jobs is needed in order
+      // to flatten out the dependency graph below in `_instantiate()`,
+      // so that circular dependencies can't cause a deadlock by two of
+      // these `link` callbacks depending on each other.
       const dependencyJobs = [];
       const promises = this.module.link(async (specifier) => {
         const jobPromise = this.loader.getModuleJob(specifier, url);
         dependencyJobs.push(jobPromise);
-        return (await jobPromise).modulePromise;
+        const job = await jobPromise;
+        return job.modulePromise;
       });
 
       if (promises !== undefined)
@@ -59,25 +64,20 @@ class ModuleJob {
     // 'unhandled rejection' warnings.
     this.linked.catch(noop);
 
-    // instantiated == deep dependency jobs wrappers instantiated,
-    // module wrapper instantiated
+    // instantiated == deep dependency jobs wrappers are instantiated,
+    // and module wrapper is instantiated.
     this.instantiated = undefined;
   }
 
-  async instantiate() {
-    if (!this.instantiated) {
-      return this.instantiated = this._instantiate();
+  instantiate() {
+    if (this.instantiated === undefined) {
+      this.instantiated = this._instantiate();
     }
-    await this.instantiated;
-    return this.module;
+    return this.instantiated;
   }
 
-  // This method instantiates the module associated with this job and its
-  // entire dependency graph, i.e. creates all the module namespaces and the
-  // exported/imported variables.
   async _instantiate() {
     const jobsInGraph = new SafeSet();
-
     const addJobsToDependencyGraph = async (moduleJob) => {
       if (jobsInGraph.has(moduleJob)) {
         return;
@@ -87,6 +87,7 @@ class ModuleJob {
       return PromiseAll(dependencyJobs.map(addJobsToDependencyGraph));
     };
     await addJobsToDependencyGraph(this);
+
     try {
       if (!hasPausedEntry && this.inspectBrk) {
         hasPausedEntry = true;
@@ -122,19 +123,20 @@ class ModuleJob {
       }
       throw e;
     }
+
     for (const dependencyJob of jobsInGraph) {
       // Calling `this.module.instantiate()` instantiates not only the
       // ModuleWrap in this module, but all modules in the graph.
       dependencyJob.instantiated = resolvedPromise;
     }
-    return this.module;
   }
 
   async run() {
-    const module = await this.instantiate();
+    await this.instantiate();
     const timeout = -1;
     const breakOnSigint = false;
-    return { module, result: module.evaluate(timeout, breakOnSigint) };
+    await this.module.evaluate(timeout, breakOnSigint);
+    return { module: this.module };
   }
 }
 ObjectSetPrototypeOf(ModuleJob.prototype, null);

lib/internal/process/execution.js

@@ -10,8 +10,9 @@ const path = require('path');
 const {
   codes: {
     ERR_INVALID_ARG_TYPE,
-    ERR_UNCAUGHT_EXCEPTION_CAPTURE_ALREADY_SET
-  }
+    ERR_UNCAUGHT_EXCEPTION_CAPTURE_ALREADY_SET,
+    ERR_EVAL_ESM_CANNOT_PRINT,
+  },
 } = require('internal/errors');
 
 const {
@@ -39,6 +40,9 @@ function tryGetCwd() {
 }
 
 function evalModule(source, print) {
+  if (print) {
+    throw new ERR_EVAL_ESM_CANNOT_PRINT();
+  }
   const { log, error } = require('internal/console/global');
   const { decorateErrorStack } = require('internal/util');
   const asyncESM = require('internal/process/esm_loader');

lib/internal/vm/module.js

@@ -219,8 +219,7 @@ class Module {
         'must be one of linked, evaluated, or errored'
       );
     }
-    const result = this[kWrap].evaluate(timeout, breakOnSigint);
-    return { __proto__: null, result };
+    await this[kWrap].evaluate(timeout, breakOnSigint);
   }
 
   [customInspectSymbol](depth, options) {

src/module_wrap.cc

@@ -375,7 +375,13 @@ void ModuleWrap::Evaluate(const FunctionCallbackInfo<Value>& args) {
     return;
   }
 
-  args.GetReturnValue().Set(result.ToLocalChecked());
+  // If TLA is enabled, `result` is the evaluation's promise.
+  // Otherwise, `result` is the last evaluated value of the module,
+  // which could be a promise, which would result in it being incorrectly
+  // unwrapped when the higher level code awaits the evaluation.
+  if (env->isolate_data()->options()->experimental_top_level_await) {
+    args.GetReturnValue().Set(result.ToLocalChecked());
+  }
 }
 
 void ModuleWrap::GetNamespace(const FunctionCallbackInfo<Value>& args) {
@@ -387,13 +393,17 @@ void ModuleWrap::GetNamespace(const FunctionCallbackInfo<Value>& args) {
   Local<Module> module = obj->module_.Get(isolate);
 
   switch (module->GetStatus()) {
-    default:
+    case v8::Module::Status::kUninstantiated:
+    case v8::Module::Status::kInstantiating:
       return env->ThrowError(
-          "cannot get namespace, Module has not been instantiated");
+          "cannot get namespace, module has not been instantiated");
     case v8::Module::Status::kInstantiated:
     case v8::Module::Status::kEvaluating:
     case v8::Module::Status::kEvaluated:
+    case v8::Module::Status::kErrored:
       break;
+    default:
+      UNREACHABLE();
   }
 
   Local<Value> result = module->GetModuleNamespace();
@@ -616,19 +626,19 @@ MaybeLocal<Value> ModuleWrap::SyntheticModuleEvaluationStepsCallback(
   TryCatchScope try_catch(env);
   Local<Function> synthetic_evaluation_steps =
       obj->synthetic_evaluation_steps_.Get(isolate);
+  obj->synthetic_evaluation_steps_.Reset();
   MaybeLocal<Value> ret = synthetic_evaluation_steps->Call(context,
       obj->object(), 0, nullptr);
   if (ret.IsEmpty()) {
     CHECK(try_catch.HasCaught());
   }
-  obj->synthetic_evaluation_steps_.Reset();
   if (try_catch.HasCaught() && !try_catch.HasTerminated()) {
     CHECK(!try_catch.Message().IsEmpty());
     CHECK(!try_catch.Exception().IsEmpty());
     try_catch.ReThrow();
     return MaybeLocal<Value>();
   }
-  return ret;
+  return Undefined(isolate);
 }
 
 void ModuleWrap::SetSyntheticExport(const FunctionCallbackInfo<Value>& args) {