Compartment support for dynamic import and importMetaHook

[kriskowal]

[update April 20, 2022]:

Description

SES shim should support the features of dynamic import (import) and import metadata (import.meta). These must be implemented in a manner that is host neutral and afford the possibility of lazy, on-demand initialization of import.meta for use cases like import.meta.resolve that would be prohibitively expensive if constructed for modules that do not utter import.meta in their source.

Design

In @endo/static-module-record we employ a source to source transformation that produces a “first-party static module record” that ses understands how to load, link, and initialize when returned through the importHook. In SES, the module-instance.js module is responsible for the runtime component for evaluating modules. We must extend this calling convention to support dynamic import and import metadata.

Also, because static module records are committed to archives and stored, the calling convention for modules generated by older versions of @endo/static-module-record must be preserved.

I propose that we do this by rewriting dynamic import(specifier) and import.meta in the module-to-program transformation to forms like $h_import() and $h_import_meta, including the ZWJ so the hidden symbol censorship still applies. The module-instance.js linker must pass suitable values for these into the generated module functor. The dynamic import function must be a closure that applies the resolveHook relative to the current moduleSpecifier.

Since creating $h_import_meta and $h_import are avoidable if they were not uttered by the source, or compiled by an earlier version of @endo/static-module-record, the newer StaticModuleRecord constructor should add new properties to the record, like __usesImport and __usesImportMeta, as an indication for whether these should be made.

$h_import_meta must be an object with a null prototype.

The importHook (to be called loadHook in a future revision #420) will need to be able to return a “module record envelope” that has an optional meta property. The record.meta property is an object to copy to the $h_import_meta object, either with the semantics of Object.assign($h_import_meta, record.meta) or Object.defineProperties($h_import_meta, Object.getOwnPropertyDescriptors(record.meta)), on which we need consensus among the champions of the TC39 Compartment Proposal.

We would, either in this change or a future change to be factored out of this issue, also implement a importMetaHook. The importMetaHook would receive $h_import_meta and have an opportunity to augment it and freeze it before the object is revealed to guest code. The module machinery must only call the hook if import.meta is uttered in the module code. The host function must not be called on the stack of guest code.

We will need consensus among the Compartments champions for the time that the importMetaHook function must be called, but I’m guessing that the most obvious answer would be immediately before initializing the module, without any opportunity for other events to be interleaved. So, an exception thrown in importMetaHook would be observed as the permanent revocation of the promise for the initialization of the module, poisoning any module that depends on it.

Security Considerations

The $h_import internal should not be utterable by guest code except as a function call. Access to the function object should not be possible. It should be frozen by module-instance.js on the off-chance that it is accidentally exposed to guest code.

The $h_import_meta object must be freezable by either the host or the guest. The design of importMetaHook affords this possibility. I suspect that TC39 will want the import.meta to be mutable unless the host or guest elects to freeze it, but on this we should also get consensus from the Compartment Proposal champions and eventually reviewers.

---

[from #303, May 11, 2020]:

Code running in a compartment needs access to a dynamic import expression that closes over the compartment’s module loader. This should present import.meta according to whatever is returned by the compartment’s importMetaHook(specifier).

---

*[original, from May 4, 2020]:

Currently, transform-module does introduce an import endowment (need to verify). There is a note in the code that we need to add a facility for hardening the import function. The existing code also arranges for an import.meta and import.meta.url and we need to instead facilitate weaving an importMetaHook through the Compartment constructor options to make this behavior customizable.