Define host environment interactions for modules

[littledan]

Whether or not #223 lands (I'm a fan!), Realms have the ability to import modules through import(). I think we should have a somewhat clear idea of what this means in some environments, which has some amount of buy-in from those environments, to get to Stage 3.

Maybe this doesn't need host environment spec text to block Stage 3, but at least thought through and written down. There may be implications for host hooks needed. It'd be good to avoid repeating the experience with modules and Promises where the layering and full spec text took time to nail down after standardization.

I don't know much about many JS environments, so it'd be great to have some more help from the committee.

[littledan]

Here's a very rough outline of how I could picture Realms working for HTML:

• Each Realm has its own module map
  • HTML's module map caches modules by absolute URL
  • Each Realm will have an independent copy of all modules which get imported--modules are never shared between Realms
  • After calling CreateRealm(), a host hook needs to run to allow the host to create the module map appropriately. (Edit: This is Layering: Host hook to initialize new Realms #230)
• Each Realm has a base URL
  • HTML uses the base URL to absolute-ize relative URLs, turning a module specifier into a cache key.
  • Edit: The base URL of the new Realm is the base URL of the surrounding document (rationale)
  • The base URL can be undefined, which would trigger an error if a relative URL is used as a module specifier
  • I'd suggest that the base URL could be set by an option to the Realm constructor.
    • Strawperson concrete API: new Realm({ base: "https://foo.com" }). base would be a string in the options bag which the host environment can use as it wants. The host hook called on Realm initialization would have the chance to post-process/validate this (e.g., in HTML's case, parse it as a URL relative to the enclosing document's base URL, and throw if it doesn't validate).
    • Note that giving this base URL doesn't give any particular power, it just helps you out with making the URL absolute. Also, this notion of "base path" seems to make sense in environments that use local files as well (not sure if other environments might be using specifiers in other ways that don't match)
• Modules in Realms would be fetched similarly to modules outside of Realms
  • The fetch request's client would be the enclosing environment settings object of the whole document, which implies inheriting certain settings and policies, a connection pool, etc.
• For interaction with the import map proposal (I'm not suggesting we block Stage 3 for this part):
  • Edit: Realms would simply inherit their import map from the surrounding environment.
  • Maybe there's a way to pass arbitrary host-interpreted values to the Realm constructor, e.g., the import map? Or, HTML could add an API to the internals of the new Realm to create an import map, using the Realm creation hook, if that's not discouraged by the Realm proposal champions?

cc @domenic @annevk @kmiller68 @hiroshige-g @codehag @syg

[littledan]

Note, I don't consider passing up a base URL and import maps to be strong requirements; we could say that module specifiers always have to be absolute URLs in Realms. However, we should have the host hook in place to set up the Realm, which HTML can use to initialize an empty module map and set the base URL to null.

[ljharb]

No, we can’t say that, because module specifiers are not URLs in the language - only in specific hosts.

[littledan]

@ljharb This is clear. What I'm suggesting is that we could pass a string up to the host, which it could interpret how it wants--I think many hosts have a concept of a "base directory" that modules are resolved relative to, so it will be useful in a broadly similar way. Just like module specifiers themselves are not specified in their interpretation, this "base" string would not be either.

[littledan]

Based on the comment at https://github.com/tc39/proposal-realms/pull/223/files#r385853699 , my understanding is that the champion group's proposal for HTML is that the base URL of Realms created with the Realm API be the base URL of the surrounding document, and that there not be an API to set the base URL (maybe this would come from a future compartments/evaluator API). This seems like a fine design to me.

[erights]

Hi @littledan yes. This would be a job for host hooks, and the champions of both proposals agreed that all host hooks that would naturally apply to either Realm or Compartment be migrated into the SES / Compartment proposal, which is proceeding on that basis.

[littledan]

@erights OK, I'm glad to hear that the champion group is comfortable for Realms to, initially in HTML, inherit these settings from their enclosing environment. This makes the integration much more straightforward, and makes me comfortable punting on things like base and other aspects of module evaluation. Hopefully we can draft an HTML integration PR soon which will clarify things further.

[littledan]

@wycats and I discussed this, and he was a bit skeptical of each Realm having its own, entirely separate, module graph. He says it doesn't match their current usage of the Node vm module. We may want to think this through some more.

[littledan]

See HTML proposal at whatwg/html#5339 , implementing the plan at #225 (comment) .

To follow up on @wycats's concern, we may consider that module graphs with more sharing could be provided by the Evaluator/Compartment proposal's import trap. Short of that, it's hard for me to think of a concrete API that would allow more control over sharing. Personally, I like the current division of the Realm proposal vs the future Evaluator/Compartment proposal, but I can understand if some people are eager for the more complete featureset.

[leobalter]

I believe the EcmaScript part is well defined and @littledan is doing a good follow up in whatwg/html#5339 as mentioned and agreed in the plenaries. I believe it's best to sunset this discussion in order to focus the following discussion in the integration part only. Otherwise, we can reopen this to track status.