HostResolveImportedModule is underspecified WRT ES Source Modules

[allenwb]

HostResolveImportedModule is intentionally underspecified to allow for the possibility of importing additional implementation-defined kinds of modules or alternative mechanisms (EG, dynamic modules) for introducing modules into a running program.

However, the current specification is does not adequately state what is required in the normal case where the imported module is a Source Text Module as defined by ECMA-262. Nowhere is it stated in ECMA-262 how such module must be processed.

This problem can be corrected by adding an additional item to the bullet list of requirements in 15.2.1.18. The new requirement should be:

• If the referencingModule, specifier pair identifies the source text of a standard ECMAScript Module, the returned Module Record must be a SourceTextModuleRecord created as if by the ParseModule abstract operation passed the source text and the current realm as arguments.

This item should probably be the second bullet of the requirements list.

Note that most readers probably assumed this requirement but itfor the spec. to be complete it should be made explicit.

[domenic]

Hmm, isn't the proposed text kind of a tautology? That is, isn't the definition of "When does referencingModule, specifier identity the soruce text of a standard ECMAScript Module?" answered by precisely "when HostResolveImportedModule returns a SourceTextModuleRecord, given those inputs"?

Edit: I guess the current realm requirement might be novel.

[domenic]

Hmm, isn't the proposed text kind of a tautology? That is, isn't the definition of "When does referencingModule, specifier identity the soruce text of a standard ECMAScript Module?" answered by precisely "when HostResolveImportedModule returns a SourceTextModuleRecord, given those inputs"?

Edit: I guess the current realm requirement might be novel.

[allenwb]

@domenic It is implementation (host) dependent how it is determined that a specifier pair corresponds to a source text that needs to be processed as a standard ECMAScript /Module/. (and that could be explicitly stated). But once that determination is made, then ParseModule needs to be used to create the SourceTextModuleRecord as it is that abstract operations that specify parsing and early error detection and how the required fields of the SourceTextModuleRecord are populated for subsequent use in other parts of the spec. Up to now, we didn't actually say that.

The spec. currently requires that all ModuleRecords have a [[Realm]] field which is why ParseModule takes a realm parameter. I said "current realm" because I assumed that the current realm is the realm of the referencing module. (that probably needs to be verified from the spec.) Whether the current realm usage in the module specification is complete or accurate is probably a separable issue.

BTW, I it should be obvious that I also think the HTML spec needs to explicitly invoke ParseModule in its HostResolveImportedModule (or indirectly in something that it calls)

[allenwb]

@domenic It is implementation (host) dependent how it is determined that a specifier pair corresponds to a source text that needs to be processed as a standard ECMAScript /Module/. (and that could be explicitly stated). But once that determination is made, then ParseModule needs to be used to create the SourceTextModuleRecord as it is that abstract operations that specify parsing and early error detection and how the required fields of the SourceTextModuleRecord are populated for subsequent use in other parts of the spec. Up to now, we didn't actually say that.

The spec. currently requires that all ModuleRecords have a [[Realm]] field which is why ParseModule takes a realm parameter. I said "current realm" because I assumed that the current realm is the realm of the referencing module. (that probably needs to be verified from the spec.) Whether the current realm usage in the module specification is complete or accurate is probably a separable issue.

BTW, I it should be obvious that I also think the HTML spec needs to explicitly invoke ParseModule in its HostResolveImportedModule (or indirectly in something that it calls)

[domenic]

In HTML, ParseModule is called ahead of time; HostResolveImportedModule just does a lookup into a table of already-parsed URLs -> Source Text Module Records.

I guess I am just not seeing how a host environment could violate the proposed bullet. The host environment's spec could always just redefine it away by saying "oh yeah, that pair did not identify the source text of a standard ECMAScript module."

[domenic]

In HTML, ParseModule is called ahead of time; HostResolveImportedModule just does a lookup into a table of already-parsed URLs -> Source Text Module Records.

I guess I am just not seeing how a host environment could violate the proposed bullet. The host environment's spec could always just redefine it away by saying "oh yeah, that pair did not identify the source text of a standard ECMAScript module."

[allenwb]

It might be better to think about ParseModule is as if it was named CreateSourceTextModuleRecordFromSourceText. ParseModule is the only place in ECMA-262 that defines how that happens. If a spec. wants to say that it is producing a well-formed SourceTextModuleRecord it either needs to explicitly invoke ParseModule or it needs to duplicate all of the essential steps carried out by ParseModule.

Performing the work of ParseModule ahead of time is already explicitly allowed. See the note at the end of that subclause. The ParseModule abstract operation actually does not have to be directly performed within HostResolveImportedModule, but it has to happen somewhere. I don't see this happening (explicitly) anywhere within the HTML spec for <script type=module>. My guess is that the most natural place to do this is part of the internal module script tree fetching procedure. Or, perhaps you say something about this in the HTML definition of module record. Or perhaps some where else. But it really needs to happen somewhere so that the proper ECMA-262 mandated processing of a module source text into a SourceTextModuleRecord is required to happen.

The ES spec. is general enough to allow a host to essentially do anything with an import specifier. It should even be possible to do things like import shared libraries or C header files. But regarding your last point, the purpose of the proposed bullet is to state what the ES spec. requires in the situation when the import specifier pair actually does identify the source text of a standard ES module. Without it, there is nothing in the ECMA-262 that says that the standard semantics of importing an ES module depends upon HostResolveImportdModule returning a SourceTextModuleRecord or exactly how such a SourceTextModuleRecord is constructed from the source text identified by the specifier.

[allenwb]

It might be better to think about ParseModule is as if it was named CreateSourceTextModuleRecordFromSourceText. ParseModule is the only place in ECMA-262 that defines how that happens. If a spec. wants to say that it is producing a well-formed SourceTextModuleRecord it either needs to explicitly invoke ParseModule or it needs to duplicate all of the essential steps carried out by ParseModule.

Performing the work of ParseModule ahead of time is already explicitly allowed. See the note at the end of that subclause. The ParseModule abstract operation actually does not have to be directly performed within HostResolveImportedModule, but it has to happen somewhere. I don't see this happening (explicitly) anywhere within the HTML spec for <script type=module>. My guess is that the most natural place to do this is part of the internal module script tree fetching procedure. Or, perhaps you say something about this in the HTML definition of module record. Or perhaps some where else. But it really needs to happen somewhere so that the proper ECMA-262 mandated processing of a module source text into a SourceTextModuleRecord is required to happen.

The ES spec. is general enough to allow a host to essentially do anything with an import specifier. It should even be possible to do things like import shared libraries or C header files. But regarding your last point, the purpose of the proposed bullet is to state what the ES spec. requires in the situation when the import specifier pair actually does identify the source text of a standard ES module. Without it, there is nothing in the ECMA-262 that says that the standard semantics of importing an ES module depends upon HostResolveImportdModule returning a SourceTextModuleRecord or exactly how such a SourceTextModuleRecord is constructed from the source text identified by the specifier.

[domenic]

If you Ctrl+F for ParseModule in https://html.spec.whatwg.org/ you will find it in https://html.spec.whatwg.org/#creating-a-module-script

[domenic]

If you Ctrl+F for ParseModule in https://html.spec.whatwg.org/ you will find it in https://html.spec.whatwg.org/#creating-a-module-script

[domenic]

Regarding whether this bullet adds anything of value to the spec beyond a tautology around the definition of "identifies the source text of a standard ECMAScript Module", I still didn't really see that point being addressed, but I guess we can just agree to disagree; it's not a big deal if things that are IMO redundant end up added to the spec.

[domenic]

Regarding whether this bullet adds anything of value to the spec beyond a tautology around the definition of "identifies the source text of a standard ECMAScript Module", I still didn't really see that point being addressed, but I guess we can just agree to disagree; it's not a big deal if things that are IMO redundant end up added to the spec.

[allenwb]

Excellent, creating a module script is what I was looking for. I should have CTRL-F'ed rather than just reading spec text. I'm now happy WRT the HTML spec. it's already doing what I'm suggesting should be required.

Your tautology doesn't say anything about requiring the actions of ParseModule to occur in that case. That may be a reasonable assumption, but a good spec. should eliminate the need to make such assumptions. It should instead, state exactly what is required. That is what the additional bullet does.

[allenwb]

Excellent, creating a module script is what I was looking for. I should have CTRL-F'ed rather than just reading spec text. I'm now happy WRT the HTML spec. it's already doing what I'm suggesting should be required.

Your tautology doesn't say anything about requiring the actions of ParseModule to occur in that case. That may be a reasonable assumption, but a good spec. should eliminate the need to make such assumptions. It should instead, state exactly what is required. That is what the additional bullet does.