New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add <script type="module"> and module resolution/fetching/evaluation #443

Merged
merged 1 commit into from Jan 20, 2016

Conversation

@domenic
Member

domenic commented Dec 22, 2015

This adds support for <script type="module"> for loading JavaScript modules, as well as all the infrastructure necessary for resolving, fetching, parsing, and evaluating module graphs rooted at such <script type="module">s.

This was spurred on by a request over in whatwg/loader#83 (comment). This should take care of whatwg/loader#83, whatwg/loader#84, and whatwg/loader#82. There is not much overlap with the current loader spec, which is primarily concerned with reflective modules and the author-customizable loading pipeline. This patch is much more about changing HTML's processing model for the script element, and integrating module execution the same way HTML currently integrates script execution. It deals with questions like "when do modules execute relative to HTML parsing" or "how does module fetching/parsing/evaluation integrate with the event loop".

This needs substantial review! Preferably from implementers! It is a complicated topic and I am sure I got some details wrong, in addition to the discussion points noted below.

Here are the decisions that I incorporated while speccing this, which might not be immediately obvious. Some are bolded to indicate they need discussion/help/bikeshedding.

  • Always use the UTF-8 decoder, ignoring the charset="" attribute or the Content-Type header.
  • Disallow module responses which don't have a JavaScript MIME type for their Content-Type header. This is basically building in X-Content-Type-Options: nosniff behavior by default for module scripts.
  • Always use the "cors" fetch mode, with the crossorigin="" attribute controlling the credentials mode: omitted => "omit", "anonymous" => "same-origin", "use-credentials" => "include".
  • Modules are memoized per realm.
  • Use <script defer>-like semantics: do not execute until parsing is finished, and execute in order. The async attribute can be used to opt in to execute as soon as possible (but still no blocking).
  • For module resolution:
    • Allow absolute URLs and anything starting with "./" or "../" or "/", with the latter set being then interpreted as relative URLs. Everything else (e.g. bare specifiers like "jquery") fails for now, and no automatic ".js" is appended.
    • We throw a TypeError for module specifiers that are not parseable as absolute URLs and do not start with "./" or "../". (Honestly I could see arguments for any of EvalError, ReferenceError, SyntaxError, or URIError [sic]. Maybe we should repurpose EvalError.)
  • We use the page base URL as the base URL for resolving relative import specifiers for inline module scripts, and the response's URL (not the request's!) as the base URL for resolving relative import specifiers in external module scripts or imported modules in the tree.
  • In contrast, we dedupe fetches based on the request's URL.

To make review easier, I am hosting a compiled version: singlepage, multipage. Sections of note (links go to singlepage):

  • Main <script> section: largely contains authoring guidance updates. New diagram is also available, although the dropbox view doesn't show it inline since we are using absolute URLs.
  • <script> processing model section
    • Prepare a script is significantly different, and asynchronously creates "the script's script", instead of letting that be done by "execute a script block".
    • Execute a script block is simpler, and mostly delegates to the "run a classic script" or "run a module script" algorithms, surrounded by DOM-related stuff.
    • There is a new concept of "the script is ready" which generalizes the current spec's "The task that the networking task source places on the task queue once fetching has completed must do x", to allow it to work for module trees additionally.
  • The scripting section gets a few subsections to do the heavy lifting:
    • Definitions contains definitions for classic scripts and module scripts as separate types, and adds the module map to all environment settings objects.
    • Fetching scripts now includes algorithms that "asynchronously complete". Let me know what you think of this formulation. I need the algorithms to exit quickly, then run a bunch of steps in parallel or in reaction to a task, but then call back to their caller with a result.
    • Creating scripts has a pretty straightforward "create a module script" and "create a classic script"
    • Calling scripts has the old "run a classic script" algorithm but also the new "run a module script" algorithm. Fetching is completed by that point.
    • Integration with the JavaScript module system contains HostResolveImportedModule that is just a lookup in the module map manipulated inside "fetching scripts."
  • Script settings for browsing contexts was not changed but was moved under browsing contexts instead of under scripting.

/cc @whatwg/loader @bterlson @ajklein @Constellation

@caridy

This comment has been minimized.

Show comment
Hide comment
@caridy

caridy Dec 23, 2015

Notes from our conversation via IRC, plus some extra thoughts:

  • somehow, this has to connect to the default loader per realm.
  • it seems that <script type="module"> does not go through the loader for resolving or fetching but import statements always do (this separation is already covered by this patch).
  • module's map should be the loader.registry where the url is the key, and this can be set once the url is resolved, so other import statements can resolve to the same registry entry (mostly for circular dependencies).
  • when storing new entries in loader.registry we need to create a new entry to wire up the loader and the key, (e.g.: let entry = new ModuleStatus(<loader>, <key:url>);) and adding it to the registry, (e.g.: <loader>.registry.set(<key:url>, entry);).
  • all operations on the registry instance are sync.
  • once the source is fetched, and the source text module record is created, the entry can be resolved, (e.g.: entry.resolve('instantiate', <source-text-module-record>););
  • errors can also be tracked (e.g.: entry.reject('fetch', <error-fetching>); or entry.reject('instantiate', <error-creating-new-source-text-module-record>);), and this could happen before or after inserting it into the registry.
  • once the source text module record is created and resolved into the entry (as described above), the final step is to put the evaluation phase in motion by calling entry.load("ready"); (we might add an alternative way to do so), this defers the rest of the pipeline to the loader, returning a promise of the evaluation that resolves to an exotic namespace object, which, for the purpose of this patch, is irrelevant.

caridy commented Dec 23, 2015

Notes from our conversation via IRC, plus some extra thoughts:

  • somehow, this has to connect to the default loader per realm.
  • it seems that <script type="module"> does not go through the loader for resolving or fetching but import statements always do (this separation is already covered by this patch).
  • module's map should be the loader.registry where the url is the key, and this can be set once the url is resolved, so other import statements can resolve to the same registry entry (mostly for circular dependencies).
  • when storing new entries in loader.registry we need to create a new entry to wire up the loader and the key, (e.g.: let entry = new ModuleStatus(<loader>, <key:url>);) and adding it to the registry, (e.g.: <loader>.registry.set(<key:url>, entry);).
  • all operations on the registry instance are sync.
  • once the source is fetched, and the source text module record is created, the entry can be resolved, (e.g.: entry.resolve('instantiate', <source-text-module-record>););
  • errors can also be tracked (e.g.: entry.reject('fetch', <error-fetching>); or entry.reject('instantiate', <error-creating-new-source-text-module-record>);), and this could happen before or after inserting it into the registry.
  • once the source text module record is created and resolved into the entry (as described above), the final step is to put the evaluation phase in motion by calling entry.load("ready"); (we might add an alternative way to do so), this defers the rest of the pipeline to the loader, returning a promise of the evaluation that resolves to an exotic namespace object, which, for the purpose of this patch, is irrelevant.
@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 23, 2015

Member

And, most importantly:

  • All of the above can be taken care of in the loader spec, or as follow-up PRs here to improve loader spec integration once the loader spec is implementable, and do not block reviewing or merging this PR to complete the "milestone 0" work in the loader roadmap and give implementers and authors something they can work with. :)
Member

domenic commented Dec 23, 2015

And, most importantly:

  • All of the above can be taken care of in the loader spec, or as follow-up PRs here to improve loader spec integration once the loader spec is implementable, and do not block reviewing or merging this PR to complete the "milestone 0" work in the loader roadmap and give implementers and authors something they can work with. :)
@DigiTec

This comment has been minimized.

Show comment
Hide comment
@DigiTec

DigiTec Dec 28, 2015

I got hung up on type=module. Is it a good idea to bind a loading behavior to the type which today is used by so many tools to specify the content type itself? This is kind of leaning overly heavy on a bunch of toolchains doing the right thing.

Maybe use a new Boolean attribute?

<script module ...>

DigiTec commented Dec 28, 2015

I got hung up on type=module. Is it a good idea to bind a loading behavior to the type which today is used by so many tools to specify the content type itself? This is kind of leaning overly heavy on a bunch of toolchains doing the right thing.

Maybe use a new Boolean attribute?

<script module ...>
@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 28, 2015

Member

We need to use the type attribute so that downlevel browsers do not interpret the contents as scripts.

type="" has never had anything to do with the content type, in reality. It has a number of values which signify "JavaScript", only one of which is the JS mimetype, and any other value signifies "do not execute this". This patch adds a third value, "module", with its own semantics.

Member

domenic commented Dec 28, 2015

We need to use the type attribute so that downlevel browsers do not interpret the contents as scripts.

type="" has never had anything to do with the content type, in reality. It has a number of values which signify "JavaScript", only one of which is the JS mimetype, and any other value signifies "do not execute this". This patch adds a third value, "module", with its own semantics.

@DigiTec

This comment has been minimized.

Show comment
Hide comment
@DigiTec

DigiTec Dec 28, 2015

Mozilla and Microsoft both use this attribute though and so do tools. Some of which validate it or rewrite it.

Legacy MS browsers even hit the registry to look up custom scripting languages. So adding a value here is a bit dangerous to older browsers.

I understand the desire to prevent parsing but I think it may have worse side effects than it is saving from.

DigiTec commented Dec 28, 2015

Mozilla and Microsoft both use this attribute though and so do tools. Some of which validate it or rewrite it.

Legacy MS browsers even hit the registry to look up custom scripting languages. So adding a value here is a bit dangerous to older browsers.

I understand the desire to prevent parsing but I think it may have worse side effects than it is saving from.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 28, 2015

Member

I don't think we have any other choice, unless we want to disallow inline module scripts entirely.

I don't think any danger posed by script type="module" could possibly be greater than the danger posed by the many existing pages with type="handlebars". If legacy browsers/Mozilla/Microsoft are already coping with the web at large, which contains many such instances, they will be able to cope with the web using type="module".

Member

domenic commented Dec 28, 2015

I don't think we have any other choice, unless we want to disallow inline module scripts entirely.

I don't think any danger posed by script type="module" could possibly be greater than the danger posed by the many existing pages with type="handlebars". If legacy browsers/Mozilla/Microsoft are already coping with the web at large, which contains many such instances, they will be able to cope with the web using type="module".

@DigiTec

This comment has been minimized.

Show comment
Hide comment
@DigiTec

DigiTec Dec 28, 2015

There are many usages of type, for instance, all of the WebGL samples generally use a script block with a type set to something which identifies them as either a vertex or fragment shader.

And at least in IE/Edge our mechanism for type is to parse out a text/ application/ prefix, then treat the remainder as a language. For Edge we'll look this up against known languages, which all bind to Chakra. For legacy IE we'll hit the registry and see if we can find an IActiveScript*

If we are bent on using the type for this, should we not identify the module language itself? Saying module seems to imply the one and only, forever, etc... It doesn't imply room for versioning or a language change of the syntax in the future. If this is the ES6 modules proposal, then should we consider a type that at least identifies the version and parsing semantics? es6-module for instance?

You made one other statement about disallowing inline module scripts. Is that a terrible idea? inline scripts are pretty bad in general. They cause stalls in the pre-parser, etc... This can be mitigated with various defer/async attributes, and we are starting to imply those by saying type=module per your prosal, but this now means that the concept of modules has to be baked into a lot of places, including the pre-parser (at least in our case, and I realize this is a technical detail that may not apply to other implementations). I haven't thought through all of the conditions of using inline scripts but since they don't have a download context, it may be non-trivial for us to implement the remainder of the algorithms. Will have to spend some time thinking through that once I'm back in the office, if Travis hasn't already jumped on it by then.

DigiTec commented Dec 28, 2015

There are many usages of type, for instance, all of the WebGL samples generally use a script block with a type set to something which identifies them as either a vertex or fragment shader.

And at least in IE/Edge our mechanism for type is to parse out a text/ application/ prefix, then treat the remainder as a language. For Edge we'll look this up against known languages, which all bind to Chakra. For legacy IE we'll hit the registry and see if we can find an IActiveScript*

If we are bent on using the type for this, should we not identify the module language itself? Saying module seems to imply the one and only, forever, etc... It doesn't imply room for versioning or a language change of the syntax in the future. If this is the ES6 modules proposal, then should we consider a type that at least identifies the version and parsing semantics? es6-module for instance?

You made one other statement about disallowing inline module scripts. Is that a terrible idea? inline scripts are pretty bad in general. They cause stalls in the pre-parser, etc... This can be mitigated with various defer/async attributes, and we are starting to imply those by saying type=module per your prosal, but this now means that the concept of modules has to be baked into a lot of places, including the pre-parser (at least in our case, and I realize this is a technical detail that may not apply to other implementations). I haven't thought through all of the conditions of using inline scripts but since they don't have a download context, it may be non-trivial for us to implement the remainder of the algorithms. Will have to spend some time thinking through that once I'm back in the office, if Travis hasn't already jumped on it by then.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 28, 2015

Member

There are many usages of type, for instance, all of the WebGL samples generally use a script block with a type set to something which identifies them as either a vertex or fragment shader.

Sure. Nothing about this proposal prevents that from continuing. As long as you don't use one of the reserved values, you can just treat that as a data-storage attribute like data-whatever. That's not changed by introducing one new reserved value.

If we are bent on using the type for this, should we not identify the module language itself? Saying module seems to imply the one and only, forever, etc... It doesn't imply room for versioning or a language change of the syntax in the future. If this is the ES6 modules proposal, then should we consider a type that at least identifies the version and parsing semantics? es6-module for instance?

Versioning on the web is a classic antipattern, so there's no reason to include that here. Every version must be backward-compatible, so there's no need to specify the version. And pretending that "es6" is a coherent conceptual entity is not good; in reality there are varying levels of support in various engines, including parts of ES6 and parts of ES7 and so on. Even some ES5 is not implemented.

We could name it "js-module" if you really think that the word "module" is too generic. But I don't think we should do that. Similarly to how <script> by itself means "JavaScript legacy script", <script type="module"> should mean "JavaScript module script". And those extra few characters do hurt ergonomics slightly. The vast majority of developers will just curse us for making them type them every time---"of course it's a JS module; what other type of module is there?"

If we need in the future to add new ways of processing script texts, we can invent new reserved values for them (e.g. <script type="wasm">). Hopefully those new script texts won't go down JavaScript's path and will support modules from the beginning, so there won't be any need to specify wasm vs. wasm-module or similar.

You made one other statement about disallowing inline module scripts. Is that a terrible idea?

I was at first sympathetic to this idea, but upon reflection I think it'd be surprising to developers and hurt rapid experimentation. E.g. you could not write modules in a jsbin-like environment. Or I guess you could, using data: URLs in src, but that's just punishing authors.

I haven't thought through all of the conditions of using inline scripts but since they don't have a download context, it may be non-trivial for us to implement the remainder of the algorithms.

The spec largely follows the existing spec for non-module inline scripts, so it would be surprising if this causes you problems. I'd love to hear about them if so, so that we can fix them!!

Member

domenic commented Dec 28, 2015

There are many usages of type, for instance, all of the WebGL samples generally use a script block with a type set to something which identifies them as either a vertex or fragment shader.

Sure. Nothing about this proposal prevents that from continuing. As long as you don't use one of the reserved values, you can just treat that as a data-storage attribute like data-whatever. That's not changed by introducing one new reserved value.

If we are bent on using the type for this, should we not identify the module language itself? Saying module seems to imply the one and only, forever, etc... It doesn't imply room for versioning or a language change of the syntax in the future. If this is the ES6 modules proposal, then should we consider a type that at least identifies the version and parsing semantics? es6-module for instance?

Versioning on the web is a classic antipattern, so there's no reason to include that here. Every version must be backward-compatible, so there's no need to specify the version. And pretending that "es6" is a coherent conceptual entity is not good; in reality there are varying levels of support in various engines, including parts of ES6 and parts of ES7 and so on. Even some ES5 is not implemented.

We could name it "js-module" if you really think that the word "module" is too generic. But I don't think we should do that. Similarly to how <script> by itself means "JavaScript legacy script", <script type="module"> should mean "JavaScript module script". And those extra few characters do hurt ergonomics slightly. The vast majority of developers will just curse us for making them type them every time---"of course it's a JS module; what other type of module is there?"

If we need in the future to add new ways of processing script texts, we can invent new reserved values for them (e.g. <script type="wasm">). Hopefully those new script texts won't go down JavaScript's path and will support modules from the beginning, so there won't be any need to specify wasm vs. wasm-module or similar.

You made one other statement about disallowing inline module scripts. Is that a terrible idea?

I was at first sympathetic to this idea, but upon reflection I think it'd be surprising to developers and hurt rapid experimentation. E.g. you could not write modules in a jsbin-like environment. Or I guess you could, using data: URLs in src, but that's just punishing authors.

I haven't thought through all of the conditions of using inline scripts but since they don't have a download context, it may be non-trivial for us to implement the remainder of the algorithms.

The spec largely follows the existing spec for non-module inline scripts, so it would be surprising if this causes you problems. I'd love to hear about them if so, so that we can fix them!!

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 28, 2015

I am hoping some discussion of how this proposal fits in to how developers will use modules will be helpful.

It's my understanding that <script type="module"> as proposed here allows devs to use import statements inside of JS embedded in HTML files. This creates a 'root' module, parsed as an ES module, that loads, parses, and registers all of the imports, the runs the body of the script to, eg initialize a web app.

The resulting root module is a bit odd: it's anonymous (correct?). What -- if any -- relationship exists between multiple <script type="module"> modules?

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

I have seen requests for named modules created with <script type="module"> . There might be arguments for such a feature in future, but I think we should avoid it initially.

Similarly I think support for defer and async should be delayed. We have existing solutions to support the use cases covered by these features.

In fact, the feature unique to this proposal is blocking rendering and synchronously loaded code. To say this another way, <script type="module" src="./foo.js"> is not equivalent to what we can do absent this proposal:

<script>
System.import('./foo.js');
</script>

because the latter is async.

johnjbarton commented Dec 28, 2015

I am hoping some discussion of how this proposal fits in to how developers will use modules will be helpful.

It's my understanding that <script type="module"> as proposed here allows devs to use import statements inside of JS embedded in HTML files. This creates a 'root' module, parsed as an ES module, that loads, parses, and registers all of the imports, the runs the body of the script to, eg initialize a web app.

The resulting root module is a bit odd: it's anonymous (correct?). What -- if any -- relationship exists between multiple <script type="module"> modules?

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

I have seen requests for named modules created with <script type="module"> . There might be arguments for such a feature in future, but I think we should avoid it initially.

Similarly I think support for defer and async should be delayed. We have existing solutions to support the use cases covered by these features.

In fact, the feature unique to this proposal is blocking rendering and synchronously loaded code. To say this another way, <script type="module" src="./foo.js"> is not equivalent to what we can do absent this proposal:

<script>
System.import('./foo.js');
</script>

because the latter is async.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 28, 2015

Member

The resulting root module is a bit odd: it's anonymous (correct?).

In the sense that it cannot be imported, yes.

What -- if any -- relationship exists between multiple <script type="module"> modules?

None. Each creates a separate tree of module dependencies.

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

That's a great point. We should be able to make that fail pretty easily. I'll work on that.

I have seen requests for named modules created with <script type="module">. There might be arguments for such a feature in future, but I think we should avoid it initially.

Agreed. It could be useful but it's not necessary to get things off the ground.

Similarly I think support for defer and async should be delayed. We have existing solutions to support the use cases covered by these features.

defer behavior is the default. We could consider adding async-style behavior in the future to allow execution before parsing of the entire page is finished. But yeah, future stuff.

In fact, the feature unique to this proposal is blocking rendering and synchronously loaded code.

That's not accurate. Nothing in this proposal blocks rendering or synchronously loads code.

System.import('./foo.js');

There is no spec for what this would do in a browser, so I don't know quite what you're referring to. It's certainly not something you can do absent this proposal.

Member

domenic commented Dec 28, 2015

The resulting root module is a bit odd: it's anonymous (correct?).

In the sense that it cannot be imported, yes.

What -- if any -- relationship exists between multiple <script type="module"> modules?

None. Each creates a separate tree of module dependencies.

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

That's a great point. We should be able to make that fail pretty easily. I'll work on that.

I have seen requests for named modules created with <script type="module">. There might be arguments for such a feature in future, but I think we should avoid it initially.

Agreed. It could be useful but it's not necessary to get things off the ground.

Similarly I think support for defer and async should be delayed. We have existing solutions to support the use cases covered by these features.

defer behavior is the default. We could consider adding async-style behavior in the future to allow execution before parsing of the entire page is finished. But yeah, future stuff.

In fact, the feature unique to this proposal is blocking rendering and synchronously loaded code.

That's not accurate. Nothing in this proposal blocks rendering or synchronously loads code.

System.import('./foo.js');

There is no spec for what this would do in a browser, so I don't know quite what you're referring to. It's certainly not something you can do absent this proposal.

@caridy

This comment has been minimized.

Show comment
Hide comment
@caridy

caridy Dec 28, 2015

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

That's a great point. We should be able to make that fail pretty easily. I'll work on that.

We could do this by relying on the source text module record returned by ParseModule(), which contains the internal slot [[LocalExportEntries]], [[StarExportEntries]] and [[IndirectExportEntries]]. But I will argue against this. We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

caridy commented Dec 28, 2015

As I don't see any exception for the keyword export I suppose it would be allowed, but it seems like it would be confusing to devs: the statement has no meaning since there is no mechanism to import this anonymous module (correct?).

That's a great point. We should be able to make that fail pretty easily. I'll work on that.

We could do this by relying on the source text module record returned by ParseModule(), which contains the internal slot [[LocalExportEntries]], [[StarExportEntries]] and [[IndirectExportEntries]]. But I will argue against this. We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 28, 2015

Member

We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

Hmm, now I am not sure. Could you find any existing Node module or similar that can be used in this dual manner? In practice I cannot see when this would be a good idea...

Member

domenic commented Dec 28, 2015

We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

Hmm, now I am not sure. Could you find any existing Node module or similar that can be used in this dual manner? In practice I cannot see when this would be a good idea...

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 28, 2015

defer behavior is the default.

Ok I see that in your proposal now. That's too bad. It makes complexity the default behavior. No reason to argue on this point, I know it's a lost cause.

System.import('./foo.js');

There is no spec for what this would do in a browser,

I was referring to
http://whatwg.github.io/loader/#system-loader-instance and http://whatwg.github.io/loader/#loader-import

johnjbarton commented Dec 28, 2015

defer behavior is the default.

Ok I see that in your proposal now. That's too bad. It makes complexity the default behavior. No reason to argue on this point, I know it's a lost cause.

System.import('./foo.js');

There is no spec for what this would do in a browser,

I was referring to
http://whatwg.github.io/loader/#system-loader-instance and http://whatwg.github.io/loader/#loader-import

@caridy

This comment has been minimized.

Show comment
Hide comment
@caridy

caridy Dec 29, 2015

We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

Hmm, now I am not sure. Could you find any existing Node module or similar that can be used in this dual manner? In practice I cannot see when this would be a good idea...

I have seen this before with express apps, while they create an express app and listen for incoming traffic, and exporting the app instance for test or for other more generic aggregation.

Aside from that, I think about this as a reflective form of import "foo.js";, which is perfectly valid whether or not foo.js exports something.

caridy commented Dec 29, 2015

We can expect a module that executes some initialization, and exports some stuff in case they are used in a composited way, and that is ok.

Hmm, now I am not sure. Could you find any existing Node module or similar that can be used in this dual manner? In practice I cannot see when this would be a good idea...

I have seen this before with express apps, while they create an express app and listen for incoming traffic, and exporting the app instance for test or for other more generic aggregation.

Aside from that, I think about this as a reflective form of import "foo.js";, which is perfectly valid whether or not foo.js exports something.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

I guess that is convincing. OK, I'll leave it as-is.

Member

domenic commented Dec 29, 2015

I guess that is convincing. OK, I'll leave it as-is.

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 29, 2015

Would you consider restricting <script type="module"> to appear after <body>? That would ensure that the implicit defer matched the declaration order. The resulting error message could be a great way for new devs to learn that normal script blocks.

johnjbarton commented Dec 29, 2015

Would you consider restricting <script type="module"> to appear after <body>? That would ensure that the implicit defer matched the declaration order. The resulting error message could be a great way for new devs to learn that normal script blocks.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

Would you consider restricting <script type="module"> to appear after <body>?

How? You mean in a validator or something?

That would ensure that the implicit defer matched the declaration order.

What does that mean? Are you also proposing doing the same for <script defer>?

The resulting error message could be a great way for new devs to learn that normal script blocks.

Why would an error message for <script type="module"> be a good teaching tool for something about legacy scripts?

Member

domenic commented Dec 29, 2015

Would you consider restricting <script type="module"> to appear after <body>?

How? You mean in a validator or something?

That would ensure that the implicit defer matched the declaration order.

What does that mean? Are you also proposing doing the same for <script defer>?

The resulting error message could be a great way for new devs to learn that normal script blocks.

Why would an error message for <script type="module"> be a good teaching tool for something about legacy scripts?

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 29, 2015

Overall I believe your goal with the implicit defer setting is to avoid blocking rendering the way HTML's <script> tag blocks. I am suggesting that you can achieve that goal without "secretly" changing the default behavior of the <script> tag as your proposal requires. We would achieve your goal with less mystery for developers and in a way more consistent with HTML's original design.

Would you consider restricting <script type="module"> to appear after <body>?
How? You mean in a validator or something?

Detecting renderable elements in the HTML parser after the module tag does not seem very difficult. The approach most consistent with HTML's accept-all parsing strategy is for the error to result in the script tag emitting console error and failing to execute.

That would ensure that the implicit defer matched the declaration order.
What does that mean?

As you know, defer on the <script> tag means

This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed. 

The clearest way to express this execution order in a language like HTML is to place the tag at the bottom of the document. Using an attribute to express order was lame; making that lameness a default in some cases but not others does not create less lameness.

Are you also proposing doing the same for <script defer>?

I am proposing that the <script> tag's defer attribute be unaffected by your proposal. Specifically, the <script> tag would not have a different default value for defer when type="module". Instead, the effect you wish to create would be achieved by a different and simpler (for developers) solution.

The resulting error message could be a great way for new devs to learn that normal script blocks.

Why would an error message for <script type="module"> be a good teaching tool for something about legacy scripts?

You and I both expect this feature to be widely adopted. Many naive developers will place <script type="module"> in their document the same way they would place legacy script tags. If the result is an error such as "Module tags cannot block HTML rendering, they must appear after all renderable elements", they will be forced to place the new form at the bottom. As a side effect, some will learn that script blocks HTML and even more will simply adopt the habit of placing script tags at the bottom. Both of these secondary benefits further your goal.

johnjbarton commented Dec 29, 2015

Overall I believe your goal with the implicit defer setting is to avoid blocking rendering the way HTML's <script> tag blocks. I am suggesting that you can achieve that goal without "secretly" changing the default behavior of the <script> tag as your proposal requires. We would achieve your goal with less mystery for developers and in a way more consistent with HTML's original design.

Would you consider restricting <script type="module"> to appear after <body>?
How? You mean in a validator or something?

Detecting renderable elements in the HTML parser after the module tag does not seem very difficult. The approach most consistent with HTML's accept-all parsing strategy is for the error to result in the script tag emitting console error and failing to execute.

That would ensure that the implicit defer matched the declaration order.
What does that mean?

As you know, defer on the <script> tag means

This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed. 

The clearest way to express this execution order in a language like HTML is to place the tag at the bottom of the document. Using an attribute to express order was lame; making that lameness a default in some cases but not others does not create less lameness.

Are you also proposing doing the same for <script defer>?

I am proposing that the <script> tag's defer attribute be unaffected by your proposal. Specifically, the <script> tag would not have a different default value for defer when type="module". Instead, the effect you wish to create would be achieved by a different and simpler (for developers) solution.

The resulting error message could be a great way for new devs to learn that normal script blocks.

Why would an error message for <script type="module"> be a good teaching tool for something about legacy scripts?

You and I both expect this feature to be widely adopted. Many naive developers will place <script type="module"> in their document the same way they would place legacy script tags. If the result is an error such as "Module tags cannot block HTML rendering, they must appear after all renderable elements", they will be forced to place the new form at the bottom. As a side effect, some will learn that script blocks HTML and even more will simply adopt the habit of placing script tags at the bottom. Both of these secondary benefits further your goal.

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 29, 2015

Sorry one more quick comment: <script defer type="module"> would be valid anywhere and its semantics would be consistent with <script defer>. Only the default case would be forced to the end of the document.

johnjbarton commented Dec 29, 2015

Sorry one more quick comment: <script defer type="module"> would be valid anywhere and its semantics would be consistent with <script defer>. Only the default case would be forced to the end of the document.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

I don't see any benefits in making the source document order impact execution order, so I think we have a fundamental disagreement.

Member

domenic commented Dec 29, 2015

I don't see any benefits in making the source document order impact execution order, so I think we have a fundamental disagreement.

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 29, 2015

Source document order affects execution order now, so we can't disagree about that.

The only issue here is whether the script tag default for defer should change. I don't think it should.

johnjbarton commented Dec 29, 2015

Source document order affects execution order now, so we can't disagree about that.

The only issue here is whether the script tag default for defer should change. I don't think it should.

@meandmycode

This comment has been minimized.

Show comment
Hide comment
@meandmycode

meandmycode Dec 29, 2015

Just to play naive a bit here but since this is the first I've seen of a topic I and I'm sure many others have been waiting on, this doesn't restrict developers from doing <script src="es6.js"></script> correct? I know you mentioned avoiding older browsers from trying to run them but.. who cares?

They'll just bail out on the first import or export they find, why is this any different from any other new introduced syntax like arrow functions, get/sets, destructuring or future things like async?

Developers already have to "cope" with that today and it'll always be a minor pain point.. (but we transpile and like any other web feature, watch adoption rates of capable browsers) so what in the future when the import/export spec is upgraded or we add more syntax, add another type for that? it seems like this is going down a route of becoming yet another weird web thing in a few years time.

Thanks,

meandmycode commented Dec 29, 2015

Just to play naive a bit here but since this is the first I've seen of a topic I and I'm sure many others have been waiting on, this doesn't restrict developers from doing <script src="es6.js"></script> correct? I know you mentioned avoiding older browsers from trying to run them but.. who cares?

They'll just bail out on the first import or export they find, why is this any different from any other new introduced syntax like arrow functions, get/sets, destructuring or future things like async?

Developers already have to "cope" with that today and it'll always be a minor pain point.. (but we transpile and like any other web feature, watch adoption rates of capable browsers) so what in the future when the import/export spec is upgraded or we add more syntax, add another type for that? it seems like this is going down a route of becoming yet another weird web thing in a few years time.

Thanks,

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

this doesn't restrict developers from doing

Developers can do that, but the resulting file will be parsed as a legacy script, instead of a module script. Thus there will be a syntax error for the first import or export statement, and sloppy mode will be the implicit default, and top-level declarations will create global variables, a few other minor differences. I'm not sure if that's what you meant by "bail out". You can still use new ES features implemented in the browser you're programming against though.

why is this any different from any other new introduced syntax

Because it's not just new syntax: it's drastically new semantics, per the above list. It also has a greatly changed execution model, as it's impossible to execute before dependencies are loaded, so you can't execute in a blocking fashion as is done with legacy scripts.

in the future when the import/export spec is upgraded or we add more syntax, add another type for that

There are no plans to add any new script types to JavaScript, although indeed other languages like wasm might want to do so.

Member

domenic commented Dec 29, 2015

this doesn't restrict developers from doing

Developers can do that, but the resulting file will be parsed as a legacy script, instead of a module script. Thus there will be a syntax error for the first import or export statement, and sloppy mode will be the implicit default, and top-level declarations will create global variables, a few other minor differences. I'm not sure if that's what you meant by "bail out". You can still use new ES features implemented in the browser you're programming against though.

why is this any different from any other new introduced syntax

Because it's not just new syntax: it's drastically new semantics, per the above list. It also has a greatly changed execution model, as it's impossible to execute before dependencies are loaded, so you can't execute in a blocking fashion as is done with legacy scripts.

in the future when the import/export spec is upgraded or we add more syntax, add another type for that

There are no plans to add any new script types to JavaScript, although indeed other languages like wasm might want to do so.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

@johnjbarton it sounds like what you want to do is develop a validator or lint tool for your project's HTML that imposes your preferred ordering of scripts-after-elements, since you think it's important that source order reflect execution order. I don't plan to enforce that style in this spec.

Member

domenic commented Dec 29, 2015

@johnjbarton it sounds like what you want to do is develop a validator or lint tool for your project's HTML that imposes your preferred ordering of scripts-after-elements, since you think it's important that source order reflect execution order. I don't plan to enforce that style in this spec.

@johnjbarton

This comment has been minimized.

Show comment
Hide comment
@johnjbarton

johnjbarton Dec 29, 2015

Please read what I wrote. I am not discussing style. I am objecting -- in as nice a way as I can ;-) -- to your proposal to change the default behavior of the <script> tag. I did this by proposing an alternative with the same effect. You can disagree without characterizing my suggestion as some style thing.

johnjbarton commented Dec 29, 2015

Please read what I wrote. I am not discussing style. I am objecting -- in as nice a way as I can ;-) -- to your proposal to change the default behavior of the <script> tag. I did this by proposing an alternative with the same effect. You can disagree without characterizing my suggestion as some style thing.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

I am not changing the default behavior of the script tag. I am introducing a new type of script, module scripts, in addition to the two existing ones---legacy script, which executes in a blocking fashion (potentially modified by boolean defer/async modifiers), and opaque script, which does not execute (and does not have boolean modifiers). Module scripts have their own separate third semantics, of executing after parsing is complete and all their dependencies are loaded and executed. (They also do not have boolean modifiers.)

Member

domenic commented Dec 29, 2015

I am not changing the default behavior of the script tag. I am introducing a new type of script, module scripts, in addition to the two existing ones---legacy script, which executes in a blocking fashion (potentially modified by boolean defer/async modifiers), and opaque script, which does not execute (and does not have boolean modifiers). Module scripts have their own separate third semantics, of executing after parsing is complete and all their dependencies are loaded and executed. (They also do not have boolean modifiers.)

@meandmycode

This comment has been minimized.

Show comment
Hide comment
@meandmycode

meandmycode Dec 29, 2015

Thanks for the prompt reply @domenic, much appreciated, by bail out I meant older browsers would error rather than wrongly interpreting a modern script.

I agree that there are big changes to how execution works when import and export are introduced, but I didn't think that would mean developers needed to tip off the browser when that was happening, from my understanding, import and export had to be top level and the first expressions within the source, as such I expect that script loading logic would change so the parser would only determine the execution model once it reaches the first expression, if it encountered an import (or export if that matters) then it would follow the newer rules of execution, otherwise the legacy method.

I'm sure this has been discussed to death elsewhere and there is sound logic as to why, but just wanted to try understand.

meandmycode commented Dec 29, 2015

Thanks for the prompt reply @domenic, much appreciated, by bail out I meant older browsers would error rather than wrongly interpreting a modern script.

I agree that there are big changes to how execution works when import and export are introduced, but I didn't think that would mean developers needed to tip off the browser when that was happening, from my understanding, import and export had to be top level and the first expressions within the source, as such I expect that script loading logic would change so the parser would only determine the execution model once it reaches the first expression, if it encountered an import (or export if that matters) then it would follow the newer rules of execution, otherwise the legacy method.

I'm sure this has been discussed to death elsewhere and there is sound logic as to why, but just wanted to try understand.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

Yeah, developers definitely need to tip off the browser as to which type of script they are using. import and export do not need to be the first expressions in a module, and in fact do not need to appear in modules at all; you can still have a module that is used for its side effects, but gets the other benefits of modules like automatic strict mode and no global variables from top-level declarations.

Member

domenic commented Dec 29, 2015

Yeah, developers definitely need to tip off the browser as to which type of script they are using. import and export do not need to be the first expressions in a module, and in fact do not need to appear in modules at all; you can still have a module that is used for its side effects, but gets the other benefits of modules like automatic strict mode and no global variables from top-level declarations.

@meandmycode

This comment has been minimized.

Show comment
Hide comment
@meandmycode

meandmycode Dec 29, 2015

Ah, interesting, I suppose some transpilers are enforcing import and export to be first for simplicity so had assumed this was also the spec. I think it's a shame that developers (especially end user developers) would need to do this hinting, has there been discussion about having something similar to use strict that let the source declare itself as a module, this at least puts the burden on the authoring developer vs the consuming developer.

Again, appreciate you sharing your insights here.

meandmycode commented Dec 29, 2015

Ah, interesting, I suppose some transpilers are enforcing import and export to be first for simplicity so had assumed this was also the spec. I think it's a shame that developers (especially end user developers) would need to do this hinting, has there been discussion about having something similar to use strict that let the source declare itself as a module, this at least puts the burden on the authoring developer vs the consuming developer.

Again, appreciate you sharing your insights here.

@Constellation

This comment has been minimized.

Show comment
Hide comment
@Constellation

Constellation Dec 29, 2015

Member

Great start :D! <script type="module"> is very important step to push module system into the browser. I'm really positive to this change :)

Always use the UTF-8 decoder, ignoring the charset="" attribute or the Content-Type header.

Is it better rejecting interpretation when the content-type's charset does not match to the UTF-8?
e.g. Content-Type: application/javascript; charset=euc-jp

We do a URL parse instead of a basic URL parse, i.e., we allow blob URLs theoretically. (I am not sure about this and happy to change it.)

We reuse src attribute for the module path. Personally, I'm OK with that.

And I have several questions.

  1. Is there any request in the spec that attempts to resolve dependent modules? I think the user agent need to perform requestSatisfy operation in the loader spec before executing the module in deferred phase.
  2. Can we request fetch, translate, instantiate and satisfy operation before reaching deferred phase? That means, "Can we perform fetching of the whole module graph without waiting defer phase?"
Member

Constellation commented Dec 29, 2015

Great start :D! <script type="module"> is very important step to push module system into the browser. I'm really positive to this change :)

Always use the UTF-8 decoder, ignoring the charset="" attribute or the Content-Type header.

Is it better rejecting interpretation when the content-type's charset does not match to the UTF-8?
e.g. Content-Type: application/javascript; charset=euc-jp

We do a URL parse instead of a basic URL parse, i.e., we allow blob URLs theoretically. (I am not sure about this and happy to change it.)

We reuse src attribute for the module path. Personally, I'm OK with that.

And I have several questions.

  1. Is there any request in the spec that attempts to resolve dependent modules? I think the user agent need to perform requestSatisfy operation in the loader spec before executing the module in deferred phase.
  2. Can we request fetch, translate, instantiate and satisfy operation before reaching deferred phase? That means, "Can we perform fetching of the whole module graph without waiting defer phase?"
@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 29, 2015

Member

Yay, thanks for your review and your questions!

Is it better rejecting interpretation when the content-type's charset does not match to the UTF-8?

I modeled this after web workers, which ignore other encodings. I'm open to changing it but for now I'm assuming that we designed web workers this way on purpose. Maybe this is something browsers should give guidance on though in the console if they detect a mismatch... I'm happy to add a note to that effect, both here and in web workers.

We reuse src attribute for the module path.

It kind of depends on what you mean by "module path". The idea is that src is always URLs, and never goes through any custom resolution logic. (Neither the simple custom resolution logic in this patch, or any more complicated custom resolution logic if author-customizable loaders are implemented.) That way src gets treated the same as any other URL-containing HTML attribute.

Is there any request in the spec that attempts to resolve dependent modules? I think the user agent need to perform requestSatisfy operation in the loader spec before executing the module in deferred phase.

Yes, this happens automatically. In step 13 of "run a module script", there is a recursive call to HostResolveImportedModule for each dependency; HostResolveImportedModule is what does the resolution and fetching.

Evaluation only happens after all calls to HostResolveImportedModule, recursively, have returned; that is step 14.2 of "run a module script", after step 13 completes.

The loader spec is actually a separate spec that for now does not interact with this one. In the future if the loader spec gets its browser part finished it may be possible to phrase parts of this spec in terms of its phases. But for now this spec is self-contained and is meant to be implemented independent of the loader spec, since the loader spec doesn't deal with browsers. So there is no need to think about requestSatisfy or similar.

Can we request fetch, translate, instantiate and satisfy operation before reaching deferred phase? That means, "Can we perform fetching of the whole module graph without waiting defer phase?"

Again, these operations do not have meaning in the current spec. But I think I understand what you are getting at.

You can fetch the modules at any time. This is covered by the following note, which already exists in the spec:

For performance reasons, user agents may start fetching the script (as defined above) as soon as the src attribute is set, instead, in the hope that the element will be inserted into the document (and that the crossorigin attribute won't change value in the meantime). Either way, once the element is inserted into the document, the load must have started as described in this step. If the UA performs such prefetching, but the element is never inserted in the document, or the src attribute is dynamically changed, or the crossorigin attribute is dynamically changed, then the user agent will not execute the script so obtained, and the fetching process will have been effectively wasted.

Maybe I should make it clear that this also applies to import statements.

You can also do anything unobservable, as long as your observable consequences are the same as specified. In particular, you could do parsing or instantiation or any such thing, as long as you delay any error reporting until the time specified in the spec (which is defer time). The ES spec contains a note about some of this:

An implementation may parse module source text and analyze it for Early Error conditions prior to the evaluation of ParseModule for that module source text. However, the reporting of any errors must be deferred until the point where this specification actually performs ParseModule upon that source text.

Maybe I should add a specific note to the HTML spec along these lines too.

You cannot do evaluation until the whole graph (rooted at a particular <script type="module">) completes instantiation. So evaluation must necessarily be done at defer timing.

Member

domenic commented Dec 29, 2015

Yay, thanks for your review and your questions!

Is it better rejecting interpretation when the content-type's charset does not match to the UTF-8?

I modeled this after web workers, which ignore other encodings. I'm open to changing it but for now I'm assuming that we designed web workers this way on purpose. Maybe this is something browsers should give guidance on though in the console if they detect a mismatch... I'm happy to add a note to that effect, both here and in web workers.

We reuse src attribute for the module path.

It kind of depends on what you mean by "module path". The idea is that src is always URLs, and never goes through any custom resolution logic. (Neither the simple custom resolution logic in this patch, or any more complicated custom resolution logic if author-customizable loaders are implemented.) That way src gets treated the same as any other URL-containing HTML attribute.

Is there any request in the spec that attempts to resolve dependent modules? I think the user agent need to perform requestSatisfy operation in the loader spec before executing the module in deferred phase.

Yes, this happens automatically. In step 13 of "run a module script", there is a recursive call to HostResolveImportedModule for each dependency; HostResolveImportedModule is what does the resolution and fetching.

Evaluation only happens after all calls to HostResolveImportedModule, recursively, have returned; that is step 14.2 of "run a module script", after step 13 completes.

The loader spec is actually a separate spec that for now does not interact with this one. In the future if the loader spec gets its browser part finished it may be possible to phrase parts of this spec in terms of its phases. But for now this spec is self-contained and is meant to be implemented independent of the loader spec, since the loader spec doesn't deal with browsers. So there is no need to think about requestSatisfy or similar.

Can we request fetch, translate, instantiate and satisfy operation before reaching deferred phase? That means, "Can we perform fetching of the whole module graph without waiting defer phase?"

Again, these operations do not have meaning in the current spec. But I think I understand what you are getting at.

You can fetch the modules at any time. This is covered by the following note, which already exists in the spec:

For performance reasons, user agents may start fetching the script (as defined above) as soon as the src attribute is set, instead, in the hope that the element will be inserted into the document (and that the crossorigin attribute won't change value in the meantime). Either way, once the element is inserted into the document, the load must have started as described in this step. If the UA performs such prefetching, but the element is never inserted in the document, or the src attribute is dynamically changed, or the crossorigin attribute is dynamically changed, then the user agent will not execute the script so obtained, and the fetching process will have been effectively wasted.

Maybe I should make it clear that this also applies to import statements.

You can also do anything unobservable, as long as your observable consequences are the same as specified. In particular, you could do parsing or instantiation or any such thing, as long as you delay any error reporting until the time specified in the spec (which is defer time). The ES spec contains a note about some of this:

An implementation may parse module source text and analyze it for Early Error conditions prior to the evaluation of ParseModule for that module source text. However, the reporting of any errors must be deferred until the point where this specification actually performs ParseModule upon that source text.

Maybe I should add a specific note to the HTML spec along these lines too.

You cannot do evaluation until the whole graph (rooted at a particular <script type="module">) completes instantiation. So evaluation must necessarily be done at defer timing.

@zxqfox

This comment has been minimized.

Show comment
Hide comment
@zxqfox

zxqfox Dec 30, 2015

Would be great to also have some way to load a bunch of modules at once. Something like a bunch of <script type="module" src="/Module-URI.js">module sources</script> or just <script type="module" src="//cdn/popular-library-as-a-module.js"></script> to force loading asap.

In my universe I have like hundreds of micromodules bundled into the single or few js files.

zxqfox commented Dec 30, 2015

Would be great to also have some way to load a bunch of modules at once. Something like a bunch of <script type="module" src="/Module-URI.js">module sources</script> or just <script type="module" src="//cdn/popular-library-as-a-module.js"></script> to force loading asap.

In my universe I have like hundreds of micromodules bundled into the single or few js files.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 30, 2015

Member

@zxqfox You can do that pretty easily, either with multiple <script type="module" src="..."> tags or with:

<script type="module">
import "https://example.com/module1.js";
import "https://example.com/module2.js";
import "https://example.com/module3.js";
</script>

This should be pretty rare though, as the number of top-level modules in an application is usually 1, maybe 2. In practice it's more likely that you will have a dependency tree of lots of non-top-level modules fanning out from one or two top-level modules via statements like import $ from "https://example.com/jquery.js" or import { doStuff } from "./stuff-doer.js".

Member

domenic commented Dec 30, 2015

@zxqfox You can do that pretty easily, either with multiple <script type="module" src="..."> tags or with:

<script type="module">
import "https://example.com/module1.js";
import "https://example.com/module2.js";
import "https://example.com/module3.js";
</script>

This should be pretty rare though, as the number of top-level modules in an application is usually 1, maybe 2. In practice it's more likely that you will have a dependency tree of lots of non-top-level modules fanning out from one or two top-level modules via statements like import $ from "https://example.com/jquery.js" or import { doStuff } from "./stuff-doer.js".

@zxqfox

This comment has been minimized.

Show comment
Hide comment
@zxqfox

zxqfox Dec 30, 2015

Yeah, sure. It's a dependency tree, not a top-level bush.
Here is an example of core bundle with a bunch of modules https://yastatic.net/bem-core/latest/desktop/bem-core.dev.js. Each module is a modules.define call with imports passed as a second param.

I'm trying to adopt this spec to my reality and I like it. Good job! The only thing disturbing me is HTML requirement for ES modules. Unfortunately, we can't pack a bunch of modules into the single file like we doing it atm, right? But we shouldn't do that with h/2, because it's not the problem anymore, right? ;-)

zxqfox commented Dec 30, 2015

Yeah, sure. It's a dependency tree, not a top-level bush.
Here is an example of core bundle with a bunch of modules https://yastatic.net/bem-core/latest/desktop/bem-core.dev.js. Each module is a modules.define call with imports passed as a second param.

I'm trying to adopt this spec to my reality and I like it. Good job! The only thing disturbing me is HTML requirement for ES modules. Unfortunately, we can't pack a bunch of modules into the single file like we doing it atm, right? But we shouldn't do that with h/2, because it's not the problem anymore, right? ;-)

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 30, 2015

Member

You can certainly pack a bunch of modules into a single file. You'll need a bundling technique, but you seem to already have one of those; you can continue using that, or any of the other bundlers (webpack, browserify, etc.). This feature is more for people who want their dependency tree to be loaded by the browser, without a build step.

HTTP/2 certainly helps give you the best of both worlds (no build step but also pretty good performance). But in the real world I hear that it is not as fast as a bundle in at least Chrome's implementation (something about switching between the foreground and network threads too often). But anyone who cares about production-level performance needs a build step anyway, for e.g. minification or tree-shaking.

If we were to try to solve the "HTTP2 is not as fast as bundling in Chrome" problem with a spec, I'm not sure how we'd go about it. (Or if it'd be worthwhile to do so, given that that still doesn't solve minification or tree-shaking.) But if we did want to do that I imagine it'd look like something along the lines of the Packaging on the Web draft, since it's not just JS you want to bundle---it's CSS, images, and HTML files too. So it's pretty out-of-scope for this pull request.

Member

domenic commented Dec 30, 2015

You can certainly pack a bunch of modules into a single file. You'll need a bundling technique, but you seem to already have one of those; you can continue using that, or any of the other bundlers (webpack, browserify, etc.). This feature is more for people who want their dependency tree to be loaded by the browser, without a build step.

HTTP/2 certainly helps give you the best of both worlds (no build step but also pretty good performance). But in the real world I hear that it is not as fast as a bundle in at least Chrome's implementation (something about switching between the foreground and network threads too often). But anyone who cares about production-level performance needs a build step anyway, for e.g. minification or tree-shaking.

If we were to try to solve the "HTTP2 is not as fast as bundling in Chrome" problem with a spec, I'm not sure how we'd go about it. (Or if it'd be worthwhile to do so, given that that still doesn't solve minification or tree-shaking.) But if we did want to do that I imagine it'd look like something along the lines of the Packaging on the Web draft, since it's not just JS you want to bundle---it's CSS, images, and HTML files too. So it's pretty out-of-scope for this pull request.

@zxqfox

This comment has been minimized.

Show comment
Hide comment
@zxqfox

zxqfox Dec 30, 2015

Thanks for clarification!

Packaging on the Web? Oh, that's what I've missed! Thanks for pointing.

zxqfox commented Dec 30, 2015

Thanks for clarification!

Packaging on the Web? Oh, that's what I've missed! Thanks for pointing.

Show outdated Hide outdated source
data-x="origin">origins</span>, whether error information will be exposed.</p>
<span>CORS settings attribute</span>. For scripts that are not JavaScript modules, it controls
whether error information will be exposed, when the script is obtained from other <span
data-x="origin">origins</span>. JavaScript modules are TODO.</p>

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

I think for JavaScript modules we want to require CORS. Since JavaScript modules use new syntax and different parsing rules, they would introduce new ways to leak data from existing file types if we did not require CORS.

@annevk

annevk Jan 1, 2016

Member

I think for JavaScript modules we want to require CORS. Since JavaScript modules use new syntax and different parsing rules, they would introduce new ways to leak data from existing file types if we did not require CORS.

This comment has been minimized.

@domenic

domenic Jan 1, 2016

Member

Right, that was the intent. However, I am unsure about how to handle the crossorigin="" attribute in particular. All the existing places it can be used are for resources that don't require CORS, and so all the spec text surrounding it seems to be geared toward that. I see two options and would love your help deciding between them, or finding a better one:

  • Ignore the crossorigin="" attribute always, and use the same CORS policy (mode = "cors", credentialsMode = "omit"? or a different credentialsMode.)
  • Come up with a slightly different meaning for the crossorigin="" attribute as applies to module scripts:
    • For img/legacy script/link, missing = ("no-cors", null); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")
    • For script type="module", missing = ("cors", "omit"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")

I guess the latter is not too bad. It will require a bit of modification to the spec's "CORS settings attribute" concept, but the author-facing differences are not a big deal. What do you think?

@domenic

domenic Jan 1, 2016

Member

Right, that was the intent. However, I am unsure about how to handle the crossorigin="" attribute in particular. All the existing places it can be used are for resources that don't require CORS, and so all the spec text surrounding it seems to be geared toward that. I see two options and would love your help deciding between them, or finding a better one:

  • Ignore the crossorigin="" attribute always, and use the same CORS policy (mode = "cors", credentialsMode = "omit"? or a different credentialsMode.)
  • Come up with a slightly different meaning for the crossorigin="" attribute as applies to module scripts:
    • For img/legacy script/link, missing = ("no-cors", null); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")
    • For script type="module", missing = ("cors", "omit"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")

I guess the latter is not too bad. It will require a bit of modification to the spec's "CORS settings attribute" concept, but the author-facing differences are not a big deal. What do you think?

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

Well, e.g., <track> does require CORS, but it defaults to "same-origin" and requires crossorigin="" to be specified for "cors". When integrity="" is specified you get a similar effect. I would be okay with defaulting to ("cors", "same-origin") though, rather than ("same-origin", "same-origin").

@annevk

annevk Jan 1, 2016

Member

Well, e.g., <track> does require CORS, but it defaults to "same-origin" and requires crossorigin="" to be specified for "cors". When integrity="" is specified you get a similar effect. I would be okay with defaulting to ("cors", "same-origin") though, rather than ("same-origin", "same-origin").

This comment has been minimized.

@domenic

domenic Jan 1, 2016

Member

Ah interesting I didn't see track. I don't quite follow how it defaults to same-origin though. Per https://html.spec.whatwg.org/#sourcing-out-of-band-text-tracks:attr-media-crossorigin it seems to use the same logic as all the others.

In any case is this your proposed mapping? missing = ("cors", "same-origin"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")?

@domenic

domenic Jan 1, 2016

Member

Ah interesting I didn't see track. I don't quite follow how it defaults to same-origin though. Per https://html.spec.whatwg.org/#sourcing-out-of-band-text-tracks:attr-media-crossorigin it seems to use the same logic as all the others.

In any case is this your proposed mapping? missing = ("cors", "same-origin"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")?

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

Per https://html.spec.whatwg.org/#start-the-track-processing-model I think it always ends up as "same-origin", which is a bug I will fix tomorrow 😟.

What you suggest there is what seems easiest for developers, yes. Although we might want to make missing ("same-origin", "same-origin") so it's clear CORS is needed (and it's clear CORS is used when inspecting the HTML). Perhaps best to ask web developers for input here if they prefer brevity or being explicit.

@annevk

annevk Jan 1, 2016

Member

Per https://html.spec.whatwg.org/#start-the-track-processing-model I think it always ends up as "same-origin", which is a bug I will fix tomorrow 😟.

What you suggest there is what seems easiest for developers, yes. Although we might want to make missing ("same-origin", "same-origin") so it's clear CORS is needed (and it's clear CORS is used when inspecting the HTML). Perhaps best to ask web developers for input here if they prefer brevity or being explicit.

This comment has been minimized.

@domenic

domenic Jan 1, 2016

Member

Oh, there's another important consideration. What do we do about module fetches originating from import statements? I see two options:

  • Inherit the mode + credentials mode from their root <script type="module"> (whose mode + credentials mode is controlled by the crossorigin attribute).
  • Decide upon one policy that applies to all such fetches. If we do this we might want to make the crossorigin attribute a no-op, since it seems kind of pointless to allow customizable policy only for the root module fetches.
@domenic

domenic Jan 1, 2016

Member

Oh, there's another important consideration. What do we do about module fetches originating from import statements? I see two options:

  • Inherit the mode + credentials mode from their root <script type="module"> (whose mode + credentials mode is controlled by the crossorigin attribute).
  • Decide upon one policy that applies to all such fetches. If we do this we might want to make the crossorigin attribute a no-op, since it seems kind of pointless to allow customizable policy only for the root module fetches.

This comment has been minimized.

@jakearchibald

jakearchibald Jan 2, 2016

Collaborator

Having type="module" use CORS will catch people out, but I don't see it being a huge problem. People who want to host modules for others to use should have enough server control to add headers.

For script type="module", missing = ("cors", "omit"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")

If we're having CORS by default, this makes the most sense for the src script, but what about imported scripts? I can imagine situations where you want one imported script to be requested with credentials, but not the rest. Not sure how to achieve this without adding something to the import statement itself.

I haven't looked into import statements a great deal, but it's down to the host environment to decide how to interpret the statement right? Could hack the string a bit? import "foobar #use-credentials"

@jakearchibald

jakearchibald Jan 2, 2016

Collaborator

Having type="module" use CORS will catch people out, but I don't see it being a huge problem. People who want to host modules for others to use should have enough server control to add headers.

For script type="module", missing = ("cors", "omit"); anonymous = ("cors", "same-origin"); use-credentials = ("cors", "include")

If we're having CORS by default, this makes the most sense for the src script, but what about imported scripts? I can imagine situations where you want one imported script to be requested with credentials, but not the rest. Not sure how to achieve this without adding something to the import statement itself.

I haven't looked into import statements a great deal, but it's down to the host environment to decide how to interpret the statement right? Could hack the string a bit? import "foobar #use-credentials"

This comment has been minimized.

@domenic

domenic Jan 2, 2016

Member

For imported scripts there are the two options I see above, mainly. I agree it's possible to imagine more granular control, but it would be quite tricky. I don't think we should support scenarios like a single imported script being requested with credentials, for now. The question in my mind is whether we should support importing scripts that require credentials at all.

Hacking the string is a bit tough since foobar #use-credentials is a valid relative URL. I think you'd have to use the beginning of the string, which is already reserved, and you'd have to come up with something that was an invalid relative URL starting character, which might not exist. In general I'd prefer these strings to be simpler.

@domenic

domenic Jan 2, 2016

Member

For imported scripts there are the two options I see above, mainly. I agree it's possible to imagine more granular control, but it would be quite tricky. I don't think we should support scenarios like a single imported script being requested with credentials, for now. The question in my mind is whether we should support importing scripts that require credentials at all.

Hacking the string is a bit tough since foobar #use-credentials is a valid relative URL. I think you'd have to use the beginning of the string, which is already reserved, and you'd have to come up with something that was an invalid relative URL starting character, which might not exist. In general I'd prefer these strings to be simpler.

This comment has been minimized.

@annevk

annevk Jan 4, 2016

Member

@domenic technically using spaces is not valid inside a URL so they could be used as separator. Where we first split the value on spaces and then do something.

The last time this was discussed with folks I think inheriting the policy from <script> was considered acceptable and if folks needed more control they would have to write their own loader.

@annevk

annevk Jan 4, 2016

Member

@domenic technically using spaces is not valid inside a URL so they could be used as separator. Where we first split the value on spaces and then do something.

The last time this was discussed with folks I think inheriting the policy from <script> was considered acceptable and if folks needed more control they would have to write their own loader.

This comment has been minimized.

@annevk

annevk Jan 4, 2016

Member

Note that things like referrerpolicy are probably also best inherited. Though with integrity things fall flat. I guess at that point you will need scripted loaders...

@annevk

annevk Jan 4, 2016

Member

Note that things like referrerpolicy are probably also best inherited. Though with integrity things fall flat. I guess at that point you will need scripted loaders...

Show outdated Hide outdated source
<li>If <var>the script block's type string</var> is an <span>ASCII case-insensitive</span>
match for any <span>JavaScript MIME type</span>, <span>the script's type</span> is "<code
data-x="">legacy</code>".</li>

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

Since TC39 does not call this legacy I don't think we should either. (They reserve legacy for octal syntax.) It seems there are JavaScript modules and JavaScript scripts. Or do you expect to change this?

@annevk

annevk Jan 1, 2016

Member

Since TC39 does not call this legacy I don't think we should either. (They reserve legacy for octal syntax.) It seems there are JavaScript modules and JavaScript scripts. Or do you expect to change this?

This comment has been minimized.

@domenic

domenic Jan 1, 2016

Member

I would say in some sense TC39 considers them legacy, although not the kind of legacy you can ever get rid of. See e.g. the arguments in https://www.w3.org/Bugs/Public/show_bug.cgi?id=25868#c5.

That said I am totally fine calling it something different. But I think it's too confusing to say script vs. module when both use the script element. Module script vs. (some-other-adjective) script seems like the way to go to me. "Classic script"?

@domenic

domenic Jan 1, 2016

Member

I would say in some sense TC39 considers them legacy, although not the kind of legacy you can ever get rid of. See e.g. the arguments in https://www.w3.org/Bugs/Public/show_bug.cgi?id=25868#c5.

That said I am totally fine calling it something different. But I think it's too confusing to say script vs. module when both use the script element. Module script vs. (some-other-adjective) script seems like the way to go to me. "Classic script"?

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

I like classic. Perhaps TC39 can rename Script to Classic and ScriptOrModule to Script.

@annevk

annevk Jan 1, 2016

Member

I like classic. Perhaps TC39 can rename Script to Classic and ScriptOrModule to Script.

Show outdated Hide outdated source
<p class="note">Only one of these two pieces of state is set.</p>
<p class="note">Only one of these two pieces of state is set. If <span>the script's type</span>
is "<code data-x="">module</code>", and it has a <code data-x="attr-script-src">src</code>
attribute, then both pieces of state will be ignored anyway.</p>

This comment has been minimized.

@annevk

annevk Jan 1, 2016

Member

Adding the src conditional makes this confusing, since for "legacy" they are ignored if src is missing and for module they will be too.

@annevk

annevk Jan 1, 2016

Member

Adding the src conditional makes this confusing, since for "legacy" they are ignored if src is missing and for module they will be too.

This comment has been minimized.

@domenic

domenic Jan 1, 2016

Member

Yeah that conditional just seems wrong, thanks for catching.

@domenic

domenic Jan 1, 2016

Member

Yeah that conditional just seems wrong, thanks for catching.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 16, 2016

Member

Oh, no, happy to wait! I'd actually been meaning to ask you for review given your recent work in the area but it just slipped my mind.

Member

domenic commented Jan 16, 2016

Oh, no, happy to wait! I'd actually been meaning to ask you for review given your recent work in the area but it just slipped my mind.

@Ms2ger

This comment has been minimized.

Show comment
Hide comment
@Ms2ger

Ms2ger Jan 18, 2016

Member

Still reading, will continue tomorrow. Looks like a big improvement so far!

Member

Ms2ger commented Jan 18, 2016

Still reading, will continue tomorrow. Looks like a big improvement so far!

@Ms2ger

This comment has been minimized.

Show comment
Hide comment
@Ms2ger

Ms2ger Jan 19, 2016

Member
+  <p><img src="/images/asyncdefer.svg" style="width: 80%; min-width: 820px;" alt="With &lt;script&gt;, parsing is interrupted by fetching and execution. With &lt;script defer&gt;, fetching is parallel to parsing and execution takes place after all parsing has finished. And with &lt;script async&gt;, fetching is parallel to parsing but once it finishes parsing is interrupted to execute the script. The story for &lt;script type=&quot;module&quot;&gt; is similar to &lt;script defer&gt;, but the dependencies must be fetched as well; the story for &lt;script type=&quot;module&quot; async&gt; is similar to &lt;script async&gt; with extra dependency fetching."></p>

Avoid "must" in the alt text here.

+ const substitutions = new Map([
+   ["witness", "these dudes I know"]

"witnesses"

+  Once it is set, either to a <span>script</span> in the case of success
+  or to null in the case of failure,

I don't really like using "null" to mean "failure"; I'd use Result::Err in Rust. Not sure I have something better than "to error in the case of failure".

+     <li>If <var>the script block's type string</var> is an <span>ASCII case-insensitive</span>
+     match for the string "<code data-x="">module</code>", <span data-x="concept-script-type">the
+     script's type</span> is "<code data-x="">module</code>".</li>

Did you intend to allow spaces around the "module" string? (I believe it would be somewhat simpler not to allow it in Servo.)

+   <li id="establish-script-block-source">
+         <li><p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+         <var>source text</var> and <var>settings</var>.</p></li>

+         <li><p>If this returns error, set <span data-x="concept-script-script">the script's
+         script</span> to null. Otherwise, set <span data-x="concept-script-script">the script's
+         script</span> to <var>script</var>.</p></li>

I don't see where "creating a classic script" returns error.

   <p>When the user agent is required to <dfn data-x="execute the script block">execute a script
-  block</dfn>, it must run the following steps:</p>
+  block</dfn>, it must run the following steps. For the purposes of these steps, the script is
+  considered to be from an <i>external file</i> if while the <span>prepare a script</span>
+  algorithm above was running for this script, the <code>script</code> element had a <code
+  data-x="attr-script-src">src</code> attribute specified.</p>

Perhaps it would be better to say "The script is from an external file." somewhere in prepare-a-script; followup?

+    <p>Otherwise<span>queue a task</span> to <span>fire a simple event</span> named <code

Missing space.

+  <p>A <dfn>classic script</dfn> has:</p>

Maybe "… additionally has" (same for module script).

+   <dt>A <dfn>module map</dfn></dt>
+
+   <dd>
+
+    <p>A map of <span>absolute URL</span>s to <span>module script</span>s, initially empty. It is
+    is used to ensure that imported JavaScript modules are only fetched, parsed, and evaluated
+    once. Entries may contain the value "<code data-x="">fetching</code>" instead of a
+    <span>module script</span> as a signal that the module is being fetched.</p>

The "may" here doesn't feel quite right. Maybe A map of <span>absolute URL</span>s to <span>module script</span>s or placeholder values "<code data-x="">fetching</code>",?

Oh, but it can also contain null. So perhaps A map of <span>absolute URL</span>s to values that are a <span>module script</span>, null, or a placeholder value "<code data-x="">fetching</code>",

+    <p>Return from this algorithm, and run the remaining steps as part of the fetch's
+    <span>process response</span> for the <span data-x="concept-response">response</span>
+    <var>response</var>.</p>

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

+   <li>
+    <p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+    <var>source text</var> and <var>settings object</var>.</p>

+    <p>If <var>response</var> was <span>CORS-cross-origin</span>, then pass the <var>muted
+    errors</var> flag to the <span data-x="creating a classic script">create a classic
+    script</span> algorithm as well.</p>

This is so much easier to follow, thank you!

+     <li><p>The result of <span data-x="extract a MIME type">extracting a MIME type</span> from
+     <var>response</var>'s <span data-x="concept-response-header-list">header list</span> (ignoring
+     parameters) is not a <span>JavaScript MIME type</span>.</p></li>

Since classic scripts don't check the header, add a note?

+   <li><p>Let <var>script</var> be a new <span>classic script</span> that this algorithm will
+   subsequently initialise.</p></li>

+   <li><p>If <span data-x="concept-bc-noscript">scripting is disabled</span> for the given
+   <span>environment settings object</span>'s <span>responsible browsing context</span>, then abort
+   these steps, as if the script source described a program that did nothing but return
+   void.</p></li>

This made a lot more sense in an algorithm that didn't return anything; not so much in an algortihm that's supposed to return a script.

+  <p>To <dfn data-x="creating a module script">create a module script</dfn>, given some script
+  source, an <span>environment settings object</span>, a script base URL, and a <span>CORS settings
+  attribute</span> state:</p>

+   <li><p>If <span data-x="concept-bc-noscript">scripting is disabled</span> for the given
+   <span>environment settings object</span>'s <span>responsible browsing context</span>, then abort
+   these steps, as if the script source described a program that did nothing but return
+   void.</p></li>

Same comment.

+   <li><p>Let <var>result</var> be <span
+   data-x="js-ParseModule">ParseModule</span>(the provided script source, <var>realm</var>,
+   <var>script</var>).</p></li>

ParseModule takes two arguments.

Looks like it shouldn't anymore, but https://tc39.github.io/ecma262/ is out of date. sigh

+   <li><p>If <var>result</var> is a List of errors, <span>report the exception</span> given by the
+   first element of <var>result</var> for <var>script</var>, return null, and abort these
+   steps.</p></li>

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

+  <p>The steps to <span id="prepare-to-run-a-callback"></span><dfn>prepare to run script</dfn> with

Nit: wrap the span around the dfn.

+   <li><p>Let <var>resolved module script</var> be the value of the entry in <var>module map</var>
+   whose key is <var>url</var>. If no such entry exists, throw a <code>TypeError</code> exception
+   and abort these steps.</p></li>

Also need to handle null and maybe "fetching" here.

+      <p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+      <var>source</var>, <var>settings object</var>.</p>

comma -> "and", I think.

I didn't review the execution bits in detail.

Member

Ms2ger commented Jan 19, 2016

+  <p><img src="/images/asyncdefer.svg" style="width: 80%; min-width: 820px;" alt="With &lt;script&gt;, parsing is interrupted by fetching and execution. With &lt;script defer&gt;, fetching is parallel to parsing and execution takes place after all parsing has finished. And with &lt;script async&gt;, fetching is parallel to parsing but once it finishes parsing is interrupted to execute the script. The story for &lt;script type=&quot;module&quot;&gt; is similar to &lt;script defer&gt;, but the dependencies must be fetched as well; the story for &lt;script type=&quot;module&quot; async&gt; is similar to &lt;script async&gt; with extra dependency fetching."></p>

Avoid "must" in the alt text here.

+ const substitutions = new Map([
+   ["witness", "these dudes I know"]

"witnesses"

+  Once it is set, either to a <span>script</span> in the case of success
+  or to null in the case of failure,

I don't really like using "null" to mean "failure"; I'd use Result::Err in Rust. Not sure I have something better than "to error in the case of failure".

+     <li>If <var>the script block's type string</var> is an <span>ASCII case-insensitive</span>
+     match for the string "<code data-x="">module</code>", <span data-x="concept-script-type">the
+     script's type</span> is "<code data-x="">module</code>".</li>

Did you intend to allow spaces around the "module" string? (I believe it would be somewhat simpler not to allow it in Servo.)

+   <li id="establish-script-block-source">
+         <li><p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+         <var>source text</var> and <var>settings</var>.</p></li>

+         <li><p>If this returns error, set <span data-x="concept-script-script">the script's
+         script</span> to null. Otherwise, set <span data-x="concept-script-script">the script's
+         script</span> to <var>script</var>.</p></li>

I don't see where "creating a classic script" returns error.

   <p>When the user agent is required to <dfn data-x="execute the script block">execute a script
-  block</dfn>, it must run the following steps:</p>
+  block</dfn>, it must run the following steps. For the purposes of these steps, the script is
+  considered to be from an <i>external file</i> if while the <span>prepare a script</span>
+  algorithm above was running for this script, the <code>script</code> element had a <code
+  data-x="attr-script-src">src</code> attribute specified.</p>

Perhaps it would be better to say "The script is from an external file." somewhere in prepare-a-script; followup?

+    <p>Otherwise<span>queue a task</span> to <span>fire a simple event</span> named <code

Missing space.

+  <p>A <dfn>classic script</dfn> has:</p>

Maybe "… additionally has" (same for module script).

+   <dt>A <dfn>module map</dfn></dt>
+
+   <dd>
+
+    <p>A map of <span>absolute URL</span>s to <span>module script</span>s, initially empty. It is
+    is used to ensure that imported JavaScript modules are only fetched, parsed, and evaluated
+    once. Entries may contain the value "<code data-x="">fetching</code>" instead of a
+    <span>module script</span> as a signal that the module is being fetched.</p>

The "may" here doesn't feel quite right. Maybe A map of <span>absolute URL</span>s to <span>module script</span>s or placeholder values "<code data-x="">fetching</code>",?

Oh, but it can also contain null. So perhaps A map of <span>absolute URL</span>s to values that are a <span>module script</span>, null, or a placeholder value "<code data-x="">fetching</code>",

+    <p>Return from this algorithm, and run the remaining steps as part of the fetch's
+    <span>process response</span> for the <span data-x="concept-response">response</span>
+    <var>response</var>.</p>

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

+   <li>
+    <p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+    <var>source text</var> and <var>settings object</var>.</p>

+    <p>If <var>response</var> was <span>CORS-cross-origin</span>, then pass the <var>muted
+    errors</var> flag to the <span data-x="creating a classic script">create a classic
+    script</span> algorithm as well.</p>

This is so much easier to follow, thank you!

+     <li><p>The result of <span data-x="extract a MIME type">extracting a MIME type</span> from
+     <var>response</var>'s <span data-x="concept-response-header-list">header list</span> (ignoring
+     parameters) is not a <span>JavaScript MIME type</span>.</p></li>

Since classic scripts don't check the header, add a note?

+   <li><p>Let <var>script</var> be a new <span>classic script</span> that this algorithm will
+   subsequently initialise.</p></li>

+   <li><p>If <span data-x="concept-bc-noscript">scripting is disabled</span> for the given
+   <span>environment settings object</span>'s <span>responsible browsing context</span>, then abort
+   these steps, as if the script source described a program that did nothing but return
+   void.</p></li>

This made a lot more sense in an algorithm that didn't return anything; not so much in an algortihm that's supposed to return a script.

+  <p>To <dfn data-x="creating a module script">create a module script</dfn>, given some script
+  source, an <span>environment settings object</span>, a script base URL, and a <span>CORS settings
+  attribute</span> state:</p>

+   <li><p>If <span data-x="concept-bc-noscript">scripting is disabled</span> for the given
+   <span>environment settings object</span>'s <span>responsible browsing context</span>, then abort
+   these steps, as if the script source described a program that did nothing but return
+   void.</p></li>

Same comment.

+   <li><p>Let <var>result</var> be <span
+   data-x="js-ParseModule">ParseModule</span>(the provided script source, <var>realm</var>,
+   <var>script</var>).</p></li>

ParseModule takes two arguments.

Looks like it shouldn't anymore, but https://tc39.github.io/ecma262/ is out of date. sigh

+   <li><p>If <var>result</var> is a List of errors, <span>report the exception</span> given by the
+   first element of <var>result</var> for <var>script</var>, return null, and abort these
+   steps.</p></li>

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

+  <p>The steps to <span id="prepare-to-run-a-callback"></span><dfn>prepare to run script</dfn> with

Nit: wrap the span around the dfn.

+   <li><p>Let <var>resolved module script</var> be the value of the entry in <var>module map</var>
+   whose key is <var>url</var>. If no such entry exists, throw a <code>TypeError</code> exception
+   and abort these steps.</p></li>

Also need to handle null and maybe "fetching" here.

+      <p>Let <var>script</var> be the result of <span>creating a classic script</span> using
+      <var>source</var>, <var>settings object</var>.</p>

comma -> "and", I think.

I didn't review the execution bits in detail.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 19, 2016

Member

Thanks very much @Ms2ger!! I addressed almost all your comments. Here are the ones I didn't.

I don't really like using "null" to mean "failure"; I'd use Result::Err in Rust. Not sure I have something better than "to error in the case of failure".

I sympathize, but it seems simpler to just go with this.

Did you intend to allow spaces around the "module" string? (I believe it would be somewhat simpler not to allow it in Servo.)

I didn't really intend to. Does anyone know what we do for similar situations? The only "surrounded by spaces" I can find are all about URLs. What about other enumerated attributes?

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

Agreed it's strange. What do you think of just nesting all remaining steps? Better or meh?

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

Oh, great catch. I started trying to fix this to delay the error until later, but it was enough work that I had to step back and question whether it was a good idea. And in the end I think the as-specced model is correct. (Maybe we should queue a task to avoid being completely sync; let me know what you think.)

The reason is that unlike with scripts, module parsing needs to happen early; it's done in order to determine what the module tree is. There's a very clear separation between the parsing phase for the whole tree, and then the evaluation phase. Delaying errors from the parsing phase until the evaluation phase is not really a very good reflection of what's going on---unlike with classic scripts, where the distinction between parsing and evaluation is uninteresting, so you can conceptualize both as happening at the same time.

You also run into issues like, what if there are multiple parsing errors at different points in the tree: do you race to see which one gets reported as the error, or do you collate them all up? I think just reporting them as they are encountered is the most straightforward and requires less machinery.

Nit: wrap the span around the dfn.

This causes the dfn to become a blue link, annoyingly.

Member

domenic commented Jan 19, 2016

Thanks very much @Ms2ger!! I addressed almost all your comments. Here are the ones I didn't.

I don't really like using "null" to mean "failure"; I'd use Result::Err in Rust. Not sure I have something better than "to error in the case of failure".

I sympathize, but it seems simpler to just go with this.

Did you intend to allow spaces around the "module" string? (I believe it would be somewhat simpler not to allow it in Servo.)

I didn't really intend to. Does anyone know what we do for similar situations? The only "surrounded by spaces" I can find are all about URLs. What about other enumerated attributes?

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

Agreed it's strange. What do you think of just nesting all remaining steps? Better or meh?

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

Oh, great catch. I started trying to fix this to delay the error until later, but it was enough work that I had to step back and question whether it was a good idea. And in the end I think the as-specced model is correct. (Maybe we should queue a task to avoid being completely sync; let me know what you think.)

The reason is that unlike with scripts, module parsing needs to happen early; it's done in order to determine what the module tree is. There's a very clear separation between the parsing phase for the whole tree, and then the evaluation phase. Delaying errors from the parsing phase until the evaluation phase is not really a very good reflection of what's going on---unlike with classic scripts, where the distinction between parsing and evaluation is uninteresting, so you can conceptualize both as happening at the same time.

You also run into issues like, what if there are multiple parsing errors at different points in the tree: do you race to see which one gets reported as the error, or do you collate them all up? I think just reporting them as they are encountered is the most straightforward and requires less machinery.

Nit: wrap the span around the dfn.

This causes the dfn to become a blue link, annoyingly.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Jan 20, 2016

Member

Typically enumerated attributes do not allow spaces around them. E.g., data:text/html,<input type=" range"> gives you a text input control.

Member

annevk commented Jan 20, 2016

Typically enumerated attributes do not allow spaces around them. E.g., data:text/html,<input type=" range"> gives you a text input control.

@zcorpan

This comment has been minimized.

Show comment
Hide comment
@zcorpan

zcorpan Jan 20, 2016

Member

This causes the dfn to become a blue link, annoyingly.

Use data-x=""?

Member

zcorpan commented Jan 20, 2016

This causes the dfn to become a blue link, annoyingly.

Use data-x=""?

@Ms2ger

This comment has been minimized.

Show comment
Hide comment
@Ms2ger

Ms2ger Jan 20, 2016

Member

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

Agreed it's strange. What do you think of just nesting all remaining steps? Better or meh?

Maybe a little better. I think the better solution would be to change Fetch to have callback arguments, but that's not something that should be discussed here. Feel free to ignore for now.

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

...

I haven't made up my mind; happy to defer to @bzbarsky if he has an opinion.

Member

Ms2ger commented Jan 20, 2016

This is a bit of a strange way to structure this algorithm, since the rest of the algorithm is a callback, while it looks like straight-line code until you read it in detail. I'm not sure how to improve it given the way Fetch is written, though.

Agreed it's strange. What do you think of just nesting all remaining steps? Better or meh?

Maybe a little better. I think the better solution would be to change Fetch to have callback arguments, but that's not something that should be discussed here. Feel free to ignore for now.

This will dispatch the error event sync from prepare-a-script, while classic defer scripts dispatch it from execute-a-script-block. Is this intentional?

...

I haven't made up my mind; happy to defer to @bzbarsky if he has an opinion.

@bzbarsky

This comment has been minimized.

Show comment
Hide comment
@bzbarsky

bzbarsky Jan 20, 2016

Collaborator

In general, I'm very wary of sync error or load events, because we have concrete evidence that web sites get into sync infinite loops when that happens, by responding to the event in a way that retriggers the same event. This is why, for example, error events for everything in the platform right now are async even if the error condition is detected sync...

Collaborator

bzbarsky commented Jan 20, 2016

In general, I'm very wary of sync error or load events, because we have concrete evidence that web sites get into sync infinite loops when that happens, by responding to the event in a way that retriggers the same event. This is why, for example, error events for everything in the platform right now are async even if the error condition is detected sync...

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 20, 2016

Member

That makes sense. Would it suffice to just queue a task to report the exception, instead of reporting it synchronously from create a script? That matters mostly for the inline case.

Member

domenic commented Jan 20, 2016

That makes sense. Would it suffice to just queue a task to report the exception, instead of reporting it synchronously from create a script? That matters mostly for the inline case.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 20, 2016

Member

Use data-x=""?

That would defeat the point, which is to have a data-x="old-id" to preserve links.

Member

domenic commented Jan 20, 2016

Use data-x=""?

That would defeat the point, which is to have a data-x="old-id" to preserve links.

@bzbarsky

This comment has been minimized.

Show comment
Hide comment
@bzbarsky

bzbarsky Jan 20, 2016

Collaborator

@domenic So I just talked to @Ms2ger about this a bit more, and he pointed out that the context here is basically inline <script type=module>. In that case, firing error sync is probably fine; that's what inline type="application/javascript" does anyway, right?

Collaborator

bzbarsky commented Jan 20, 2016

@domenic So I just talked to @Ms2ger about this a bit more, and he pointed out that the context here is basically inline <script type=module>. In that case, firing error sync is probably fine; that's what inline type="application/javascript" does anyway, right?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 20, 2016

Member

Yeah, that's true. We have a chance to make it different for modules if you think that would be better in general, though.

Member

domenic commented Jan 20, 2016

Yeah, that's true. We have a chance to make it different for modules if you think that would be better in general, though.

@bzbarsky

This comment has been minimized.

Show comment
Hide comment
@bzbarsky

bzbarsky Jan 20, 2016

Collaborator

I think it's fine to just keep it the same; less confusing.

Collaborator

bzbarsky commented Jan 20, 2016

I think it's fine to just keep it the same; less confusing.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Jan 20, 2016

Member

That would defeat the point, which is to have a data-x="old-id" to preserve links.

Hmm, why not <span id="old-id" data-x="">...</span>? I don't think data-x just becomes an ID.

Member

annevk commented Jan 20, 2016

That would defeat the point, which is to have a data-x="old-id" to preserve links.

Hmm, why not <span id="old-id" data-x="">...</span>? I don't think data-x just becomes an ID.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 20, 2016

Member

OK, cool. So I'm going to fix the "must" Ms2ger found (and maybe the old-id business), then merge this, after I walk to work. Speak now if you think there's something left to fix!

Member

domenic commented Jan 20, 2016

OK, cool. So I'm going to fix the "must" Ms2ger found (and maybe the old-id business), then merge this, after I walk to work. Speak now if you think there's something left to fix!

Add <script type="module"> and module resolution/fetching/evaluation
This adds support for <script type="module"> for loading JavaScript modules, as well as all the infrastructure necessary for resolving, fetching, parsing, and evaluating module graphs rooted at such <script type="module">s.

Some decisions encoded here include:

- Always use the UTF-8 decoder, ignoring the charset="" attribute or the Content-Type header parameter.
- Always use the "cors" fetch mode, and use the crossorigin="" attribute to decide between "omit"/"same-origin"/"include" for the credentials mode.
- Require that the response be served with a JavaScript MIME type for its Content-Type header.
- Use <script defer>-like semantics by default: do not execute until parsing is finished, and execute in order. The async="" attribute can be used to opt in to an "execute as soon as possible, in any order" semantic. Unlike for classic scripts, in no cases do we block further parsing or script execution until the module tree is finished loading. This applies to both inline and external scripts.
- For module resolution, allow absolute URLs and anything that starts with "./", "../", or "/", with this latter set being interpreted as relative URLs. Everything else (e.g. bare specifiers like "jquery") fails for now. No automatic ".js" is appended.
- Modules are memoized based on their request URL, whereas import specifiers inside the module are resolved based on the module's response URL.

In the course of adding this functionality, large parts of script execution, fetching, and creation were refactored, including moving around some sections and reorganizing others. Conceptually, scripts are now either "classic scripts", "module scripts", or "data blocks". The result should generally be clearer and easier to follow.
@hax

This comment has been minimized.

Show comment
Hide comment
@hax

hax May 30, 2016

Contributor

Is there any way can feature detect script type=module support?

Contributor

hax commented May 30, 2016

Is there any way can feature detect script type=module support?

domenic added a commit that referenced this pull request Dec 19, 2016

Add examples of how module maps are keyed
This transposes some of the wisdom from
#443 (comment) into the
spec.

domenic added a commit that referenced this pull request Jan 11, 2017

Add examples of how module maps are keyed
This transposes some of the wisdom from
#443 (comment) into the
spec.
@graingert

This comment has been minimized.

Show comment
Hide comment
@graingert

graingert Sep 11, 2017

@hax no need. Every browser that supports <script type="module" src="/static/new-src.js></script> should support ignoring <script nomodule src="/static/legacy-src.js"></script>

graingert commented Sep 11, 2017

@hax no need. Every browser that supports <script type="module" src="/static/new-src.js></script> should support ignoring <script nomodule src="/static/legacy-src.js"></script>

@hax

This comment has been minimized.

Show comment
Hide comment
@hax

hax Sep 11, 2017

Contributor

@graingert When I asked this question there was no nomodule attribute.

Contributor

hax commented Sep 11, 2017

@graingert When I asked this question there was no nomodule attribute.

@stefaneidelloth

This comment has been minimized.

Show comment
Hide comment
@stefaneidelloth

stefaneidelloth Dec 26, 2017

Is it possible to import the module that has been defined in a <script type="module"> tag to another <script type="module"> tag in the same html file? Or to another JavaScript file that has been loaded by the html file? Also see following SO question: https://stackoverflow.com/questions/47982205/how-to-import-es6-module-that-has-been-defined-in-script-type-module-tag-ins

stefaneidelloth commented Dec 26, 2017

Is it possible to import the module that has been defined in a <script type="module"> tag to another <script type="module"> tag in the same html file? Or to another JavaScript file that has been loaded by the html file? Also see following SO question: https://stackoverflow.com/questions/47982205/how-to-import-es6-module-that-has-been-defined-in-script-type-module-tag-ins

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Jan 3, 2018

Member

The answer you got there seems correct. https://whatwg.org/faq#adding-new-features and https://whatwg.org/working-mode might be of interest if you wish to pursue the idea of making it possible somehow.

Member

annevk commented Jan 3, 2018

The answer you got there seems correct. https://whatwg.org/faq#adding-new-features and https://whatwg.org/working-mode might be of interest if you wish to pursue the idea of making it possible somehow.

@snuggs snuggs referenced this pull request Mar 29, 2018

Merged

HTTP Resource Representation Variants #164

29 of 29 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment