createScript / fetch / shouldFetch hooks

[guybedford]

This implements three new hooks to open up the loader a little more. The docs contains the full explanation, but the gist is:

1. Module types calls System.shouldFetch(), and then does a fetch based on that check. This allows eg setting up CSS modules for files not ending in .css, or ensuring all modules are fetched.
2. Instead of calling global fetch, System.fetch is a global fetch alias, allowing the fetch function itself to be hooked. Effectively opening up to any custom transforms etc as an alternative to the transform hook.
3. When doing a script load, createScript hooking allows hooking the script creation. This can be useful to inject custom authentication / integrity etc.

The hope is the above enables new types of userland plugins. The reality is no one will likely notice this PR even landed until one person in 6 months time decides they want the feature, and are grateful to find it. But hey I had a fun Sunday afternoon celebrating 10k stars.

[guybedford]

This also fixes up the MIME checks in module types to properly enforce full MIME validation too.

[guybedford]

I've also refactored the global error listener.

[guybedford]

With strict MIME checking, this now catches all unknown MIMEs, including the HTML error.

[guybedford]

This resolves #1847 and #1863.

[joeldenning]

Would it save bytes to do Object.assign(script, {charset: 'utf-8', ...})

[guybedford]

No Object.assign in IE11 :)

[appsou]

Uncaught SyntaxError: Unexpected token export when import ES6 module

es6.js file
export var demo = "hello";

[joeldenning]

Hi @appsou, could you clarify what your comment means? Are you reporting a bug with a current version of systemjs? Or are you reporting a problem with this PR?

Note that systemjs does not support loading ES modules, as explained here and here. It primarily supports loading System.register modules.

[joeldenning]

lgtm

[frehner]

When doing a script load, createScript hooking allows hooking the script creation. This can be useful to inject custom authentication / integrity etc.

For what it's worth, I've wondered about adding integrity to the scripts that systemjs creates. Haven't necessarily needed it, but I wondered about how and if I could do it :)

[joeljeske]

Is anything blocking this PR?

[guybedford]

@joeldenning what are your thoughts here?

[joeldenning]

createScript makes a lot of sense, but with JSON, CSS, and HTML modules now having uncertain status, I don't know if it's a good idea to commit to a public hookable API (shouldFetch, fetch) for those kinds of modules. I think I'd lean towards only exposing createScript until there is more certainty around those module types, to avoid exposing an API we can't really stay behind as things change.

[guybedford]

This doesn't just apply to module types though - it also permits an alternative to the transform loader model, and enables features such as custom fetch protocols (eg if someone wanted to implement the ability to fetch over RPC / with authorization etc).

To be honest those features are the more useful part of this PR than its effect on module types architecture, although for the module types architecture it does also handle #2086 as per #2086 (comment).

[joeldenning]

Fair enough - in that case I think these are good changes.

[jdfrench44]

This feature would be helpful for me - I'm currently experiencing an issue where the browser isn't sending necessary cookies when requesting scripts, and it would be solved if I could set crossOrigin: 'use-credentials' in createScript hook.

[joeljeske]

Is there anything blocking the merge?

[joeldenning]

No - just forgot to merge it. I'll merge master in and then merge this in.

[joeldenning]

I'll publish this as part of 6.2.0 once #2114 is reviewed and merged.

[joeldenning]

Published in https://github.com/systemjs/systemjs/releases/tag/6.2.0

[jssoriao]

do we have any examples of using this createScript hook to provide a bearer token? All I've seen is the default.

<script>
    System.constructor.prototype.createScript = function (url) {
        const script = document.createElement('script');
        script.charset = 'utf-8';
        script.async = true;
        script.crossOrigin = 'anonymous';
        script.src = url;
        return script;
      };
</script>

What should I do here to add the auth header?

[joeldenning]

Hi @jsoriao - we discussed this in the single-spa slack workspace at https://single-spa.slack.com/archives/C8R6U7MT7/p1586336163337900.

You cannot configure arbitrary headers to be sent with script tags, but you can tell the browser to automatically send the Authorization header by setting script.crossOrigin = 'credentials' and then having the server respond with WWW-Authorization header. Once the user is authenticated with that server, the browser will automatically send the correct header. Alternatively, you can use the shouldFetch and fetch hooks to manually set the authorization header's value.

[sscaff1]

How would I send a JWT token when fetching a script? That's not clear to me.
Instead of System.import, I would use System.fetch with the authorization header, but then how do I load the MFE?

  const applications = constructApplications({
    routes,
    loadApp({ name }) {
      // return System.import(name);
      // what do I do here instead of System.import?
    },
  });

[joeldenning]

@sscaff1 to tell SystemJS to use fetch() rather than <script>, create a shouldFetch hook that always returns true. Once you've done that, you can implement a fetch hook that adds a JWT token to the request.