SystemJS import from Webpack bundle

[davispuh]

I can't figure out how to import modules from Webpack bundle.

Basically I have several libraries using AMD.
If I use import map

{
    "imports": {
        "jquery": "./jquery.js",
        "somelib": "./somelib.js",
    }
}

Then System.import("somelib") works fine.

But if I bundle it with Webpack

module.exports = {
    entry: [ "jquery.js", "somelib.js" ],
    output: {
        libraryTarget: "system",
        filename: "bundle.js"
    },
    module: {
        rules: [{
                parser: {
                    amd: true,
                    commonjs: true,
                    system: false
               }
        }]
    },
    resolve: {
        alias: {
			"jquery": "jquery.js"
			"somelib": "somelib.js"
		}
    }
};

Then System.import("somelib") doesn't load it from Webpack bundle.
In generated bundle.js there is

System.register([], function(__WEBPACK_DYNAMIC_EXPORT__) {
	return {
		execute: function() {
			__WEBPACK_DYNAMIC_EXPORT__(
/******/ (function(modules) {
[...]

But looks like names aren't registered.
Note that I don't want to System.import("bundle") but individual library, because whole bundle is already loaded by regular <script> and I don't want to load that dynamically.

[kouts]

I have set in my config

parser: {
  system: false
},

and then I have an entry point e.g. main.js in which I import and then export accordingly

import $ from 'jquery';
import somelib from './somelib.js';
export { $, somelib };

[davispuh]

If I include libs directly as webpack entries then yeah I don't get any exports.
Now when I add this main.js as entry with these exports I do get them in bundle.js
like

/* harmony reexport (default from non-harmony) */ __webpack_require__.d(__webpack_exports__, "jquery", function() { return _vendor_jquery_dist_jquery_js__WEBPACK_IMPORTED_MODULE_0___default.a; });

But System.import('jquery') still doesn't work, it says Error: Unable to resolve specifier 'jquery'

[davispuh]

So basically it appears what I want to do isn't implemented, but I got it working with a little hack.

Now my main.js looks like

let jquery = require('./jquery.js');
let somelib = require('./somelib.js');

export { jquery, somelib };

and webpack config

    output: {
        libraryTarget: "amd-require",
        filename: "bundle.js"
    },
    module: {
        rules: [{
                parser: {
                    amd: true,
                    commonjs: true,
                    system: false
               }
        }]
    },

Before including bundle.js I include this loader I hacked up

((global, systemjs) => {

	global.require = (deps, callback) => {

		delete global.require;

		let baseUrl;
		const baseEl = document.querySelector('base[href]');
		if (baseEl) {
			baseUrl = baseEl.href;
		}

		if (!baseUrl && typeof location !== 'undefined') {
			baseUrl = location.href.split('#')[0].split('?')[0];
			const lastSepIndex = baseUrl.lastIndexOf('/');
			if (lastSepIndex !== -1) {
				baseUrl = baseUrl.slice(0, lastSepIndex + 1);
			}
		}

		const importMap = JSON.parse('{"imports":{"jquery":"./jquery.js","somelib":"./somelib.js"}}');

		const resolveName = (name) => {
			return baseUrl + importMap.imports[name].substring(2);
		}

		const exporter = (exports) => {
			for (let name in exports) {
				let value = exports[name];
				systemjs.set(resolveName(name), { useDefault: true, default: value });
			}
			return value;
		}

		exporter(callback());
	};
})(self, System);

It would be really nice if SystemJS offered public API for setting importMap (basically also bundle it with webpack), then I wouldn't have to hack this much.

[kouts]

My importMap is just the main file
<script type="systemjs-importmap">{"imports":{"main":"main.bundle.js"}}</script>
Also I have libraryTarget: "system" and AFAIK there's no need for

parser: {
  amd: true,
  commonjs: true,
}

[davispuh]

My importMap is just the main file
<script type="systemjs-importmap">{"imports":{"main":"main.bundle.js"}}</script>

yeah, I'm not using SystemJS to load bundle.js itself. Currently I'm just using regular <script> to include it.
Basically my goal was to always use System.import("somelib") in code, independently if it's bundled inside bundle.js or it's a separate file and I can take it out of bundle without changing any of import's. Now finally I have achieved that, but it's not pretty because support is lacking.

Also I have libraryTarget: "system" and AFAIK there's no need for

parser: {
  amd: true,
  commonjs: true,
}

There is, because otherwise AMD define's aren't rewriten, I'm including libraries which use those.

[jolyndenning]

Before I jump in with a suggestion on this, can you help me understand what you're going for? Are you trying to create a single webpack bundle that registers jquery and some-lib into SystemJS, so that you can use them as separate in-browser modules?

If so, check out the following repo that shows how to do this - https://github.com/joeldenning/systemjs-issue-2092

[davispuh]

Before I jump in with a suggestion on this, can you help me understand what you're going for? Are you trying to create a single webpack bundle that registers jquery and some-lib into SystemJS, so that you can use them as separate in-browser modules?

Yeah exactly, but I didn't consider that solution because https://github.com/systemjs/systemjs/blob/master/docs/api.md#systemregisterdeps-declare says "Note: Named System.register is not supported, only anonymous definitions.". And somewhere else read that it's deprecated or something. But yeah it looks a lot simpler this way. Best way would be if webpack would output it directly so that I wouldn't need to write System.register for each library.

Also another nice feature would be if could map relations between individual modules/libraries and bundles. For example in import map I could specify that jquery is inside bundle, so then when I import System.import("jquery") it would load respective bundle (which of course contains multiple other libs).

[jolyndenning]

Docs say "Note: Named System.register is not supported, only anonymous definitions."

Named register is supported through the named-register extra. I just updated the docs in 45ba380 to clarify this.

Best way would be if webpack would output it directly so that I wouldn't need to write System.register for each library.

I agree that that would be nice. The systemjs-builder and old versions of jspm used to support exactly that. However, webpack does not. I was the one who authored the libraryTarget system within webpack (webpack/webpack#8864) and webpack doesn't really have the concept of one bundle that is registering multiple modules into a module registry.

In fact, not even the browser specifications currently support one javascript file that registers multiple modules into the browser's module registry. ES modules are "one file = one module" right now. I imagine future spec work will add ability to do this, since it's a no-brainer for performance. However, right now SystemJS is one of the only tools that makes this possible.

I am closing this since I don't think there's an issue here with SystemJS and have shown a valid way of solving the problem. Feel free to comment further or reopen.

[davispuh]

It really wasn't clear to me how to implement this use case, but now finally it all makes sense and thanks for example!

webpack doesn't really have the concept of one bundle that is registering multiple modules into a module registry.

That's strange, to me I see it as very useful feature, so that your code doesn't need to know how modules are bundled, especially for AMD modules.

In fact, not even the browser specifications currently support one javascript file that registers multiple modules into the browser's module registry. ES modules are "one file = one module" right now. I imagine future spec work will add ability to do this, since it's a no-brainer for performance. However, right now SystemJS is one of the only tools that makes this possible

That's why we have libraries and it's great that it's possible to do this with SystemJS. But yeah I don't see that it would be good idea to use ES modules directly in browser for large applications as usually you want each feature implement in separate module which means downloading 100+ modules will hurt performance and bundles solves this issue hence webpack.

I am closing this since I don't think there's an issue here with SystemJS and have shown a valid way of solving the problem.

There is few things that could be improved - expose import map in API to be able to update/query it outside of SystemJS. That would allow to bundle it with webpack.
Also seems, there isn't any way to import anonymous System.register when it's included with regular <script>. Basically getOrCreateLoad could be split in like 2-3 separate functions and part of it exposed in API so could do System.createFromRegistration(System.getRegister())

[jolyndenning]

expose import map in API to be able to update/query it outside of SystemJS

The import map spec currently provides no way to programatically add or modify import maps see spec. In fact, the spec recently changed so that only one import map is even allowed in the page.

Also seems, there isn't any way to import anonymous System.register when it's included with regular <script>

Correct - this is intentional. You can load it via <script type="systemjs-module"> but not via <script>. Same as normal ES modules.

Basically getOrCreateLoad could be split in like 2-3 separate functions and part of it exposed in API so could do System.createFromRegistration(System.getRegister())

See #2058 for discussion on some changes to the hooks exposed from SystemJS.

So could do System.createFromRegistration(System.getRegister())

This is what the System.instantiate hook is. See #2090, https://github.com/systemjs/systemjs/blob/master/src/extras/module-types.js, and https://github.com/systemjs/systemjs/blob/master/src/extras/amd.js for examples. I just realized that instantiate is not documented anywhere, so I'm going to add some documentation for it now.

[jolyndenning]

See #2094 for docs regarding instantiate.

[davispuh]

The import map spec currently provides no way to programatically add or modify import maps see spec. In fact, the spec recently changed so that only one import map is even allowed in the page.

SystemJS still could implement it, because it can be superset of official spec and provide additional features.

Also seems, there isn't any way to import anonymous System.register when it's included with regular <script>

Correct - this is intentional. You can load it via <script type="systemjs-module"> but not via <script>. Same as normal ES modules.

I don't see why? My use case would be webpack bundle with bunch of System.register. Because it's transpiled to non-ES6 and not actual ES module.

Basically getOrCreateLoad could be split in like 2-3 separate functions and part of it exposed in API so could do System.createFromRegistration(System.getRegister())

See #2058 for discussion on some changes to the hooks exposed from SystemJS.

So could do System.createFromRegistration(System.getRegister())

This is what the System.instantiate hook is. See #2090, https://github.com/systemjs/systemjs/blob/master/src/extras/module-types.js, and https://github.com/systemjs/systemjs/blob/master/src/extras/amd.js for examples. I just realized that instantiate is not documented anywhere, so I'm going to add some documentation for it now.

That's not true, instantiate expects URL not result from System.register. Use case would be to create from already loaded resource.

[jolyndenning]

SystemJS still could implement it, because it can be superset of official spec and provide additional features.

True - my preference is towards waiting for a proposed API that we polyfill instead of creating our own in SystemJS. One thing to note is that the import maps spec has been quite resistant to modifying import maps / URL resolution whatsoever after the first module is downloaded, which is relevant to the programmatic API. Another thing to note is that, with SystemJS, you can already adjust an import map before it is used by inserting an additional import map into the DOM before that first module is loaded. An example of a project that does so is https://github.com/joeldenning/import-map-overrides

I don't see why? My use case would be webpack bundle with bunch of System.register. Because it's transpiled to non-ES6 and not actual ES module.

SystemJS is a polyfill for browser features. The browser specs do not let you load a module without putting type="module" on your <script>.

SystemJS explores some experimental browser features and even invents a couple new features, such as named System.register, which afaik has no counterpart in the current browser spec landscape. However, SystemJS is largely going for polyfill-like behavior.

That's not true, instantiate expects URL not result from System.register. Use case would be to create from already loaded resource.

Creating a module based on an already loaded resource is not a use case I understand and seems pretty far away from my understanding of how javascript modules work - a loaded resource maps 1:1 with registering a module in every spec I am aware of. The single unique thing about modules is their URL. Exposing getOrCreateLoad would expose the the SystemJS load object, which is internal and not exposed anywhere (afaik) to users of SystemJS. I'd rather keep it that way. Between hooking System.instantiate and using System.set() / System.register(), I think you can accomplish everything you need.