Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] BREAKING: interop, take 2 #92

Merged
merged 11 commits into from Aug 31, 2016

Conversation

Projects
None yet
4 participants
@Rich-Harris
Copy link
Contributor

commented Aug 30, 2016

This supersedes #91. Summary:

  • Sourcemaps are no longer generated for CommonJS modules (explanation below)
  • Transformed modules are smaller, largely because namespaces are no longer needlessly generated
  • The default export of a transformed CommonJS module is module.exports, or, if exports.__esModule === true, exports.default (previously it would always use .default if it was present)
  • If a CommonJS module imports another CommonJS module, .default interop no longer takes place – this fixes bugs like rollup/rollup#866

I won't lie: the approach here feels a little bit crazy. I think that's just the price of having good (and efficient) interop.

Before the explanation, a couple of comparisons:

Basic CommonJS importing

Source

// main.js
var foo = require( './foo' );
module.exports = foo * 2;

// foo.js
module.exports = 21;

Before

'use strict';

function interopDefault(ex) {
  return ex && typeof ex === 'object' && 'default' in ex ? ex['default'] : ex;
}

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var foo = createCommonjsModule(function (module) {
module.exports = 21;
});

var foo$1 = interopDefault(foo);


var require$$0 = Object.freeze({
  default: foo$1
});

var main = createCommonjsModule(function (module) {
var foo = interopDefault(require$$0);
module.exports = foo * 2;
});

var main$1 = interopDefault(main);

module.exports = main$1;

After

'use strict';

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var require$$0 = createCommonjsModule(function (module) {
module.exports = 21;
});

var main = createCommonjsModule(function (module) {
var foo = require$$0;
module.exports = foo * 2;
});

module.exports = main;

Inline require statements

Source

// main.js
module.exports = function () {
  return require( './multiply' )( 2, require( './foo' ) );
};

// multiply.js
module.exports = function ( a, b ) {
  return a * b;
};

// foo.js
module.exports = 1;

Before

'use strict';

function interopDefault(ex) {
  return ex && typeof ex === 'object' && 'default' in ex ? ex['default'] : ex;
}

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var multiply = createCommonjsModule(function (module) {
module.exports = function ( a, b ) {
  return a * b;
};
});

var multiply$1 = interopDefault(multiply);


var require$$1 = Object.freeze({
  default: multiply$1
});

var foo = createCommonjsModule(function (module) {
module.exports = 1;
});

var foo$1 = interopDefault(foo);


var require$$0 = Object.freeze({
  default: foo$1
});

var main = createCommonjsModule(function (module) {
module.exports = function () {
  return interopDefault(require$$1)( 2, interopDefault(require$$0) );
};
});

var main$1 = interopDefault(main);

module.exports = main$1;

After

'use strict';

function createCommonjsModule(fn, module) {
  return module = { exports: {} }, fn(module, module.exports), module.exports;
}

var require$$1 = createCommonjsModule(function (module) {
module.exports = function ( a, b ) {
  return a * b;
};
});

var require$$0 = createCommonjsModule(function (module) {
module.exports = 1;
});

var main = createCommonjsModule(function (module) {
module.exports = function () {
  return require$$1( 2, require$$0 );
};
});

module.exports = main;

How it works

Basically, when you import a CommonJS module from an ES module, you're no longer importing the module itself but a proxy, which imports the actual CommonJS module but with a prefixed ID (\0commonjs-required:/path/to/cjs-module.js). The proxy handles the ES <-> CJS interop – deciding whether module.exports or exports.default should be the default export, and adding named exports – while the module itself (with the prefixed ID) always just has a single default export which is module.exports. Its dependencies are re-declared as prefixed imports.

Meanwhile, if a module with a prefixed ID (i.e., imported by a proxy or a CommonJS module) isn't a CommonJS module, we need a different kind of interop layer, which imports the underlying ES module and exports-as-default either its default export (if there is one) or the entire namespace. That way, we don't need to reify namespaces for everything that a CommonJS module imports, only ES modules.

An unfortunate side-effect of this is that sourcemaps no longer work for CommonJS modules, because the source code lives in virtual modules, which get excluded. I haven't been able to think of a good solution to this. Honestly, I think it's a small trade-off for more reliable interop and more efficient code, given that sourcemaps are mostly useful when you're debugging your own code.

I think this covers all the bases. @rollup/collaborators and others – would welcome any feedback on this before we commit to this road! Sorry for the rambly and complex explanation.

Rich-Harris added some commits Aug 30, 2016

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2016

@TrySound don't suppose you have any insight into why this is failing on Windows?!

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Aug 31, 2016

Ah, shit. This won't work in its current form. We can't use 'virtual modules' because they get disregarded by other plugins – so code isn't transformed etc. Back to the drawing board

(╯°□°)╯︵ ┻━┻

@Rich-Harris Rich-Harris changed the title BREAKING: interop, take 2 [WIP] BREAKING: interop, take 2 Aug 31, 2016

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Aug 31, 2016

Okay, back in business. I've flipped things around such that the 'real module' contains the actual module contents, while the 'virtual module' (\0commonjs-proxy:/path/to/module.js) is the proxy. The CommonJS module handles its own default interop and all the named exports, but also exports __moduleExports, which proxies (which are now just very thin wrappers) use to pass module.exports through to CommonJS consumers unmolested.

We have to jump through some slightly ugly hoops in order to prevent the entry module (if it's CommonJS) exporting __moduleExports, since it's an internal thing that you don't want messing up the bundle exports – an edge case, but a significant one. Those hoops involve hijacking all the other resolvers to determine whether we're looking at the entry module or not.

This now plays nicely with other plugins, and no longer breaks sourcemap support.

Now if we could just figure out why this isn't building on Windows...

@TrySound

This comment has been minimized.

Copy link
Member

commented Aug 31, 2016

I'll check it out home.

@calvinmetcalf

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2016

idea: only wrap the commonjs modules if there is a top level return

@TrySound

This comment has been minimized.

Copy link
Member

commented Aug 31, 2016

Is top level return valid for commonjs modules?

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Aug 31, 2016

@calvinmetcalf yeah, I've been thinking about this sort of thing – ideally we would be able to turn this...

module.exports = 42;

...into this...

export default 42;

..and this...

exports.foo = 'bar';

...into this...

export var foo = 'bar';

...but that's separate to this issue, which is purely about interop. We'll need to revisit it separately. (Also, I don't want to spend too much time reducing the incentives for CommonJS holdouts to join us in 2016 already...)

@calvinmetcalf

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2016

ah I was more thinking if your doing a big rewrite, now would be the time
to throw that in (and my main goal is for using older libraries with ES6
goals)

On Wed, Aug 31, 2016 at 11:26 AM Rich Harris notifications@github.com
wrote:

@calvinmetcalf https://github.com/calvinmetcalf yeah, I've been
thinking about this sort of thing – ideally we would be able to turn this...

module.exports = 42;

...into this...

export default 42;

..and this...

exports.foo = 'bar';

...into this...

export var foo = 'bar';

...but that's separate to this issue, which is purely about interop. We'll
need to revisit it separately. (Also, I don't want to spend too much time
reducing the incentives for CommonJS holdouts to join us in 2016 already...)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#92 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABE4n2WUszhfOyThoftQZAXGC5SGvG6pks5qlZ0kgaJpZM4Jw3l1
.

@calvinmetcalf

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2016

yes

On Wed, Aug 31, 2016 at 11:56 AM Bogdan Chadkin notifications@github.com
wrote:

Is top level return valid for commonjs modules?


You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#92 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABE4nxREJz2WLLMZQJTDKHW1GUfqh41-ks5qlZdEgaJpZM4Jw3l1
.

TrySound added some commits Aug 31, 2016

}
})
.join( '\n' );
transformBundle ( code ) {

This comment has been minimized.

Copy link
@TrySound

TrySound Aug 31, 2016

Member

Looks like dead and expensive code.

@Rich-Harris Rich-Harris merged commit b35bce4 into master Aug 31, 2016

4 checks passed

continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@Rich-Harris Rich-Harris deleted the interop-take-2 branch Aug 31, 2016

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Aug 31, 2016

Awesome, thanks @TrySound. Have released this is 4.0.0.

@calvinmetcalf

ah I was more thinking if your doing a big rewrite, now would be the time
to throw that in

in my experience a big rewrite is exactly the wrong time to change the behaviour 😀 Better to lock in the bug fixes and modernise the codebase etc then tackle the wishlist. It should actually be a little bit easier to get this stuff working with the current design – I'm thinking we try to convert it the nice way, then bug out and fall back to the current createCommonjsModule approach if we can't (because of early return, assigning exports to some other variable, passing exports to a function, etc...)

@piuccio

This comment has been minimized.

Copy link

commented Sep 3, 2016

Just FYI. I'm using this on a project with both react and preact. Simply upgrading this plugin made the gzip bundle smaller.

68B saved when using preact, 6kB when using react. The reason why there's a big saving with react is because some useless (hope so) exported modules disappeared.

Thanks

@Rich-Harris

This comment has been minimized.

Copy link
Contributor Author

commented Sep 3, 2016

@piuccio awesome! thanks for sharing those numbers. Should fall even further once we can transform CJS modules without wrapping them in createCommonjsModule(...) – have made a decent start on this but it turns out there are some changes needed in Rollup core (which I'm working on now) to fully support it

@piuccio

This comment has been minimized.

Copy link

commented Sep 3, 2016

Glad to know that more bytes can be saved. For correctness I should mention that the savings on react are due mostly to #93, while the savings on preact can certainly be attributed to this PR. Thanks for the great work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.