Skip to content

Latest commit

 

History

History
261 lines (178 loc) · 13.4 KB

readme.md

File metadata and controls

261 lines (178 loc) · 13.4 KB

ES2015 module detection in Node.js (June 2016)

What is this about?

The Node TSC has been publicly discussing how to implement ES2015 modules in future versions of Node.js for a while now. There have been several proposals and one has become a draft.

However, some in the community feel that this solution is not cool and ask in-defense-of-.js to reconsider this draft.

By reading this summary, you can ...

  • ... learn about the problems of the implementation
  • ... get a sense for the pro and cons of the given proposals.
  • ... start to discuss this issue on a higher level.

About ES2015 modules

The Involved Parties

TC39

The EcmaScript TC39 - Technical Commitee is a group of about 20 respected members that work on the specification of EcmaScript (aka. JavaScript). They have the power to make change the details of ES2015 modules to work better with Node.js.

Node TSC

The Node Technical Steering Committee is a group of 12 long-term contributors to Node.js that make decisions on the future of Node.js. They decide on how the implementation of ES2016 modules will look like going forward.

Node.js engineers

Once ES2016 modules are a unavoidable part of the Node ecosystem, depending on the implementation chosen the work for every Node.js engineer to update their project will be different. Also developers need to interact with other developers and as such interoperability is a key-concern for them as well.

Frontend engineers

ES2016 modules are supposed to be also implemented by browsers. This has not yet happened but with Chakra being the first to hide it behind a flag it is making quick progress. Frontend developers though also tend to increasingly use npm packages. As such they are interested in better tooling around such things as ES2015 module to regular JavaScript transpilers such as babel.

NPM

NPM Inc. maintains the system at the center of Node.js also called NPM. Their stake in this is that people will in future want to download packages with CommonJS and/or ES2015 modules. NPM also maintains the documentation of the package.json which plays a center role in several proposals.

The Issue

The ES6 module syntax seems like it is just syntax sugar. And if it were then this discussion would be over for a long time already. Unfortunately details of the specification make ES6 modules a on a fundamental level incompatible to CommonJS modules that are used by Node.js. One file has to contain either a CommonJS module or a ES6 module.

NPM has roughly 300,000 packages. Those packages are written as (or compiled to) CommonJS modules. If ES2015 modules are introduced we would suddenly be facing packages that are not compatible with CommonJS and by extension not compatible with node versions that do not support ES2015 modules.

This means we are left with

  • CommonJS & ES2015 module files
  • CommonJS-only/CommonJS+ES2015-mixed/ES2015 packages

The Proposals

1) In-Source pragma (rejected)

This detection would use some string in the file like "use modules"; to identify which module system should be used for this file.

(Rejected due to complexity for tooling and implementation as well as due to the constant code tax - this string would need to be in every file)

more here

2) New file extension (Node.js TSC draft)

The type of the module (CommonJS or ES2015) is detected using the a special file ending. Several file endings have been discussed, in the end .mjs has been identified as the best choice.

more here

3) Content-Sniffing (rejected)

The content would be pre-parsed to identify whether its a CommonJS or ES2015 module.

(Rejected because the detection is fuzzy and could lead to user confusion and problems with tooling)

more here

4) White-/Blacklist patterns in package.json

Patterns written in the package.json specify whether a file is a CommonJS or a ES2015 module. There have been a few variants of this proposal with more or less complex specifications.

more here and here

5) CommonJS or ES2015 switch in package.json

One property in the package.json specifies the module type for all files in the package (sub-packages excluded).

more here

(Note: this is originally variant 4f. Since the implementation consequences are very different it has been named 5.)

Comparisons

Runtime

CPU cycles and Memory when running node. (5=neglectible, 1=cpu-heavy)

Proposal Explanation
Extension 5 Only the file-ending has to be tested, no other overhead
White-/Blacklist 2 The file-name has to be tested against a potentially complicated list
ES2015 switch 4 The first file of a is slower because the package.json needs to be read but every further file after has the same speed.

Implementation

Effort it takes to bring this implementation to Node.js. (5=much, 1=little)

Proposal Explanation
Extension 5 if/else switch, thats it.
White-/Blacklist 3 Significantly more difficult than if/else
ES2015 switch 1 It requires NPM to implement variants of packages into the package system. However: it is important to mention that the Node.js-side should be rather trivial.

Education

Effort to learn the new system. (5=easy, 1=hard)

Proposal Explanation
Extension 5 File endings can be easily studied.
White-/Blacklist 2 Trying to memorize this complex system is hard.
ES2015 switch 4 Mostly straight forward, deployment of variants might be a bit harder to learn than if there were only extensions

Development

Increased development difficulty through the proposal. (5=almost-none, 1=a-lot)

Proposal Explanation
Extension 3 Every developer has to setup his tools to work with the new file extension
White-/Blacklist 1 It is not immediately clear which files are ES2015 and which are CommonJS. The tools might mistake that just as much as developers
ES2015 switch 2 The developer has to learn that there are different modes and has to look it up once per package he is working on.

Legacy Development

Effort for a developer now to use a ES2015-only module. (5=little, 1=much)

Proposal Explanation
Extension 4 If a package decides to switch from .js to .mjs entirely the developers might need to adjust their import statements from require('a-package/a-module.js') to require('a-package/a-module.mjs) if no legacy support is present.
White-/Blacklist 3 It is not immediately clear which files are ES2015 and which are CommonJS. The tools might mistake that just as much as developers.
ES2015 switch 4 Nothing changed but they have to know that old node.js version might not be supported with new packages.

Legacy Package Development

Effort a developer has to take to make her package legacy compatible. (5=little, 1=much)

Proposal Explanation
Extension 3 Always deploy the package with a .mjs and a .js file at the same place (with the same name).
White-/Blacklist 3 The developer has a lot of control over how the packages are built but at the same time she needs to gain control and remember how the system is setup. Human error can easily happen.
ES2015 switch 3 The compiler will pretty much work like anywhere else except that it is easier to specify different dependencies for different variants.

Downloads

Change of a file-size per package. (5=good, 1=bad)

Proposal Explanation
Extension 1 Has to contain every file twice for legacy packages.
White-/Blacklist 1 Has to contain every file twice for legacy packages.
ES2015 switch 5 Only the package is downloaded - no change in download-size.

Tooling

Impact that the change has on tools. (5=simple, 1=complicated)

Proposal Explanation
Extension 4 The .mjs file only needs to be added as file exension.
White-/Blacklist 2 By default all editors should mostly work but it seems unlikely that a specification will be consitently adhered-to by all editors.
ES2015 switch 5 By default all editors should mostly work. To implement the few edge cases would be simple too.

Further considerations

ES2016 modules without package.json

Node.js can be just called like this: $ node some.js in any given folder. The folder does not need to contain a package.json and as consequence all of the proposals that rely on the package.json will not work. Of the given proposals only a specific file-ending (like .mjs) can make sure that modern modules can easily be called from node directly. For the other proposals you might need something like $ node --es6 some.js.

Changing the ES2015 specification

As mentioned a lot earlier: This discussion could have been avoided if the TC39 would be able to fix the parsing incompatibilities in the specification a lot of this discussion would be void.

However: this is unlikely that this will happen because the TC39 worked hard to arrive at this specification and there are good reasons for the incompatibilities. (Note: reference needed)

Possibilities of package variants

For 5) package variants would need to be implemented in NPM. Once NPM packages has the infrastructure to support variants, those variants can give the opporunity to implement other things:

  • Frontend packages could be presented as a variant of a package containing the frontend part of a package.
  • Meteor packages that are now stored on athmosphere because they contain both frontend and backend packages. It would make it easier for Meteor to move to NPM.
  • If we run into future incompatibilities of JavaScript it would be possible to just add a variant type to NPM and we'd be done.
  • Variants could also be used to have other language packages in NPM: Python, C, Go,... that automatically provide Node bindings.

Feedback

If you have any question feel free to chat on gitter.

You can also open an Issue or Pull Request on github.

And you can join this poll:

<script src="https://d3v9r9uda02hel.cloudfront.net/production/static/widgets.js"></script>What ES2016 module implementation would you like to see in Node.js? 🤔