jsnext:main – should we use it, and what for?

[Rich-Harris]

note: jsnext:main has been superseded by pkg.module, which indicates the location of a file with import/export declarations.

Typically, the pkg.module file will not have other ES2015+ features unless the package explicitly states that it doesn't support older environments — in other words, best practice is still to transpile ES2015+ features other than import/export. More info here.

Original issue follows.

---

This is a follow-up to this Twitter conversation:

Exposition

In CommonJS packages, it's common to have a main field in your package.json that tells Node.js or Browserify (etc) where to find the code:

{
  "name": "some-package",
  "main": "lib/some-package.js"
}

Nowadays, thanks to Babel, it's increasingly common to author in ES6/7 and use a prepublish hook to generate distributable files:

{
  "name": "some-package",
  "main": "dist/some-package.js",
  "scripts": {
    "build": "babel src --out-dir dist",
    "prepublish": "npm run build"
  }
}

So far so good. But the dist file must be CommonJS or UMD, otherwise nothing can use it. That means we're missing an opportunity, because ES6 modules are better in lots of important ways, particularly when it comes to making efficient bundles for use in the browser. (See The Importance of import and export by @benjamn if you need persuading.)

Unfortunately, we can't just create a better UMD block that incorporates ES6 imports and exports, because in engines that don't support ES6 modules (all of them), that's a syntax error. So package authors can't distribute ES6 modules without making their packages incompatible with everything.

Enter jsnext:main

There is a proposed solution to this problem – jsnext:main – which, like main, indicates the package's entry point, except that it uses export instead of module.exports. This is great, because it means that CommonJS/UMD distributable files can co-exist with ES6 distributable files. In the future, once everyone supports ES6 imports and exports, we can simply ditch the CommonJS/UMD stuff.

But does jsnext:main mean 'entry point written with potentially unsupported ES6/7 features', or 'entry point that runs in existing engines, except for the import/export statements'? It's unclear. It sounds as though you can use ES6/7 features, the assumption being that it points to your source code, and that a consumer (such as a bundler) takes responsibility for transpiling it (e.g. with Babel):

{
  "main": "dist/some-package.js",
  "jsnext:main": "src/index.js"
}

But that's problematic. What if src/index.js uses stage 0 features? some-package might have a build process that transpiles those features, but does that mean that all consumers of some-package have to be similarly equipped? (Yes, .babelrc makes that possible, but it's still a weird and brittle process, and it may become rather more complex with Babel 6.)

A better solution: jsnext:main could instead refer to an ES6-module entry point that is otherwise ready to use:

{
  "main": "dist/some-package.cjs.js",
  "jsnext:main": "dist/some-package.es6.js"
}

Prepublish hooks make this stuff very straightforward to set up, and it means that everyone can use your package, and no-one needs to worry about your source code – they're only ever dealing with code that is ready to distribute. It's a very simple solution to the problem of serving CommonJS/UMD and ES6 builds simultaneously.

But the 'jsnext' part of jsnext:main is confusing. I'd suggested changing it to something less ambiguous, but would that just confuse people even more?

Does it go far enough?

@RReverser thinks not, and argues that engines/bundlers/whatever should be able to select different files based on feature detection. I have no idea exactly how that would work but I'm eager to hear more.

In summary

• We badly need a way for package authors to make their packages available as both CommonJS/UMD and ES6. Unless there's a way to transition from one to the other, we'll never be able to move past the legacy module formats, even when most widely-used engines support ES6 modules
• jsnext:main is one possible solution, though it's ambiguous. Personally I think that if we use it, it should be used to point to ready-to-use (other than import/export) distributable code, not source code that still needs to be run through Babel
• We could go even further and have multiple entry points based on feature detection

All thoughts welcome. Thanks

[Rich-Harris]

Also, to save @RReverser reiterating himself, I'll link to some of his earlier comments on the subject: acornjs/acorn#305. There's also some previous discussion here: rollup/rollup#106

[rtsao]

What about the case of non-main entries?

For example, given the following module:

foo
├ package.json
├ main.js
├ bar.js

It seems the following case

var foo = require('foo');

is solved in a straightforward manner with a jsnext:main entry, what about below?

var bar = require('foo/bar')
// or
import bar from 'foo/bar';

Is there a way to specify both ES5 and ES6 versions of all the non-main modules?

[RReverser]

@Rich-Harris Thanks for a thoroughly written summary!

[Rich-Harris]

@rtsao Good question. Since the foo in foo/bar is treated (by Node.js and Browserify, at least) as the directory name (i.e. it looks for node_modules/foo in parent directories), the bar part can only really have one meaning – a file called bar.js at the top of that directory. I don't think the gods of ES6 have yet decided what the spec should say about such things, but as far as Rollup (an ES6 module bundler) goes, it means the exact same thing (in fact Rollup itself imports a file from Acorn in exactly this fashion, per a suggestion from @RReverser :-)

In the past I've argued that requiring individual files in this manner should be considered an anti-pattern – it's brittle (package authors are less free to restructure their code), rarely documented or tested, and makes code less portable. To that we can add a new concern – the difficulty of selecting the right file, if a package author wishes to provide different version for different feature sets (whether that's a straightforward ES5/ES6 dichotomy or something more granularity), is multiplied.

So in summary...

Is there a way to specify both ES5 and ES6 versions of all the non-main modules?

...I don't think there is – but I don't think that's a bad thing, either. Requiring individual files is a hack that we've tolerated up till now because it's the only way to achieve granular imports – something that ES6 modules support by their very nature (or at least will if we can figure out how to distribute them!).

[RReverser]

rarely documented or tested, and makes code less portable

Depends. In the case with Acorn, for example, we provide separate well-documented entry points for generic and loose parser.

Lodash provides documented separate entry points for each particular function so that you can import only needed ones.

There are many cases, in fact, where devs make good use of structured import paths.

[Rich-Harris]

I said 'rarely', not 'never'! I should explain the 'less portable' comment – if I wrote a module that imported acorn/dist/acorn_loose (or acorn/src/loose/index), it would only run or bundle if the runtime/bundler had access to the filesystem. I couldn't (for example) use the same module in the browser, loading Acorn from a CDN. Whereas import { parse } or import { loose as parse } means the same thing everywhere.

I think we should focus on the issue of entry points though – on Twitter you mentioned the possibility of different entry points being selected based on feature detection. Can you outline how that might work? Are there any existing proposals?

[RReverser]

I referred to experiments like @getify's https://featuretests.io/ using which we can easily determine supported features on current runtime. Then, we can either explicitly define set of features needed for our code (or analyze it and build such set of features automatically) and compare two sets (just check whether supported features set is subset of used by our code). Thus we will be able to determine whether it can be executed natively or should we provide transpiled version (assume that modules are transpiled in any case for now).

Moreover, this way we could extend to various entry points by simply providing list of { features: string[], entry: string } items in package.json, where entrys are pre-built for most popular feature sets so that we don't send unneeded polyfills/helpers/etc. to browser.

[getify]

btw, the library that powers https://featuretests.io is: https://github.com/getify/es-feature-tests

That lib (and npm package of the same name) ships not only the library to do the feature testing, but also the testify CLI tool to scan files for what ES6 features are needed. You should be able to scan for the ES6 features needed, then compare that list to what the runtime test results show, and thus decide which file needs to be loaded.

FWIW, the way I kind of do this is:

1. have a sub-directory as an entry point, like bar being a directory instead of bar.js
2. then in the directory, a dummy package.json with nothing but a main: "index.js".
3. At build time, you could use testify to produce the list of features needed, annotate that into package.json.
4. Then index.js performs the runtime tests, and compares to what's in package.json, and decides which of the various files in the sub-directory is appropriate to include and run.

[Rich-Harris]

Thanks @getify, that's awesome – I wasn't aware of featuretests.io, and I love the idea of being able to easily load untranspiled code in e.g. modern browsers. Do you have any example projects where you're using this? Would love to take a look at the setup.

For the bundling case we're considering here, runtime feature detection is obviously a no-go – have you had any thoughts on what this process might look like from an npm run build script's perspective?

I do worry that we're potentially adding overwhelming complexity for package authors, who already have an extraordinarily difficult job. For example, say I'm writing a which depends on your package b. a's build script uses an ES6 module bundler to generate a browser-ready distributable, so it looks in b's package.json for an entry point that flags moduleImport and moduleExport features. But because that entry point of b also uses parameterDestructuring, a now only works in environments that support parameterDestructuring. So at the end of the build process, I probably have to run testify over the result and produce a separate entry point, and the complexity is paid forward to anyone who uses a in their own code. (Or do I pass configuration to my build script that says 'disregard any entry points that require the following features, which we're not going to bother with'?)

Whereas jsnext:main is comparatively easy to grok and concerns itself only with distribution (assuming it doesn't mean jsnext features other than modules), leaving it to package authors to decide which environments to support. Pragmatically, I think an imperfect-but-good-enough solution that has a chance of being adopted (and already has been, e.g. lodash, d3) is preferable to a technically excellent solution that would place a greater burden on package authors – especially since we're talking about an intermediate step on the road to our glorious ES6+ future.

[getify]

For the bundling case we're considering here, runtime feature detection is obviously a no-go

I don't understand all the context, but at glance I don't understand why not?

Why couldn't the bundle you ship be built like I suggest, with tests on the end system to determine which file version to load? You could even put some caching in there, where it caches the results in the package.json file or whatever (similar to how the results are cached in-browser).

[Rich-Harris]

Because if you're creating a browser bundle, all the code has to be present, unless you can assume the presence of a particular module loader that conditionally loads another file asynchronously, with all the extra configuration and complexity that implies. And that code would contain syntax errors for non-modern browsers.

Having multiple entry points (i.e. multiple bundles, for different feature sets) gives developers the option to say 'I'm going to make my life easier and just lob <script src='bundle.legacy.js'> on the page', or to rig up a module loader that is appropriately configured to load different bundles depending on available features.

[getify]

Sorry, I assumed incorrectly this was about use in node, since we were talking about package.json.

FWIW, I do the same sort of stuff (feature test in the browser, conditionally load diff files based on results) and yes I just dynamically load the script files. I use my LABjs loader for that. It doesn't need module loading, since I just load UMD style "module" files as normal scripts.

[mattdesl]

Good discussion.

Here are my issues with jsnext:main

• The name itself is confusing. Is it constantly describing the next version of JavaScript? Or is it just referring to ES2015?
• It isn't very future-proof. Eventually, we will be able to just point "main" to ES6, at which point all the modules using jsnext will seem a little dated.
• Historically, adding tooling-specific config to package.json has not turned out to be a good thing. See "browserify" field, which doesn't play nicely with other bundlers, for example.

I really would love to find a solution that doesn't involve much overhead for module authors. There's some big problems with transpiling, and it's why I still author modules in ES5 instead of using the ES6/jsnext approach.

• Testing and interactive development is hard. Things like npm link, nodemon and browser dev servers break down when you are working on several ES6 modules at once.
• It creates a bad dev experience for consumers. e.g. Very often I end up tweaking and/or reading some node_modules/foo/index.js file from Sublime; but this is not fun with transpiled code.
• Trying to support ES5 and ES6 in tandem adds some new challenges for module authors. These issues don't exist with the current ES5 workflows. A good example is how to cleanly handle foo/bar.js exports in both environments, see here.
• Then there is the topic of polyfills. If you write some code that relies on Promise, Math/Array/RegExp/String methods, etc, then your ES5 distribution will implicitly peer-depend on their polyfills. You can either include the polyfills in your ES5 distribution (more bloat), or force the consumer to add them (more complexity in using the module).
• Babel is going with exports.defaults moving forward which can (in my experience) create a different API usage depending on whether the consumer is ES5 or ES6