Node.js native ESM support

[jgonggrijp]

tl;dr: you have come to the right place to share your experiences with using the underscore@preview release. The text below describes the changes in detail, but you can skip reading it if you just came to report your findings.

I have postponed this feature until now, because having to use .cjs/.mjs extensions, pitfalls regarding state that might not be shared between CJS and ESM users, and the exports field that might break a lot of things all seemed rather rough and daunting to me. I'm biting the bullet now because I noticed that a lot of users found this disappointing, and the native ESM support is marked as "stable" as of Node.js version 15.

The main new feature is that in Node.js version 12 and later, you can now directly do named imports from the package entry point, as well as deep imports from the source modules:

// default and named imports from package entry (monolithic)
import _, { groupBy } from 'underscore';

// deep imports (modular)
import groupBy from 'underscore/modules/groupBy.js';

(The default import was already possible in Node.js 8+ due to the CommonJS interop.)

I had to jump some burning hoops to make this work. The package entry points for Node.js 12+ are newly created custom Rollup bundles: underscore-node-f.cjs, underscore-node.cjs and underscore-node.mjs. The underscore-node.mjs is a thin wrapper that takes its default export from underscore-node.cjs and its named exports from underscore-node-f.cjs. The reason for this is that CommonJS users and ESM users would otherwise see separate _ functions with different sets of mixed in functions, different values of _.templateSettings, different values of partial.placeholder, etcetera. The modules/ directory contains an extra package.json which declares that particular directory to be type: module.

Note that the new entry point bundles for Node.js do not import from modules/ or cjs/ in any way; all the functions are bundled in underscore-node-f.cjs for efficiency. I still recommend to use monolithic imports everywhere except when you are implementing extension functions or you are creating a custom Underscore.

With that out of the way, the exports field adds a mapping so that in theory, you can now also do this in Node.js 12+:

const groupBy = require('underscore/modules/groupBy.js');

For consistency with the other bundles, and looking forward to a future Underscore 2.0 that might be ESM-first, I have renamed the browser-, AMD and Node.js<12-oriented UMD bundle to underscore-umd.js. The package however still contains a copy by the old name underscore.js in order to not break existing import paths.

In theory, this is a non-breaking change; all the modules and bundles that previously existed are still there and all files published to NPM are also represented at least once in the new exports field. The main and modules field are also still there and still point to the regular UMD bundle and modules/index-all.js, respectively.

I figured I needed to test this "from the outside", so I created a separate repository that tests all flavors of imports with several tools on Node.js LTS versions 8-14 against Underscore as an installed NPM package:

https://gitlab.com/jgonggrijp/underscore-import-tests

The CI pipeline can be observed to pass against the changes in this PR over here:

https://gitlab.com/jgonggrijp/underscore-import-tests/-/merge_requests/1

Theory and pipelines aside, the real proof of the pudding is in the tasting. For this reason, I decided to do something slightly unusual. By way of review, I will publish these changes to NPM with the preview tag and invite people to install it and check that their existing setups still work and that they can use the new features.

npm i underscore@preview  # or an equivalent command with another package manager

If that is what brought you here: thanks for being here and please don't hesitate to comment!

[coveralls]

Coverage remained the same at 95.217% when pulling 62c6ad0 on jgonggrijp:node-native-esm into c627e38 on jashkenas:master.

[jgonggrijp]

The Travis build is failing because I upgraded to Rollup 2, which doesn't work with Node.js version 8. I'll address that tomorrow.

Update: fixed.

[jgonggrijp]

Tagged a new preview release (1.13.0-1) because the aliases were missing in the new Node.js native ESM build.

[jgonggrijp]

The latest Travis build failed because the throttle arguments test seems to have become more brittle lately. It happens on master as well so I don't think it is related to the changes in this branch.

The preview releases have been up for three weeks. They were downloaded over 800 times and nobody reported any problems so far. Next week (when I have more time), I'll merge this and release version 1.13.0. For real! 🎈