Yarn PlugNPlay With ESM

Original Issue: yarnpkg/berry#1149

Original Issue Demo: https://github.com/dmail/yarn-pnp

A resolution for a bug issue raised in Yarn v2 (Berry) @yarnpkg/berry on how to enable ESM module usage within a PlugNPlay (PnP) enabled enviornment. For fun, this repo also demonstrates PlugNPlay using Yarn workspaces (expands upon original issue demo), as well as showcasing Yarn v2's Zero-Install configuration setup.

How to Install

Start a terminal instance:

• Run git clone https://github.com/DaneTheory/yarn-pnp-with-esm

• cd into the cloned directory

• Run yarn

And you're done! Checkout all the awesome new features already available Yarn v2 (Berry). Really great stuff going on here.

How to Run

In the same terminal instance used for the installation process:

• Run yarn start

All output is logged to the console

Preface

As a Monorepo, the packages directory is used to define the root project scope worktree. Each node within the worktree acts as an individual piece of functionality required to build the working workspaces instance as a whole. Check out the root pacakge.json to see how this is done.

PlugNPlay handles node_module resolution. Yarn v2 moves away from relying on a .yarnrc file to handle all general yarn configuration. Instead, .yarnrc.yml is where default configs should go.

Currently, Yarn v2 doesn't recognize the .cjs extension quite like it should. This has to do with Yarn's execution lifecycle altering the entire way node natively goes about resolving require statements. Since Yarn controls the instantiation of any spawned node process (via yarn node /path/to/script.js) so we get the benefits offered with PlugNPlay, we have to "hook" into Yarn at runtime in order to make any changes directly to node.

As it stands now, Yarn generates a .pnp.js file that instructs node on how the heck it's suppossed to go about handling resolves needed to require external/internal modules (dependencies, devDependencies, peerDependencies, etc.).

Beyond performance benefits of PlugNPlay, Yarn v2 (Berry) also emphasizes the monorepo approach to building out your codebase. Yarn now offers many useful protocols (i.e. file:, portal:, exec:, and my personal favorite workspace:) in order to make the dev experience as seemless as possible. These features run largely within a virtual filesytem based context; something node does not natively understand at all. If all that sounds a bit complex, here's a breakdown:

• yarnPath is set in .yarnrc.yml and points to a path to the yarn binary that gets run.

• Upon execution, yarn spawns a series of child processes that handle figuring out denpendency resolution through the project worktree.

• When the resolution ends, the .pnp.js is auto-generated and loaded into node via a require statement that is appended to the NODE_OPTIONS procoess.env variable.

• The node process then executes any module resolutions based on the .pnp.js file acting as it's new map to go about require calls made for modules.

The Problem

Since node cannot resolve anything without first ingesting the .pnp.js dependency map, and the .pnp.js dependency map is generated by yarn which also controls the node process, how do you go about using the newer, non-experimental .mjs/.cjs features offered in node v13.x?

The Solution

One reliable way to go about solving this problem is a very simple two-step process.

First:

• We install esm as a devDependency in our project root.

• Then we change the original yarn node /path/to/script.js script to yarn node -r esm /path/to/script.js

This adds esm to the node process yarn controls. In order to ensure all our packages refereneced within our workspaces always run using the same context, we need to make one more adjustement.

Second:

• We declare a main field in our project root package.json (also known as a manifest file) that points to a file located at the same root level as our manifest.

  • (i.e. in a package.json file located at ~/path/to/my/project/package.json, declare a main key with the value of something equivelent to ./index.js)

• Create the index.js file ensuring it's located exactly where you declared it would be in the main field of the manifest file.

• Go back into your manifest file and create a new dependencies listing for your modules entry point. In this example repo, you can see it's called lib and has the value workspace:*, one of the new protocols previously mentioned.

• Jump back over into the new index.js file at your project root.

• The last and final step is simple but crucial. Refer to the actual file in this repo to see the code. Essentially, we utilize a new api only yarn offers at runtime. We create a new kind of require handler which hooks into the lifecycle yarn goes thorugh in creating the .pnp.js file. We make this new require target our root manifest, search it's dependencies, then resolve our lib module virrtually, finally exporting out the resolved instance of the the virtual require as a whole.

It all may sound really complex, and yea it definately is. However, check out this repo and you'll see the actual implementation is really very straight forward and could not be much simpler to do!

Conclusion

Feel free to open an issue here directly if you have any problems, know of potential improvements, or have any general questions at all. If you like what Yarn v2 (Berry) is shaping up to be, definatelty join in on the discussuon over at @yarnPkg/berry.

Cheers!