Implements a CMake linker

[arcanis]

What's the problem this PR addresses?

I'm interested to explore other languages than just JavaScript. Having some experience with C/C++, I'd like to start with those. The de-facto build toolchain is CMake, so I think it's a good start point. Additionally, CMake supports other languages (such as Rust), so there's a chance it could adapt to other languages as well.

How did you fix it?

I've implemented a linker that generates a single CMakeYarn.cmake file containing the location and dependencies for each package from the dependency tree. In a sense, it's very similar to PnP, but much more simple since the resolution is "static" (it only happens when CMake boots).

CMake packages retrieve their dependencies via the special find_yarn_dependencies() function (only available when running CMake via the yarn exec cmake command), and after that they can access any library exported by their dependencies. For example, this is what a very simple CMakeLists.txt would look like. As for dependency listing, it would be just as usual, in package.json (note that the dependency is set in both dependencies and peerDependencies - more on that later).

How does Yarn knows when to use the Node linker?

It uses the languageName field from the package.json. We've supported it for a while for this exact purpose, but it was mostly unused since we didn't have other linker. With this change, the languageName starts to matter.

How does it work with zipped packages?

It doesn't. All packages are extracted on the disk (currently without mutexes, although that could be improved later on).

How does it deal with postinstall scripts?

It doesn't. At first I wondered whether the linker should build the libraries by itself, but then I figured this would have been work done on the wrong layer. Yarn isn't a build tool (our primitives are fairly basic), and since we're generating the CMake tree, it's much cleaner to simply let CMake decide how to build the third-party vendors.

How do you suggest we solve the singleton problem?

For the uninitiated, the singleton problem is how C/C++ programs typically cannot accept multiple versions of the same library, because their symbols would conflict (this is different from JS, where symbols are "scoped" to their file unit).

In this linker, I've used a neat Yarn-only trick in how we interpret peer dependencies: when a package is simultaneously listed as both a regular dependency and a peer dependency, it becomes an "optional peer dependency" - resolved by default if not specified by the parent, but otherwise we use the exact same version.

I think this approach may be a good and low-effort solution to the singleton problems: by default users wouldn't have to care about listing all the transitive dependencies from their dependencies (like what would happen if all dependencies were peer dependencies), but still have the option to decide by themselves which version they want to use when a conflict arises.

The alternative would be to use a SAT solver to try to figure out the "best set of dependencies" that would match the specified constraints, but I'm not convinced this is the right approach:

• I tried it a long time ago, and the constraints are very difficult to express.
• Heuristics are exactly that - heuristics -, and it's difficult to predict which versions will be used.
• It's not clear whether conflicts happen that much, and whether it's really that much of a problem to let the developer pick the version they want to use by themselves (in practice, I suspect it will happen every once in a while, but the fix will just be to use whatever is the latest version and it'll work).

Of course that's a first iteration, and feedback will influence this line of thought.

This linker requires a specific type of manifest. Won't this be a problem?

It remains to be seen - this is a very early experiment, and the main risk is mostly whether we can convince people to use it in the first place, regardless of the exact manifest type. The manifest, by itself, is a problem that's fairly simple to fix, especially when used in conjunction with other Yarn features (like cloning subdirectories from a project, prepack scripts, etc).

Checklist

• I have read the Contributing Guide.

• I have set the packages that need to be released for my changes to be effective.

• I have verified that all automated PR checks pass.