Feature Request: Variable Substitution in package.json

[jordwalke]

Feature Request: Variable Substitution in package.json

The ability to dynamically determine packages based on variable substitution.

While there could be many great use cases for variable substitution, I will address the two most important variables that I've encountered, platform and cpu. The syntax for substitution within dependencies is .(variableName), which I'll explain later.

What variable substitution does:

"devDependencies": {
   "rebel": "ourGitUrl#0.4.4.(anyOs).(anyCpu)"
}

Would result in yarn interpreting the package.json as if it had been written as:

"devDependencies": {
   "rebel": "ourGitUrl#0.4.4.darwin.64"
}

If you are on Mac OS, 64 bit.

How It Benefits Yarn Users:

Blazing fast installs for what would normally take as much as 30 minutes on npm.

This is awesome because package authors can configure travis builds to automatically upload successful builds to ourGitUrl#0.4.4-.(anyOs).(anyCpu) where platform/cpu matches the platform/architecture that travis built on. So, that package.json snippet above is actually something that package authors could commit and publish, knowing that it will work everywhere when people begin to depend on their packages. There is no way to do this with yarn or npm today that I know of (not even with the os/cpu fields).

I imagine being able to run yarn like:

Giving The End User Controls/Overrides:

You can imagine allowing this usage of yarn, in order to include prebuilts for the exact architecture you want:

yarn install --vars -platform=linux -cpu=64

How Variable Substitution Plays Nicely

But what about OSs/CPUs that we haven't built for? What about people who use the npm client to install a package with yarn variables in it? The proposed variable syntax is intentionally chosen to be .(varName), because .(foo) is valid syntax for git tags, even if it's not substituted with anything.

So the solution, is that in addition to having travis publish #0.4.4-.darwin.64 that contains the prebuilt version of the package for darwin, we also publish the not-prebuilt version of the package from travis with #0.4.4-.(anyOs).(anyCpu) - yes, the literal string "#0.4.4-.(anyOs).(anyCpu)" including parens. That means npm clients can still depend on our packages that contain yarn specific substitution features, it's just that they end up rebuilding the package from source. It's a nice incentive to use yarn, without breaking compatibility with npm, and hopefully npm can adopt the convention if people find it useful.

Remaining questions:

• This proposal has npm compatibility via full rebuilds and with yarn we can substitute .(os)/.(cpu) across all packages, or no packages at all, but how do we substitute variables for only the subset of packages that have a tag for that os/cpu? This could be solved much later, but I imagine that the yarn client could first try substituting os/cpu and then if no git tag is found, it could fall back to the literal string #4.0.0.(anyOs).(anyCpu), and then finally #4.0.0.
• yarn git tag version constraints should also work here, although these would not be compatible with npm (sadly), but the npm incompatibility is only due to npm not supporting git tag ranges, and that has nothing to do with variable substitution.

"dependencies": {
   "rebel":
      "gitUrl#0.4.(os).(cpu) - gitUrl#0.5.(os).(cpu)"
}

[jordwalke]

@yunxing can you add this to Reason Milestones?
I can't edit it for some reason. https://github.com/yarnpkg/yarn/milestone/2

[Daniel15]

@jordwalke - I added it for you!

[jordwalke]

Thank you.

[sebmck]

This is a substantial feature request and needs to be opened on the yarnpkg/rfcs repo.

[muelli]

this is probably tracked here now: yarnpkg/rfcs#16