Support multi-project commands

[guybedford]

This attempts a new sort of spin on the Yarn workspaces concept.

Currently jspm install x -p ./project/path can be used to run a jspm command from a project that is not the current cwd.

So the idea is quite simple - allow the -p flag to accept multiple projects, and that way we can operate on many projects at once, effectively getting the monorepo tools out of a simple primitive.

For example, consider having packages/[pkgname] containing a bunch of packages in a monorepo pattern. Then the following workflows would apply:

1. jspm install -p packages/* would install dependencies in all packages.
2. jspm publish -p packages/* would publish all packages. Some kind of publish flag to only publish new versions could be used to get a proper monorepo workflow out of this.
3. jspm run test -p packages/* would run the package.json "scripts" for "test" in each of the packages.

It could be good to work towards a documentation section covering these techniques comprehensively as we polish the various workflows.

The current limitation with linking the packages together is the following -

In order to link the packages between eachother, we can use:

jspm link packages/* -p packages/*`

What this does, is updates the jspm_packages symlinks in each of the packages to reflect the local versions.

This works fine, but unfortunately has two issues:

1. It would change the package.json to include "dependencies": { "dep": "link:../dep" }, which is not what we would want - rather we would just want the linked package to be represented in the package lock file. I've created a tracking issue for this in jspm link workflow polishing, resolution to be lockfile-specific #2489.
2. It would install as dependencies all packages in eachother, whereas what we really want is a link flag to say link only if it is already a dependency. I've created a tracking issue for this in jspm link -i flag #2490.

Rather than trying to do everything at once in this PR, will tackle the above in separate follow-ups.

This is very much an initial PR to demonstrate the concepts - would be interested to hear what people think further about it too.

[fusionstrings]

Thanks a lot @guybedford for getting started with this. This is a very critical feature I am looking forward to.
I have been using yarn workspaces and I like it a lot. Similarity in implementation would be preferred.

• Running jspm <command> -p ./project/path is nice and now I am confused if I would like jspm install x -p <package-name> more.

• I would definitely wish to not needing to do manual linking and unlinking. I would prefer the yarn way.

  • Install from local if package@version found locally else download from remote.
  • Publishing not required
  • Packages are resolved by package name.

• Being able to resolve to all the files from package even non js or non main ones.

Apart from expecting the solution to be more aligned with yarn implementation, I am also curious to see how monorepo solution shapes up for following use cases.

• non package.json environments, i.e. Deno.
• non node_modules environments i.e. yarn pnp. In this case jspm will not be able to fallback on node_modules folder.

[guybedford]

Thanks for the feedback here @fusionstrings

Points definitely taken on the Yarn-level usability for the final workflow here, as mentioned this is just a first step and further work will be needed.

Being able to resolve to all the files from package even non js or non main ones.

I'm not sure I follow this point, is this about using the jspm resolver API?

Deno support may well be possible for jspm in future. My hope is to support all JS runtimes, provided they have the necessary resolver hook. Yarn Pnp is actually a direct competitor to jspm in terms of changing the resolver. I believe the jspm approach to be superior as you can still see your packages in jspm_packages, but they are symlinked to the global cache so you don't have the disk size bloat. So you get the benefits without the costs of the hacks (eg hacking fs, moving sources to a hidden global cache folder instead of keeping them project-level inspectable / debuggable). Auto install may well be a future feature, but for now I don't see that as a major workflow problem when developing libraries or apps.

[fusionstrings]

Thanks for all the hard work Sir. You inspire and educate me a lot.
With following

Being able to resolve to all the files from package even non js or non main ones.

I meant use cases where one wants to access non javascript files from a package i.e. css, icons etc.

Even though jspm doesn't intend to support non standard js behaviours but valid use cases are where other files are accessed in standard way but from one of the packages.

Like #2461 but the path could be

fetch(new URL('@scope/package/file.anytype', import.meta.url));

May be when using some plugin this can be supported as well

import @scope/package/file.anytype

So I suppose I am referring to jspm resolver API but don't know if this has already been taken care of.