Implementing yarn workspaces 🚀

[evertonfraga]

This PR improves our monorepo package management system by a large margin. There's no need for bootstrap commands anymore.

Tasks:

• Migrate this change to Yarn https://github.com/ethereumjs/ethereumjs-vm/pull/884/files
• Fix test runners that use ethereum/tests

[codecov]

Codecov Report

Merging #900 (ff12905) into ethereum-tests-submodule (d40c8df) will increase coverage by 0.40%.
The diff coverage is n/a.

Flag	Coverage Δ
block	76.35% <ø> (+1.31%)	⬆️
blockchain	77.39% <ø> (ø)
common	91.87% <ø> (ø)
ethash	82.08% <ø> (ø)
tx	100.00% <ø> (+13.95%)	⬆️
vm	?

Flags with carried forward coverage won't be shown. Click here to find out more.

[evertonfraga]

Current status:
Cache not fully restored, so ethereumjs-testing tests not being loaded correctly.

[evertonfraga]

Yarn has a problem loading git submodules from git dependencies. yarnpkg/yarn#1488

It is causing the failures shown here: https://github.com/ethereumjs/ethereumjs-vm/pull/900/checks?check_run_id=1327571708#step:7:191

[holgerd77]

Could you summarize here a bit what problems this is going to solve respectively what properties might be expected from a switch? I've lost a bit track of this.

[evertonfraga]

@holgerd77 sure!

The first reason is performance. There are issues with the combo npm + Lerna, which you seem to experience daily. Yarn workspaces is a native solution to manage dependencies in a multi-project setup, and it's just incredibly fast. A dirty lerna bootstrap takes >40s, while yarn under the same conditions takes only ~2s.

The other reason is to get rid of Lerna. It is highly opinionated and unexpectedly breaks things. I had poor experiences myself. It seems that @alcuadrado and @ricmoo already did the switch and that was my additional inspiration :)

[evertonfraga]

@holgerd77 I forgot to mention that we'll also get rid of this weird behavior:

if you run npm install from the package directory, it will remove all links to the local packages, pulling all dependencies from npm. Run npm install from the root only.

[holgerd77]

@evertonfraga Ok, I wouldn't dare to just merge this after the merge of #953, please give this an explicit note if you think it is ready for review! 😄

[holgerd77]

(there is also still a WIP label on this PR)

[evertonfraga]

@holgerd77 right!

I'd say to only merge this after you're all comfortable with the new submodule functionality, introduced at #953 .

[holgerd77]

@evertonfraga I think everyone had the chance to give reaction on #953 and we can now review and merge here once updated (still open for comments though of course until merged).

[holgerd77]

Short note: let's please wait on merging this in here until after the final releases on Tuesday. I assume there will be no side effects on this but with such a broad base tooling change you'll never know and I'd rather like to play safe here.

[holgerd77]

This can theoretically be integrated now. 😄

[holgerd77]

(@evertonfraga please let us know if you want to do yourself or if you want someone else to take this over)

[evertonfraga]

@holgerd77 @ryanio hey :)

y'all want me to rebase this branch and put it up for review?

[holgerd77]

@evertonfraga Hey ho, happy New Year 🎊, hope you had great holidays!

To your question: Yes. 😄

[Assawal]

This PR improves our monorepo package management system by a large margin. There's no need for bootstrap commands anymore.

Tasks:

• Migrate this change to Yarn https://github.com/ethereumjs/ethereumjs-vm/pull/884/files
• Fix test runners that use ethereum/tests

[ryanio]

Continuing this effort in #1395 with new npm v7 workspaces functionality 👍