EVM/Monorepo: Verkle Decoupling

[holgerd77]

This PR works towards reducing the EVM bundle size by reducing the Verkle coupling.

The one direct casting to StatelessVerkleStateManager is removed by using the StateManager interface here by adding the Verkle methods as optional methods there. I would generally like to expand on this since it reduces the direct coupling to the SVSM class.

Additionally an interface AccessWitness is added (also to Common), which we can use instead of the direct import.

And, third measure: constants and methods not depending on Verkle cryptography have been moved to a dedicated verkle module in Util. This should make sense as well and was necessary in the EVM case, since it allowed to remove the @ethereumjs/verkle dependency completely.

---

I would think the above measures should "do the job". I could not yet prove it though. When running the esbuild command from #3446 (comment) build size stayed unexpectedly on the 2.6 MB level (for EVM).

Some googling brought up that esbuild just seems to not properly tree shake on unused imports from external modules, see e.g. evanw/esbuild#1794. 🤔

I discovered the --analyze=verbose flag along (full command: npm run build && esbuild --bundle 10EVM.mjs --outfile=out1.mjs --analyze=verbose) - generally useful - which helps to see where things are still drawn in:

   ├ ../verkle/dist/esm/node/types.js ────────────────────────────────────────────────────── 226b ──── 0.0%
   │  └ ../verkle/dist/esm/node/index.js
   │     └ ../verkle/dist/esm/index.js
   │        └ ../statemanager/dist/esm/statelessVerkleStateManager.js
   │           └ ../statemanager/dist/esm/index.js
   │              └ dist/esm/evm.js
   │                 └ dist/esm/index.js
   │                    └ 10EVM.mjs

So this is for the @ethereumjs/verkle dependency, which should not be there anymore, and which brought me to googling for the esbuild issue.

I then started thinking about using another bundler (so: if esbuild might generally not deliver valid tree shaking results) trying rollup with the following:

npm i -g rollup
npm i -g @rollup/plugin-node-resolve
rollup --format=cjs --file=out1.mjs --plugin=node-resolve -- 10EVM.mjs

This gives a bunch of other warning and (at least) one error (where things break).

Not fully sure if it's worth to resume on that path (or try e.g. another bundler).

Maybe our tree shaking & bundling expert @roninjin10 can have a comment! 🙂
rollup --format=cjs --file=out1.mjs --plugin=node-resolve -- 10EVM.mjs

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.82%. Comparing base (d24ca11) to head (e63ecf0).
Report is 29 commits behind head on master.

❗ Current head e63ecf0 differs from pull request most recent head 552b7d2

Please upload reports for the commit 552b7d2 to get more accurate results.

Additional details and impacted files

Flag	Coverage Δ
blockchain	?
common	?
genesis	99.98% <ø> (?)
statemanager	?
util	?
wallet	87.25% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

[roninjin10]

This generally won't affect tree shaking. TypeScript types are always removed even if they are values or classes as long as they are used as a type as long as we use import type syntax

[roninjin10]

@holgerd77 wierd rollup didn't work but since you are already using vitest using vite is likely the easiest. I added a bundle analyzer setup you can use in this pr #3463

[g11tech]

refactor looks superclean and modular. don't have insight into tree shaking however as of now, so for that will delegate to experts :)

[holgerd77]

refactor looks superclean and modular. don't have insight into tree shaking however as of now, so for that will delegate to experts :)

This is still in the pre-tree-shaking era, so just removing dependencies so that they are not there. 😂

[gabrocheleau]

Minor note on a few of these helpers. They were initially named quite minimally getKey rather than getVerkleTreekey because the "verkle" was obvious from it being imported from the verkle package. I feel like we should likely rename these helpers more explicitly as "verkle" since they are now exported in utils (getKey could be so many things). We could perhaps do the same with VERSION_LEAF_KEY => VERKLE_VERSION_LEAF_KEY

[acolytec3]

Minor note on a few of these helpers. They were initially named quite minimally getKey rather than getVerkleTreekey because the "verkle" was obvious from it being imported from the verkle package. I feel like we should likely rename these helpers more explicitly as "verkle" since they are now exported in utils (getKey could be so many things). We could perhaps do the same with VERSION_LEAF_KEY => VERKLE_VERSION_LEAF_KEY

100% agree.

[gabrocheleau]

Looks great! as noted on the call, we will leave the verkle namespacing updates to a future call in order not to interfere with pending verkle work