all: EIP-compliant verkle tree support #23630

gballet · 2021-09-23T16:09:49Z

This PR adds a --verkle switch that makes geth use verkle trees instead of the MPT. It implements the following:

It adds a new Trie type, namely VerkleTree;
It merges the account and storage tries, using the addressing scheme described in the draft EIP;
It decouples writing to the tree from database state updates;
It only uses the snapshot for its storage - to achieve this, it creates the snapshot at genesis time;
It computes a verkle proof for the block at the end of the block execution, and publishes it with the block in a new VerkleProof field;
It makes it possible to statelessly execute/verify a block just using the data provided in the proof, thus transforming geth into a verkle light client (final proof verification isn't currently performed, due to some required changes in go-verkle, that will be implemented shortly).

This PR doesn't include:

Syncing: right now, only full sync is supported and won't be supported until
A mechanism for switching from the current MPT to verkle trees, the network has to be started from scratch.

This is a rebase of #23563, with some extra fixes.

TODO:

in gasSLoad and other locations, the initial data isn't loaded from the database and an invalid default is used. There should be something to just mark the location as accessed, and then a later load from the db (maybe concurrent in VerkleProofGeneration)
integrate the IPA-based go-verkle when it's ready
verify the proof before executing everything, this is done by adding proof verification in DeserializeVerkleProof
remove the dependency of go-verkle on go-ethereum, this breaks the linter - still broken because of a circular dependency in consensys/gruby.
- common
- rlp
~~add tests for stateless contract execution~~
make sure tests pass when --verkle isn't set

paulmillr · 2021-10-17T02:15:54Z

Does this improve performance in any way? How can we run some benchmarks?

cmd/utils/flags.go

core/genesis.go

consensus/ethash/consensus.go

consensus/misc/eip1559_test.go

gballet · 2021-11-17T09:50:47Z

core/chain_makers.go

+		} else {
+			b.statedb.SetWitness(accesses)
+		}
+	}


simplification here: since the statedb now contains an AccessWitness, one can remove it from the signature of ApplyTransaction and directly merge it there.

gballet · 2021-11-17T09:59:02Z

core/state/snapshot/snapshot.go

@@ -202,6 +203,17 @@ func New(diskdb ethdb.KeyValueStore, triedb *trie.Database, cache int, root comm
 	}
 	if err != nil {
 		if rebuild {
+			if useVerkle {


check if useVerkle is really necessary, since we have IsVerkle handy.

trie/verkle.go

verkle: Implement Trie, NodeIterator and Database ifs Fix crash in TestDump Fix TestDump Fix TrieCopy remove unnecessary traces fix: Error() returned errIteratorEnd in verkle node iterator rewrite the iterator and change the signature of OpenStorageTrie add the adapter to reuse the account trie for storage don't try to deserialize a storage leaf into an account Fix statedb unit tests (#14) * debug code * Fix more unit tests * remove traces * Go back to the full range One tree to rule them all remove updateRoot, there is no root to update store code inside the account leaf fix build save current state for Sina Update go-verkle to latest Charge WITNESS_*_COST gas on storage loads Add witness costs for SSTORE as well Charge witness gas in the case of code execution corresponding code deletion add a --verkle flag to separate verkle experiments from regular geth operations use the snapshot to get data stateless execution from block witness AccessWitness functions Add block generation test + genesis snapshot generation test stateless block execution (#18) * test stateless block execution * Force tree resolution before generating the proof increased coverage in stateless test execution (#19) * test stateless block execution * Force tree resolution before generating the proof * increase coverage in stateless test execution ensure geth compiles fix issues in tests with verkle trees deactivated Ensure stateless data is available when executing statelessly (#20) * Ensure stateless data is available when executing statelessly * Actual execution of a statless block * bugfixes in stateless block execution * code cleanup - Reduce PR footprint by reverting NewEVM to its original signature - Move the access witness to the block context - prepare for a change in AW semantics Need to store the initial values. - Use the touch helper function, DRY * revert the signature of MustCommit to its original form (#21) fix leaf proofs in stateless execution (#22) * Fixes in witness pre-state * Add the recipient's nonce to the witness * reduce PR footprint and investigate issue in root state calculation * quick build fix cleanup: Remove extra parameter in ToBlock revert ToBlock to its older signature fix import cycle in vm tests fix linter issue fix appveyor build fix nil pointers in tests Add indices, yis and Cis to the block's Verkle proof upgrade geth dependency to drop geth's common dep fix cmd/devp2p tests fix rebase issues quell an appveyor warning fix address touching in SLOAD and SSTORE fix access witness for code size touch target account data before calling make sure the proper locations get touched in (ext)codecopy touch all code pages in execution add pushdata to witness remove useless code in genesis snapshot generation testnet: fix some of the rebase/drift issues Fix verkle proof generation in block fix an issue occuring when chunking past the code size fix: ensure the code copy doesn't extend past the code size

upgrade to latest go-verkle update go-verkle to get more fixes simplify code by removing all stateless references (#25) fix verkle proof test by enforcing values alignment to 32 bytes remove unneeded KZG tag fix the stateless test Move AccessWitness into StateDB (#27) * move AccessWitness into StateDB * set Accesses in TxContext constructor * Ensures that a statedb is initialized with a witness * copy AccessWitness in StateDB.Copy. use copied state in miner worker.commit. * remove redundant line Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> Fix contract creation issue

* enable verkle on cancun block: take 2 * fix typo. make unreachable line panic message more clear

* fix: don't add PUSH1-data during contract deployment * remove import cycle in tests (#92)

…ension status (#94)

gballet · 2022-04-06T04:26:25Z

core/genesis.go

@@ -357,6 +381,20 @@ func GenesisBlockForTesting(db ethdb.Database, addr common.Address, balance *big
 	return g.MustCommit(db)
 }

+func DefaultVerkleGenesisBlock() *Genesis {


TODO: remove

* safe gas add * fix build

Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com>

* Remove the code analysis interface to reduce PR footprint * fix unit test

* upgrade branch to use bandersnatch * fix a couple issues * update precomp file

gballet · 2022-07-06T10:46:19Z

Replaced by #25253, closing this one.

gballet force-pushed the verkle-trie-proof-in-block-rebased branch 3 times, most recently from 6562bff to 936e7f1 Compare September 23, 2021 19:40

gballet mentioned this pull request Sep 24, 2021

EIP-compliant verkle tree #23563

Closed

7 tasks

gballet force-pushed the verkle-trie-proof-in-block-rebased branch from dd505fd to be9f9e9 Compare September 30, 2021 19:31

holgerd77 mentioned this pull request Oct 20, 2021

Client: Verkle Trie Integration + Proof Consumption - Tracking Issue ethereumjs/ethereumjs-monorepo#1533

Closed

6 tasks

jwasinger reviewed Oct 27, 2021

View reviewed changes

cmd/utils/flags.go Outdated Show resolved Hide resolved

jwasinger reviewed Oct 27, 2021

View reviewed changes

core/genesis.go Outdated Show resolved Hide resolved

gballet force-pushed the verkle-trie-proof-in-block-rebased branch 4 times, most recently from 9dd8aec to 60b8f90 Compare November 9, 2021 08:18

gballet mentioned this pull request Nov 11, 2021

type inheritance breaks golangci-lint ethereum/go-verkle#129

Closed

gballet commented Nov 12, 2021

View reviewed changes

consensus/ethash/consensus.go Outdated Show resolved Hide resolved

gballet force-pushed the verkle-trie-proof-in-block-rebased branch from 80f090f to afd2166 Compare November 12, 2021 16:05

gballet commented Nov 12, 2021

View reviewed changes

consensus/misc/eip1559_test.go Outdated Show resolved Hide resolved

gballet commented Nov 17, 2021

View reviewed changes

gballet force-pushed the verkle-trie-proof-in-block-rebased branch from ef6b8b7 to fb221aa Compare November 20, 2021 08:40

gballet commented Nov 20, 2021

View reviewed changes

trie/verkle.go Outdated Show resolved Hide resolved

gballet commented Nov 20, 2021

View reviewed changes

trie/verkle.go Outdated Show resolved Hide resolved

gballet mentioned this pull request Nov 23, 2021

onLeaf callback isn't called in (*VerkleTrie).Commit ethereum/go-verkle#133

Closed

gballet and others added 7 commits November 26, 2021 16:30

Upgrade go-verkle to its IPA version (#24)

719bf47

enable verkle on cancun block: take 2 (#28)

97a79f5

* enable verkle on cancun block: take 2 * fix typo. make unreachable line panic message more clear

fix rebase issues

bc06d2c

fix linter issue

e114474

merge undefined instead of panicking (#30)

ef08e51

gballet force-pushed the verkle-trie-proof-in-block-rebased branch from d0bcc59 to e1adb6f Compare March 25, 2022 10:26

ensure no overflow occurs in the chunk touch function

dd6425e

gballet force-pushed the verkle-trie-proof-in-block-rebased branch from e1adb6f to dd6425e Compare March 25, 2022 10:29

gballet added 8 commits March 25, 2022 13:54

load the full coinbase account in reward

de3964f

fix: previous commit broke stateless test

3e1417e

rework the way the gas is charged per code chunk

3068fe9

update go-verkle to grab the fix for #200

64ec2e8

fix: don't add PUSH1-data during contract deployment (#91)

fc9c06d

* fix: don't add PUSH1-data during contract deployment * remove import cycle in tests (#92)

fix: redundant proof-of-absence stems (#93)

8f01a89

test: missing children of the same internal node produce a single ext…

ca661be

…ension status (#94)

ParseNode with commitment parameter (#95)

b9adceb

gballet commented Apr 6, 2022

View reviewed changes

gballet and others added 15 commits April 12, 2022 12:00

review feedback for rebase over kiln (#97)

298d8ce

safe gas add (#98)

439be0c

* safe gas add * fix build

ensure 10-byte addresses are right-aligned in pedersen_hash (#99)

4b36e66

Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com>

Remove the code analysis interface to reduce PR footprint (#100)

336ff36

* Remove the code analysis interface to reduce PR footprint * fix unit test

fix: set IsDeployment=false outside of contract creation (#101)

ac6bc0f

remove trailing line

6856df3

verkle test: count the code chunks in the witness (#102)

5975b44

temp fix: little endian pedersen_hash (#103)

a4838ea

factor the amount of polynomial evaluations in access witness (#106)

d4ac871

reuse polynomial evaluation in account update (#108)

8831288

fix: remove redundant balance touching in access witness (#107)

bf9893d

add github workflow

fbcdf09

reuse address point evaluation in code chunking (#109)

e15eaf7

upgrade branch to use banderwagon (#111)

e3191c3

* upgrade branch to use bandersnatch * fix a couple issues * update precomp file

reduce pr footprint by reverting to codeSegment (#113)

4e2ea43

gballet mentioned this pull request Jul 6, 2022

all: EIP-compliant verkle trees #25253

Closed

gballet closed this Jul 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

all: EIP-compliant verkle tree support #23630

all: EIP-compliant verkle tree support #23630

gballet commented Sep 23, 2021 •

edited

Loading

paulmillr commented Oct 17, 2021

gballet Nov 17, 2021

gballet Nov 17, 2021

gballet Apr 6, 2022

gballet commented Jul 6, 2022

all: EIP-compliant verkle tree support #23630

all: EIP-compliant verkle tree support #23630

Conversation

gballet commented Sep 23, 2021 • edited Loading

paulmillr commented Oct 17, 2021

gballet Nov 17, 2021

Choose a reason for hiding this comment

gballet Nov 17, 2021

Choose a reason for hiding this comment

gballet Apr 6, 2022

Choose a reason for hiding this comment

gballet commented Jul 6, 2022

gballet commented Sep 23, 2021 •

edited

Loading