feat(core): adding pubdata to statekeeper and merkle tree #259

koloz193 · 2023-10-18T23:39:02Z

What ❔

Setting the ground for bigger boojum changes. Adding boojum related fields to the data model and types. Extending state tree to have a concept of StateDiffs.

Why ❔

We want to reduce the size of the overall boojum merge, this sets the basis for more extensive changes. These changes are all meant to be non-intrusive.

Checklist

PR title corresponds to the body of PR (we generate changelog entries from PRs).
Tests for the changes have been added / updated.
Documentation comments have been added / updated.
Code has been formatted via zk fmt and zk lint.

codecov · 2023-10-19T00:50:01Z

Codecov Report

Attention: 46 lines in your changes are missing coverage. Please review.

Comparison is base (3cd24c9) 35.41% compared to head (a3b4583) 35.58%.
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #259      +/-   ##
==========================================
+ Coverage   35.41%   35.58%   +0.17%     
==========================================
  Files         524      524              
  Lines       28488    28583      +95     
==========================================
+ Hits        10089    10172      +83     
- Misses      18399    18411      +12

Files	Coverage Δ
core/lib/state/src/lib.rs	`66.66% <ø> (ø)`
core/lib/types/src/block.rs	`85.18% <ø> (-0.53%)`	⬇️
core/lib/types/src/l2_to_l1_log.rs	`100.00% <ø> (ø)`
core/lib/zksync_core/src/genesis.rs	`94.16% <ø> (-0.84%)`	⬇️
...ore/lib/zksync_core/src/metadata_calculator/mod.rs	`87.50% <100.00%> (-0.74%)`	⬇️
.../lib/zksync_core/src/state_keeper/io/seal_logic.rs	`81.78% <ø> (+0.73%)`	⬆️
core/lib/dal/src/blocks_dal.rs	`71.64% <66.66%> (+0.03%)`	⬆️
core/lib/merkle_tree/src/domain.rs	`95.65% <96.15%> (+0.80%)`	⬆️
core/lib/dal/src/models/storage_block.rs	`84.48% <60.00%> (-1.11%)`	⬇️
core/lib/state/src/in_memory.rs	`84.84% <0.00%> (-5.48%)`	⬇️
... and 8 more

... and 57 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

core/lib/dal/src/models/storage_block.rs

mm-zk

(I'll add some more changes after my meetings ;-)

core/lib/dal/src/models/storage_block.rs

core/lib/types/src/block.rs

core/lib/types/src/l2_to_l1_log.rs

core/lib/types/src/fee.rs

core/lib/types/src/commitment.rs

core/lib/types/src/event.rs

core/lib/merkle_tree/src/domain.rs

mm-zk

(I'll add some more comments after my meetings ;-)

core/lib/zksync_core/src/state_keeper/batch_executor/tests/tester.rs

core/lib/zksync_core/Cargo.toml

core/lib/state/src/shadow_storage.rs

core/lib/state/src/witness.rs

core/lib/dal/migrations/20231018205126_l1_batches_boojum_fields.up.sql

core/lib/merkle_tree/src/domain.rs

core/lib/types/src/event.rs

mm-zk · 2023-10-20T12:30:28Z

from my perspective LGTM. Please update PR description (and maybe change it to real PR and not draft ;-)

core/lib/merkle_tree/src/domain.rs

core/lib/state/Cargo.toml

core/lib/types/src/commitment.rs

core/lib/types/src/event.rs

core/lib/zksync_core/src/state_keeper/batch_executor/tests/tester.rs

core/lib/dal/src/blocks_dal.rs

core/lib/dal/src/models/storage_block.rs

core/lib/state/src/rocksdb/mod.rs

core/lib/zksync_core/Cargo.toml

core/lib/types/src/fee.rs

core/lib/types/src/event.rs

core/lib/dal/src/blocks_dal.rs

core/lib/state/src/lib.rs

core/lib/state/src/rocksdb/mod.rs

core/lib/dal/src/blocks_dal.rs

#259 (comment)

🤖 I have created a release *beep* *boop* --- ## [16.2.0](core-v16.1.0...core-v16.2.0) (2023-10-26) ### Features * **basic_witness_producer_input:** Add Basic Witness Producer Input component ([#156](#156)) ([3cd24c9](3cd24c9)) * **core:** adding pubdata to statekeeper and merkle tree ([#259](#259)) ([1659c84](1659c84)) ### Bug Fixes * **db:** Fix root cause of RocksDB misbehavior ([#301](#301)) ([d6c30ab](d6c30ab)) * **en:** gracefully shutdown en waiting for reorg detector ([#270](#270)) ([f048485](f048485)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Release v1.4.2

koloz193 added 7 commits October 18, 2023 19:36

added boojum fields to db and dal

a2fc327

added boojum related fields to types

95648c0

updated batch metadata with new fields

70700bc

extend merkle tree with state diffs

878911a

reverted change to merkle tree size

d7ec27d

nonintrusive boojum updates to get server running

e818cc8

temp dead code tags for boojum fields

027b0d7

mm-zk reviewed Oct 19, 2023

View reviewed changes

core/lib/dal/src/models/storage_block.rs Show resolved Hide resolved

mm-zk reviewed Oct 19, 2023

View reviewed changes

core/lib/dal/src/models/storage_block.rs Show resolved Hide resolved

mm-zk requested changes Oct 19, 2023

View reviewed changes

added method to ReadStorage to pull enum index for storage key

0804f71

mm-zk previously requested changes Oct 19, 2023

View reviewed changes

core/lib/zksync_core/src/state_keeper/batch_executor/tests/tester.rs Outdated Show resolved Hide resolved

core/lib/zksync_core/Cargo.toml Outdated Show resolved Hide resolved

cleaned up and added comments

6b734be

slowli reviewed Oct 20, 2023

View reviewed changes

koloz193 added 2 commits October 20, 2023 16:30

Merge branch 'main' into zk-pubdata-statekeeper-merkletree

2cdd539

updated metrics calls to use vise

5e3119d

koloz193 marked this pull request as ready for review October 20, 2023 22:00

koloz193 requested a review from a team as a code owner October 20, 2023 22:00

added unit test for extracting events

5e175ad

koloz193 requested review from mm-zk and slowli October 21, 2023 00:00

popzxc reviewed Oct 23, 2023

View reviewed changes

perekopskiy requested changes Oct 23, 2023

View reviewed changes

koloz193 added 4 commits October 23, 2023 15:25

added tests for bytecode publication

776f649

updated rocksdb instance to set enum index migration

a001bfe

removed unused package import

73155a7

changed sort to unstable

3c13d34

koloz193 added 4 commits October 23, 2023 15:28

re-added macro for deserialization

1db8962

removed where clause for non null fields

6401257

updated lock file with removed packages

56f45bf

removed new_testing and ported to new

bc6d2fb

perekopskiy reviewed Oct 24, 2023

View reviewed changes

core/lib/dal/src/blocks_dal.rs Outdated Show resolved Hide resolved

core/lib/state/src/lib.rs Outdated Show resolved Hide resolved

core/lib/state/src/rocksdb/mod.rs Outdated Show resolved Hide resolved

This was referenced Oct 24, 2023

feat(core): added stats to track pubdata needed for compression #269

Closed

feat: Port boojum eth-sender changes #293

Merged

koloz193 added 3 commits October 24, 2023 09:04

removed unused fields

d5d7e77

removed leading underscore in param

1d18884

readded save missing enum indices

557e283

koloz193 requested review from perekopskiy and popzxc October 24, 2023 13:18

perekopskiy previously approved these changes Oct 24, 2023

View reviewed changes

core/lib/dal/src/blocks_dal.rs Outdated Show resolved Hide resolved

Merge branch 'main' into zk-pubdata-statekeeper-merkletree

b0c177e

koloz193 dismissed perekopskiy’s stale review via b0c177e October 24, 2023 13:59

left comment with ticket number to remove the fields

2f84ff7

koloz193 requested a review from perekopskiy October 24, 2023 14:06

perekopskiy added 2 commits October 26, 2023 14:39

Merge branch 'main' into zk-pubdata-statekeeper-merkletree

a52e066

Resolve conflicts

a3b4583

perekopskiy approved these changes Oct 26, 2023

View reviewed changes

perekopskiy added this pull request to the merge queue Oct 26, 2023

Merged via the queue into main with commit 1659c84 Oct 26, 2023
25 checks passed

perekopskiy deleted the zk-pubdata-statekeeper-merkletree branch October 26, 2023 13:06

zksync-era-bot mentioned this pull request Oct 26, 2023

chore(main): release core 16.2.0 #324

Merged

toni-calvin referenced this pull request in lambdaclass/zksync-era Aug 8, 2024

Merge pull request #259 from matter-labs/release_v1.4.2

d9baeb8

Release v1.4.2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core): adding pubdata to statekeeper and merkle tree #259

feat(core): adding pubdata to statekeeper and merkle tree #259

koloz193 commented Oct 18, 2023 •

edited

Loading

codecov bot commented Oct 19, 2023 •

edited

Loading

mm-zk left a comment

mm-zk left a comment •

edited

Loading

mm-zk commented Oct 20, 2023

feat(core): adding pubdata to statekeeper and merkle tree #259

feat(core): adding pubdata to statekeeper and merkle tree #259

Conversation

koloz193 commented Oct 18, 2023 • edited Loading

What ❔

Why ❔

Checklist

codecov bot commented Oct 19, 2023 • edited Loading

Codecov Report

mm-zk left a comment

Choose a reason for hiding this comment

mm-zk left a comment • edited Loading

Choose a reason for hiding this comment

mm-zk commented Oct 20, 2023

koloz193 commented Oct 18, 2023 •

edited

Loading

codecov bot commented Oct 19, 2023 •

edited

Loading

mm-zk left a comment •

edited

Loading