Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(prover)!: Protocol Upgrade 1.5.0 #1699

Merged
merged 28 commits into from
Apr 29, 2024
Merged

feat(prover)!: Protocol Upgrade 1.5.0 #1699

merged 28 commits into from
Apr 29, 2024

Conversation

EmilLuta
Copy link
Contributor

This PR lays the groundwork for protocol upgrade from 1.4.2 to 1.5.0. This supports new circuits and changes the job structure of the proof.

The current setup is Basic Witness Generator -> Prover -> Leaf Witness Generator -> Prover -> Node Witness Generator -> Prover -> Recursip Tip Witness Generator -> Prover -> Scheduler Witness Generator -> Prover -> Proof Compression.

Additionally, we have 4 other circuits:

  • TransientStorageSorter -- support for transient storage
  • Secp256r1Verify -- native support for Secp256 (similar to ECRecover, just different params)
  • EIP4844Repack -- EIP4844, now developed as an individual circuit
  • RecursionTip -- New aggregation layer, between node and scheduler

@EmilLuta
Initial commit - add key changes
5e78db6 
This commit updates dependencies and lays the groundwork for all key
generation aspects. With this changes, vks, sks, commitments and
verification hints can be generated from scratch for 1.5.0.
@EmilLuta EmilLuta changed the title feat!(prover): Protocol Upgrade 1.5.0 feat(prover)!: Protocol Upgrade 1.5.0 Apr 16, 2024
@EmilLuta EmilLuta mentioned this pull request Apr 19, 2024
Closed
@EmilLuta
Integrate core<->prover communication
6f45da8 
This commit updates data passing from core to prover.
Namely, Eip4844Blobs (now Eip4844BlobsWrapper) and ensures that
ProofDataHandler and ProverGateway pass messages between them, with the
jobs being stored on prover database.
@mm-zk mm-zk self-requested a review April 22, 2024 15:49
Artemka374
Artemka374 reviewed Apr 22, 2024
View reviewed changes
Copy link
Contributor

@Artemka374 Artemka374 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few nits for now, will take a deeper look tomorrow

checks-config/era.dic Outdated Show resolved Hide resolved
core/node/house_keeper/src/fri_witness_generator_jobs_retry_manager.rs Outdated Show resolved Hide resolved
core/node/house_keeper/src/fri_witness_generator_jobs_retry_manager.rs Outdated Show resolved Hide resolved
prover/prover_fri_types/src/lib.rs Show resolved Hide resolved
mm-zk
mm-zk reviewed Apr 22, 2024
View reviewed changes
etc/env/base/eth_sender.toml Show resolved Hide resolved
etc/env/file_based/general.yaml Show resolved Hide resolved
core/lib/basic_types/src/basic_fri_types.rs Outdated Show resolved Hide resolved
core/node/house_keeper/src/waiting_to_queued_fri_witness_job_mover.rs Show resolved Hide resolved
core/node/house_keeper/src/waiting_to_queued_fri_witness_job_mover.rs Show resolved Hide resolved
prover/witness_generator/src/storage_oracle.rs Show resolved Hide resolved
prover/witness_generator/src/storage_oracle.rs Show resolved Hide resolved
prover/witness_generator/src/storage_oracle.rs Show resolved Hide resolved
prover/witness_generator/src/storage_oracle.rs Show resolved Hide resolved
prover/witness_generator/src/storage_oracle.rs Outdated Show resolved Hide resolved
Artemka374
Artemka374 reviewed Apr 23, 2024
View reviewed changes
Copy link
Contributor

@Artemka374 Artemka374 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have docs somewhere in prover workspace with diagrams etc, it might make sense to update it as well

core/lib/basic_types/src/basic_fri_types.rs Show resolved Hide resolved
core/lib/basic_types/src/basic_fri_types.rs Show resolved Hide resolved
core/lib/prover_interface/src/api.rs Outdated Show resolved Hide resolved
prover/prover_dal/src/fri_witness_generator_dal.rs Outdated Show resolved Hide resolved
prover/prover_dal/src/fri_witness_generator_dal.rs Outdated Show resolved Hide resolved
prover/witness_generator/src/basic_circuits.rs Show resolved Hide resolved
@EmilLuta
Copy link
Contributor Author

Looks good!
Let's not mark this PR as breaking as it bumps major version both for core and prover and they are already bumped

@perekopskiy -- this seems a bit inconsistent, as provers will actually break, whilst before they didn't. Is this how we've always done it?

@EmilLuta EmilLuta changed the title feat(prover)!: Protocol Upgrade 1.5.0 feat(prover): Protocol Upgrade 1.5.0 Apr 24, 2024
@perekopskiy
Copy link
Contributor

@EmilLuta no, previously server breaking PR didn't touch prover directory or we didn't release prover before changes for new protocol version were merged. I thought more and now IMO we should still mark it as breaking. For prover we don't follow major_version == protocol_version anyway and for server we will change major version in release PR

@perekopskiy perekopskiy changed the title feat(prover): Protocol Upgrade 1.5.0 feat(prover)!: Protocol Upgrade 1.5.0 Apr 24, 2024
perekopskiy
perekopskiy previously approved these changes Apr 24, 2024
View reviewed changes
@Shamar
Copy link

Shamar commented Apr 24, 2024 via email

RomanBrodetski
RomanBrodetski previously approved these changes Apr 24, 2024
View reviewed changes
@EmilLuta EmilLuta mentioned this pull request Apr 25, 2024
Closed
6 tasks
@EmilLuta EmilLuta added this pull request to the merge queue Apr 29, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 29, 2024
@EmilLuta EmilLuta added this pull request to the merge queue Apr 29, 2024
@RomanBrodetski RomanBrodetski removed this pull request from the merge queue due to a manual request Apr 29, 2024
@RomanBrodetski RomanBrodetski merged commit 6a557f7 into main Apr 29, 2024
43 checks passed
@RomanBrodetski RomanBrodetski deleted the evl-1-5-0-prover-integration branch April 29, 2024 10:33
This was referenced Apr 29, 2024
Merged
Merged
RomanBrodetski pushed a commit that referenced this pull request Apr 30, 2024
@zksync-era-bot @perekopskiy
chore(main): release core 24.0.0 (#1758)
e9d41a6 
🤖 I have created a release *beep* *boop*
---


##
[24.0.0](core-v23.1.0...core-v24.0.0)
(2024-04-30)


### ⚠ BREAKING CHANGES

* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
* shared bridge
([#298](#298))

### Features

* **api:** Allow granular max response size config
([#1642](#1642))
([83c4034](83c4034))
* **api:** Improve logging for API server
([#1792](#1792))
([50fbda5](50fbda5))
* **api:** Track params for RPC methods
([#1673](#1673))
([1a34c8b](1a34c8b))
* **configs:** move ecosystem contracts to contracts
([#1606](#1606))
([9d52180](9d52180))
* EcPairing precompile as system contract
([#1761](#1761))
([3071622](3071622))
* Include create2 factory in genesis
([#1775](#1775))
([597280b](597280b))
* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
([6a557f7](6a557f7))
* **prover:** remove redundant config fields
([#1787](#1787))
([a784ea6](a784ea6))
* shared bridge
([#298](#298))
([8c3478a](8c3478a))
* **tree:** Improved tree pruning
([#1532](#1532))
([bcb192c](bcb192c))
* **vm-runner:** implement VM runner storage layer
([#1651](#1651))
([543f9e9](543f9e9))
* **vm:** Extend bootloader memory in the new version
([#1807](#1807))
([f461b28](f461b28))


### Bug Fixes

* **api:** Fix extra DB connection acquisition during tx submission
([#1793](#1793))
([9c6ed83](9c6ed83))
* **en:** correct en config vars
([#1809](#1809))
([d340fbc](d340fbc))
* **en:** Remove duplicate reorg detector
([#1783](#1783))
([3417941](3417941))
* **kl-factory:** base token ether tests
([#1746](#1746))
([6cf14a0](6cf14a0))
* **prover:** Fix panics if prover's config is not ready
([#1822](#1822))
([21d90d7](21d90d7))
* **snapshots_creator:** Remove snapshots during reverts
([#1757](#1757))
([8d587fd](8d587fd))
* **types:** Add LegacyMixedCall
([#1773](#1773))
([2b236fe](2b236fe))
* v23 api typo
([#1821](#1821))
([a11fa86](a11fa86))
* Weaker assert for protocol version for operations
([#1800](#1800))
([11898c2](11898c2))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: perekopskiy <mikeson.dp@gmail.com>
github-merge-queue bot pushed a commit that referenced this pull request May 6, 2024
@zksync-era-bot
chore(main): release prover 14.0.0 (#1772)
f6e63e1 
🤖 I have created a release *beep* *boop*
---


##
[14.0.0](prover-v13.0.0...prover-v14.0.0)
(2024-05-06)


### ⚠ BREAKING CHANGES

* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
* shared bridge
([#298](#298))

### Features

* **config:** Wrap sensitive urls
([#1828](#1828))
([c8ee740](c8ee740))
* **Prover CLI:** `status batch` command
([#1638](#1638))
([3fd6d65](3fd6d65))
* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
([6a557f7](6a557f7))
* **prover:** remove redundant config fields
([#1787](#1787))
([a784ea6](a784ea6))
* shared bridge
([#298](#298))
([8c3478a](8c3478a))
* **vm-runner:** implement VM runner storage layer
([#1651](#1651))
([543f9e9](543f9e9))


### Bug Fixes

* **en:** Remove duplicate reorg detector
([#1783](#1783))
([3417941](3417941))
* **prover:** Use all 1.5.0 groups for nodes
([#1851](#1851))
([70178e5](70178e5))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Deniallugo pushed a commit that referenced this pull request May 7, 2024
@zksync-era-bot @Deniallugo
chore(main): release prover 14.0.0 (#1772)
cd912af 
🤖 I have created a release *beep* *boop*
---


##
[14.0.0](prover-v13.0.0...prover-v14.0.0)
(2024-05-06)


### ⚠ BREAKING CHANGES

* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
* shared bridge
([#298](#298))

### Features

* **config:** Wrap sensitive urls
([#1828](#1828))
([c8ee740](c8ee740))
* **Prover CLI:** `status batch` command
([#1638](#1638))
([3fd6d65](3fd6d65))
* **prover:** Protocol Upgrade 1.5.0
([#1699](#1699))
([6a557f7](6a557f7))
* **prover:** remove redundant config fields
([#1787](#1787))
([a784ea6](a784ea6))
* shared bridge
([#298](#298))
([8c3478a](8c3478a))
* **vm-runner:** implement VM runner storage layer
([#1651](#1651))
([543f9e9](543f9e9))


### Bug Fixes

* **en:** Remove duplicate reorg detector
([#1783](#1783))
([3417941](3417941))
* **prover:** Use all 1.5.0 groups for nodes
([#1851](#1851))
([70178e5](70178e5))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thomas-nguy thomas-nguy thomas-nguy left review comments

@perekopskiy perekopskiy perekopskiy approved these changes

@RomanBrodetski RomanBrodetski RomanBrodetski left review comments

@slowli slowli Awaiting requested review from slowli

@mm-zk mm-zk Awaiting requested review from mm-zk

@Artemka374 Artemka374 Awaiting requested review from Artemka374

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@EmilLuta @perekopskiy @Shamar @RomanBrodetski @slowli @thomas-nguy @Artemka374 @mm-zk