diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index f3b7f67f424..1b71926ff56 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -31,9 +31,21 @@ # Ethrex for L2 chains - [Introduction](./l2/introduction.md) -- [Deploy an L2](./l2/deploy.md) +- [Deploy an L2](./l2/deployment/README.md) + - [Overview](./l2/deployment/overview.md) + - [Deploying a vanilla ethrex L2](./l2/deployment/vanilla.md) + - [Deploying a validium ethrex L2](./l2/deployment/validium.md) + - [Deploying a based ethrex L2](./l2/deployment/based.md) + - [Synchronous Composability PoC](./l2/deployment/synchronous_composability_poc.md) +- [Run a prover](./l2/deployment/prover/README.md) + - [Overview](./l2/deployment/prover/overview.md) + - [Run an ethrex SP1 prover](./l2/deployment/prover/sp1.md) + - [Run an ethrex RISC0 prover](./l2/deployment/prover/risc0.md) + - [Run an ethrex TDX prover](./l2/deployment/prover/tee.md) + - [Run multiple provers](./l2/deployment/prover/multi-prover.md) - [Monitoring and metrics](./l2/monitoring.md) - [Admin server](./l2/admin.md) +- [Ethrex <> Aligned](./l2/aligned_integration.md) - [Architecture](./l2/architecture/README.md) - [Overview](./l2/architecture/overview.md) - [Sequencer](./l2/architecture/sequencer.md) @@ -45,9 +57,13 @@ - [Withdraw assets](./l2/interacting/withdraw.md) - [Connect a wallet](./l2/interacting/wallet.md) - [Deploy a contract](./l2/interacting/deploy_contracts.md) + - [Blockscout for ethrex L2](./l2/interacting/blockscout.md) + - [L2 Hub](./l2/interacting/l2_hub.md) - [Fundamentals](./l2/fundamentals/README.md) - [State diffs](./l2/fundamentals/state_diffs.md) - [Block vs StateDiff](./l2/fundamentals/block_vs_state_diff_measurements.md) + - [Data availability](./l2/fundamentals/data_availability.md) + - [Execution witness](./l2/fundamentals/execution_witness.md) - [Deposits](./l2/fundamentals/deposits.md) - [Withdrawals](./l2/fundamentals/withdrawals.md) - [Smart contracts](./l2/fundamentals/contracts.md) @@ -76,7 +92,10 @@ - [L2](./developers/l2/introduction.md) - [Ethrex L2 as local development mode](./developers/l2/dev-mode.md) - [Integration tests](./developers/l2/integration-tests.md) + - [Running the Prover](./developers/l2/prover.md) - [Generate blobs for the state reconstruction test](./developers/l2/state-reconstruction-blobs.md) +- [Prover](./prover/prover.md) + - [Guest program](./prover/guest_program.md) - [Rich Accounts](./developers/rich-accounts.md) - [Debugging solidity with ethrex](./vm/levm/debug.md) - [Re-execute Ethereum with ethrex](./ethrex_replay/ethrex_replay.md) diff --git a/docs/developers/l2/prover.md b/docs/developers/l2/prover.md new file mode 100644 index 00000000000..009011dc192 --- /dev/null +++ b/docs/developers/l2/prover.md @@ -0,0 +1,154 @@ +# Running the Prover + +This guide provides instructions for setting up and running the Ethrex L2 prover for development and testing purposes. + +## Dependencies + +Before you begin, ensure you have the following dependencies installed: + +- [RISC0](https://dev.risczero.com/api/zkvm/install) + 1. `curl -L https://risczero.com/install | bash` + 2. `rzup install` +- [SP1](https://docs.succinct.xyz/docs/sp1/introduction) + 1. `curl -L https://sp1up.succinct.xyz | bash` + 2. `sp1up --version 5.0.8` +- [SOLC](https://docs.soliditylang.org/en/latest/installing-solidity.html) + +After installing the toolchains, a quick test can be performed to check if we have everything installed correctly. + +### L1 block proving + +ethrex-prover is able to generate execution proofs of Ethereum Mainnet/Testnet blocks. An example binary was created for this purpose in `crates/l2/prover/bench`. Refer to its README for usage. + +## Dev Mode + +To run the blockchain (`proposer`) and prover in conjunction, start the `Prover`, use the following command: + +```sh +make init-prover- # optional: GPU=true +``` + +### Run the whole system with the prover - In one Machine + +> [!NOTE] +> Used for development purposes. + +1. `cd crates/l2` +2. `make rm-db-l2 && make down` + - It will remove any old database, if present, stored in your computer. The absolute path of SQL is defined by [datadir](https://docs.rs/dirs/latest/dirs/fn.datadir.html). +3. `make init` + - Make sure you have the `solc` compiler installed in your system. + - Init the L1 in a docker container on port `8545`. + - Deploy the needed contracts for the L2 on the L1. + - Start the L2 locally on port `1729`. +4. In a new terminal → `make init-prover- # GPU=true`. + +After this initialization we should have the prover running in `dev_mode` → No real proofs. + +## GPU mode + +**Steps for Ubuntu 22.04 with Nvidia A4000:** + +1. Install `docker` → using the [Ubuntu apt repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) + - Add the `user` you are using to the `docker` group → command: `sudo usermod -aG docker $USER`. (needs reboot, doing it after CUDA installation) + - `id -nG` after reboot to check if the user is in the group. +2. Install [Rust](https://www.rust-lang.org/tools/install) +3. Install [RISC0](https://dev.risczero.com/api/zkvm/install) +4. Install [CUDA for Ubuntu](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_local) + - Install `CUDA Toolkit Installer` first. Then the `nvidia-open` drivers. +5. Reboot +6. Run the following commands: + +```sh +sudo apt-get install libssl-dev pkg-config libclang-dev clang +echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc +echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc +``` + +### Run the whole system with a GPU Prover + +Two separate machines are recommended for running the `Prover` and the `sequencer` to avoid resource contention. However, for development, you can run them in two separate terminals on the same machine. + +- **Machine 1 (or Terminal 1)**: For the `Prover` (GPU is recommended). +- **Machine 2 (or Terminal 2)**: For the `sequencer`/L2 node. + +1. **`Prover`/`zkvm` Setup** + 1. `cd ethrex/crates/l2` + 2. You can set the following environment variables to configure the prover: + - `PROVER_CLIENT_PROVER_SERVER_ENDPOINT`: The address of the server where the client will request the proofs from. + - `PROVER_CLIENT_PROVING_TIME_MS`: The amount of time to wait before requesting new data to prove. + 3. To start the `Prover`/`zkvm`, run: + ```sh + make init-prover- # optional: GPU=true + ``` + +2. **`ProofCoordinator`/`sequencer` Setup** + 1. `cd ethrex/crates/l2` + 2. Create a `.env` file with the following content: + ```env + # Should be the same as ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY + ETHREX_DEPLOYER_L1_PRIVATE_KEY= + # Should be the same as ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_DEPLOYER_L1_PRIVATE_KEY + ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY= + # Should be the same as ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY and ETHREX_DEPLOYER_L1_PRIVATE_KEY + ETHREX_COMMITTER_L1_PRIVATE_KEY= + # Should be different from ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY + ETHREX_PROOF_COORDINATOR_L1_PRIVATE_KEY= + # Used to handle TCP communication with other servers from any network interface. + ETHREX_PROOF_COORDINATOR_LISTEN_ADDRESS=0.0.0.0 + # Set to true to randomize the salt. + ETHREX_DEPLOYER_RANDOMIZE_CONTRACT_DEPLOYMENT=true + # Set to true if you want SP1 proofs to be required + ETHREX_L2_SP1=true + # Check the if the verification contract is present on your preferred network. Don't define this if you want it to be deployed automatically. + ETHREX_DEPLOYER_SP1_VERIFIER_ADDRESS=
+ # Set to true if you want proofs to be required + ETHREX_L2_RISC0=true + # Check the if the contract is present on your preferred network. You shall deploy it manually if not. + ETHREX_DEPLOYER_RISC0_VERIFIER_ADDRESS=
+ # Set to any L1 endpoint. + ETHREX_ETH_RPC_URL= + ``` + 3. `source .env` + +> [!NOTE] +> Make sure to have funds, if you want to perform a quick test `0.2[ether]` on each account should be enough. + +- `Finally`, to start the `proposer`/`l2 node`, run: + - `make rm-db-l2 && make down` + - `make deploy-l1 && make init-l2` (if running a risc0 prover, see the next step before invoking the L1 contract deployer) + +- If running with a local L1 (for development), you will need to manually deploy the risc0 contracts by following the instructions [here](https://github.com/risc0/risc0-ethereum/tree/main/contracts/script). +- For a local L1 running with ethrex, we do the following: + + 1. clone the risc0-ethereum repo + 1. edit the `risc0-ethereum/contracts/deployment.toml` file by adding + ```toml + [chains.ethrex] + name = "Ethrex local devnet" + id = 9 + ``` + 1. export env. variables (we are using an ethrex's rich L1 account) + ```bash + export VERIFIER_ESTOP_OWNER="0x4417092b70a3e5f10dc504d0947dd256b965fc62" + export DEPLOYER_PRIVATE_KEY="0x941e103320615d394a55708be13e45994c7d93b932b064dbcb2b511fe3254e2e" + export DEPLOYER_ADDRESS="0x4417092b70a3e5f10dc504d0947dd256b965fc62" + export CHAIN_KEY="ethrex" + export RPC_URL="http://localhost:8545" + + export ETHERSCAN_URL="dummy" + export ETHERSCAN_API_KEY="dummy" + ``` + the last two variables need to be defined with some value even if not used, else the deployment script fails. + 1. cd into `risc0-ethereum/` + 1. run the deployment script + ```bash + bash contracts/script/manage DeployEstopGroth16Verifier --broadcast + ``` + 1. if the deployment was successful you should see the contract address in the output of the command, you will need to pass this as an argument to the L2 contract deployer, or via the `ETHREX_DEPLOYER_RISC0_VERIFIER_ADDRESS=
` env. variable. + if you get an error like `risc0-ethereum/contracts/../lib/forge-std/src/Script.sol": No such file or directory (os error 2)`, try to update the git submodules (foundry dependencies) with `git submodule update --init --recursive`. + +## Configuration + +Configuration is done through environment variables or CLI flags. +You can see a list of available flags by passing `--help` to the CLI, or checkout [CLI](../../CLI.md). diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md index 5c2b4eab588..fcfe7f05e6b 100644 --- a/docs/getting-started/README.md +++ b/docs/getting-started/README.md @@ -103,7 +103,7 @@ For other CPU architectures, see the [releases page](https://github.com/lambdacl - **Interested in deploying your own L2?** - See [L2 rollup deployment](../l2/deploy.md) for launching your own rollup, deploying contracts, and interacting with your L2. + See [L2 rollup deployment](../l2/deployment/overview.md) for launching your own rollup, deploying contracts, and interacting with your L2. - **Looking to contribute or develop?** diff --git a/docs/l2/architecture/aligned_mode.md b/docs/l2/aligned_integration.md similarity index 98% rename from docs/l2/architecture/aligned_mode.md rename to docs/l2/aligned_integration.md index 32b350398d4..d936078ec4b 100644 --- a/docs/l2/architecture/aligned_mode.md +++ b/docs/l2/aligned_integration.md @@ -22,7 +22,7 @@ make build-prover- # optional: GPU=true This will generate the SP1 ELF program and verification key under: - `crates/l2/prover/src/guest_program/src/sp1/out/riscv32im-succinct-zkvm-elf` -- `crates/l2/prover/src/guest_program/src/sp1/out/riscv32im-succinct-zkvm-vk` +- `crates/l2/prover/src/guest_program/src/sp1/out/riscv32im-succinct-zkvm-vk-u32` ### 2. Deploying L1 Contracts @@ -290,7 +290,7 @@ INFO ethrex_l2::sequencer::l1_proof_verifier: Batches verified in OnChainPropose - Sends proofs to the **Aligned Batcher** instead of the `OnChainProposer` contract. - Tracks the last proof sent using the rollup store. -![Proof Sender Aligned Mode](../img/aligned_mode_proof_sender.png) +![Proof Sender Aligned Mode](./img/aligned_mode_proof_sender.png) ### Proof Verifier @@ -298,7 +298,7 @@ INFO ethrex_l2::sequencer::l1_proof_verifier: Batches verified in OnChainPropose - Monitors whether the next proof has been aggregated by Aligned. - Once verified, collects all already aggregated proofs and triggers the advancement of the `OnChainProposer` contract by sending a single transaction. -![Aligned Mode Proof Verifier](../img/aligned_mode_proof_verifier.png) +![Aligned Mode Proof Verifier](./img/aligned_mode_proof_verifier.png) ### OnChainProposer diff --git a/docs/l2/architecture/overview.md b/docs/l2/architecture/overview.md index 72ccdd814c2..b9d1d8df575 100644 --- a/docs/l2/architecture/overview.md +++ b/docs/l2/architecture/overview.md @@ -139,29 +139,30 @@ The mechanism for withdrawing funds from L2 back to L1 is explained in detail in ### Batch Commitment -An L2 batch commitment is the hash of the following things: +An L2 batch commitment contains: - The new L2 state root. -- The state diff hash or polynomial commitments, depending on whether we are using calldata or blobs. -- The Withdrawal logs merkle root. +- The latest block's hash +- The KZG versioned hash of the blobs published by the L2 +- The rolling hash of the processed privileged transactions +- The Merkle root of the withdrawal logs -The public input to the proof is then the hash of the previous batch commitment and the new one. +These are committed as public inputs of the zk proof that validates a new L2 state. ## L1 contract checks ### Commit transaction -For the `commit` transaction, the L1 verifier contract receives the following things from the sequencer: - -- The L2 batch number to be commited. -- The new L2 state root. -- The Withdrawal logs merkle root. -- The state diffs hash or polynomial commitment scheme accordingly. +For the `commit` transaction, the L1 verifier contract receives the batch commitment, as defined previously, for the new batch. The contract will then: -- Check that the batch number is the immediate successor of the last batch processed. -- Check that the state diffs are valid, either through hashing or the point evaluation precompile. +- Check that the batch number is the immediate successor of the last committed batch. +- Check that the batch has not been committed already. +- Check that the `lastBlockHash` is not zero. +- If privileged transactions were processed, it checks the submitted hash against the one in the `CommonBridge` contract. +- If withdrawals were processed, it publishes them to the `CommonBridge` contract. +- It checks that a blob was published if the L2 is running as a rollup, or that no blob was published if it's running as a validium. - Calculate the new batch commitment and store it. ### Verify transaction @@ -169,11 +170,16 @@ The contract will then: On a `verification` transaction, the L1 contract receives the following: - The batch number. -- The batch proof. +- The RISC-V Zero-Knowledge proof of the batch execution (if enabled). +- The SP1 Zero-Knowledge proof of the batch execution (if enabled). +- The TDX Zero-Knowledge proof of the batch execution (if enabled). The contract will then: -- Compute the proof public input from the new and previous batch commitments (both are already stored in the contract). +- Check that the batch number is the immediate successor of the last verified batch. +- Check that the batch has been committed. +- It removes the pending transaction hashes from the `CommonBridge` contract. +- It verifies the public data of the proof, checking that the data committed in the `commitBatch` call matches the data in the public inputs of the proof. - Pass the proof and public inputs to the verifier and assert the proof passes. - If the proof passes, finalize the L2 state, setting the latest batch as the given one and allowing any withdrawals for that batch to occur. diff --git a/docs/l2/architecture/prover.md b/docs/l2/architecture/prover.md index 2c95492d358..3e8cee05845 100644 --- a/docs/l2/architecture/prover.md +++ b/docs/l2/architecture/prover.md @@ -1,14 +1,10 @@ -# Ethrex L2 prover - -> [!NOTE] -> The shipping/deploying process and the `Prover` itself are under development. +# ethrex-prover for L2 ## Intro -The prover consists of two main components: handling incoming proving data from the `L2 proposer`, specifically from the `ProofCoordinator` component, and the `zkVM`. The `Prover` is responsible for this first part, while the `zkVM` serves as a RISC-V emulator executing code specified in `crates/l2/prover/src/guest_program/guest/src`. -Before the `zkVM` code (or guest), there is a directory called `interface`, which indicates that we access the `zkVM` through the "interface" crate. +The prover consists of two main components: handling incoming proving data from the L2 sequencer, specifically from the `ProofCoordinator` component, and the actual zkVM running and generating proofs of execution. -In summary, the `Prover` manages the inputs from the `ProofCoordinator` and then "calls" the `zkVM` to perform the proving process and generate the `groth16` ZK proof. +In summary, the prover manages the inputs from the `ProofCoordinator` and then calls the zkVM to perform the proving process and generate the zero-knowledge proof (groth16 for on-chain verification, or a compressed STARK for verification via Aligned Layer). ## Workflow @@ -20,317 +16,13 @@ sequenceDiagram participant Prover participant ProofCoordinator Prover->>+ProofCoordinator: ProofData::Request - ProofCoordinator-->>-Prover: ProofData::Response(batch_number, ProverInputs) - Prover->>+zkVM: Prove(ProverInputs) - zkVM-->>-Prover: Creates zkProof - Prover->>+ProofCoordinator: ProofData::Submit(batch_number, zkProof) - ProofCoordinator-->>-Prover: ProofData::SubmitAck(batch_number) -``` - -## How - -**Dependencies:** - -- [RISC0](https://dev.risczero.com/api/zkvm/install) - 1. `curl -L https://risczero.com/install | bash` - 2. `rzup install cargo-risczero 3.0.3` - 3. `rzup install risc0-groth16` - 4. `rzup install rust` -- [SP1](https://docs.succinct.xyz/docs/sp1/introduction) - 1. `curl -L https://sp1up.succinct.xyz | bash` - 2. `sp1up --version 5.0.8` -- [SOLC](https://docs.soliditylang.org/en/latest/installing-solidity.html) - -After installing the toolchains, a quick test can be performed to check if we have everything installed correctly. - -### L1 block proving - -ethrex-prover is able to generate execution proofs of Ethereum Mainnet/Testnet blocks. An example binary was created for this purpose in `crates/l2/prover/bench`. Refer to its README for usage. - -### Dev Mode - -To run the blockchain (`proposer`) and prover in conjunction, start the `Prover`, use the following command: - -```sh -make init-prover- # optional: GPU=true -``` - -#### Run the whole system with the prover - In one Machine - -> [!NOTE] -> Used for development purposes. - -1. `cd crates/l2` -2. `make rm-db-l2 && make down` - - It will remove any old database, if present, stored in your computer. The absolute path of SQL is defined by [datadir](https://docs.rs/dirs/latest/dirs/fn.datadir.html). -3. `make init` - - Make sure you have the `solc` compiler installed in your system. - - Init the L1 in a docker container on port `8545`. - - Deploy the needed contracts for the L2 on the L1. - - Start the L2 locally on port `1729`. -4. In a new terminal → `make init-prover- # GPU=true`. - -After this initialization we should have the prover running in `dev_mode` → No real proofs. - -### GPU mode - -**Steps for Ubuntu 22.04 with Nvidia A4000:** - -1. Install `docker` → using the [Ubuntu apt repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) - - Add the `user` you are using to the `docker` group → command: `sudo usermod -aG docker $USER`. (needs reboot, doing it after CUDA installation) - - `id -nG` after reboot to check if the user is in the group. -2. Install [Rust](https://www.rust-lang.org/tools/install) -3. Install [RISC0](https://dev.risczero.com/api/zkvm/install) -4. Install [CUDA for Ubuntu](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_local) - - Install `CUDA Toolkit Installer` first. Then the `nvidia-open` drivers. -5. Reboot -6. Run the following commands: - -```sh -sudo apt-get install libssl-dev pkg-config libclang-dev clang -echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc -echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc -``` - -#### Run the whole system with a GPU Prover - -Two servers are required: one for the `Prover` and another for the `sequencer`. If you run both components on the same machine, the `Prover` may consume all available resources, leading to potential stuttering or performance issues for the `sequencer`/`node`. - -- The number 1 simbolizes a machine with GPU for the `Prover`. -- The number 2 simbolizes a machine for the `sequencer`/L2 node itself. - -1. `Prover`/`zkvm` → prover with gpu, make sure to have all the required dependencies described at the beginning of [Gpu Mode](#gpu-mode) section. - 1. `cd ethrex/crates/l2` - 2. You can set the following environment variables to configure the prover: - - PROVER_CLIENT_PROVER_SERVER_ENDPOINT: The address of the server where the client will request the proofs from - - PROVER_CLIENT_PROVING_TIME_MS: The amount of time to wait before requesting new data to prove - -- `Finally`, to start the `Prover`/`zkvm`, run: - - `make init-prover- # optional: GPU=true` - -2. `ProofCoordinator`/`sequencer` → this server just needs rust installed. - - 1. `cd ethrex/crates/l2` - 2. Create a `.env` file with the following content: - - ```env - // Should be the same as ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY - ETHREX_DEPLOYER_L1_PRIVATE_KEY= - // Should be the same as ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_DEPLOYER_L1_PRIVATE_KEY - ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY= - // Should be the same as ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY and ETHREX_DEPLOYER_L1_PRIVATE_KEY - ETHREX_COMMITTER_L1_PRIVATE_KEY= - // Should be different from ETHREX_COMMITTER_L1_PRIVATE_KEY and ETHREX_WATCHER_L2_PROPOSER_PRIVATE_KEY - ETHREX_PROOF_COORDINATOR_L1_PRIVATE_KEY= - // Used to handle TCP communication with other servers from any network interface. - ETHREX_PROOF_COORDINATOR_LISTEN_ADDRESS=0.0.0.0 - // Set to true to randomize the salt. - ETHREX_DEPLOYER_RANDOMIZE_CONTRACT_DEPLOYMENT=true - // Set to true if you want SP1 proofs to be required - ETHREX_L2_SP1=true - // Check the if the verification contract is present on your preferred network. Don't define this if you want it to be deployed automatically. - ETHREX_DEPLOYER_SP1_VERIFIER_ADDRESS=
- // Set to true if you want proofs to be required - ETHREX_L2_RISC0=true - // Check the if the contract is present on your preferred network. You shall deploy it manually if not. - ETHREX_DEPLOYER_RISC0_VERIFIER_ADDRESS=
- // Set to any L1 endpoint. - ETHREX_ETH_RPC_URL= - ``` - - 3. `source .env` - -> [!NOTE] -> Make sure to have funds, if you want to perform a quick test `0.2[ether]` on each account should be enough. - -- `Finally`, to start the `proposer`/`l2 node`, run: - - - `make rm-db-l2 && make down` - - `make deploy-l1 && make init-l2` (if running a risc0 prover, see the next step before invoking the L1 contract deployer) - -- If running with a local L1 (for development), you will need to manually deploy the risc0 contracts by following the instructions [here](https://github.com/risc0/risc0-ethereum/tree/main/contracts/script). -- For a local L1 running with ethrex, we do the following: - - 1. clone the risc0-ethereum repo - 1. edit the `risc0-ethereum/contracts/deployment.toml` file by adding - ```toml - [chains.ethrex] - name = "Ethrex local devnet" - id = 9 - ``` - 1. export env. variables (we are using an ethrex's rich L1 account) - - ```bash - export VERIFIER_ESTOP_OWNER="0x4417092b70a3e5f10dc504d0947dd256b965fc62" - export DEPLOYER_PRIVATE_KEY="0x941e103320615d394a55708be13e45994c7d93b932b064dbcb2b511fe3254e2e" - export DEPLOYER_ADDRESS="0x4417092b70a3e5f10dc504d0947dd256b965fc62" - export CHAIN_KEY="ethrex" - export RPC_URL="http://localhost:8545" - - export ETHERSCAN_URL="dummy" - export ETHERSCAN_API_KEY="dummy" - ``` - - the last two variables need to be defined with some value even if not used, else the deployment script fails. - - 1. cd into `risc0-ethereum/` - 1. run the deployment script - ```bash - bash contracts/script/manage DeployEstopGroth16Verifier --broadcast - ``` - 1. if the deployment was successful you should see the contract address in the output of the command, you will need to pass this as an argument to the L2 contract deployer, or via the `ETHREX_DEPLOYER_RISC0_VERIFIER_ADDRESS=
` env. variable. - if you get an error like `risc0-ethereum/contracts/../lib/forge-std/src/Script.sol": No such file or directory (os error 2)`, try to update the git submodules (foundry dependencies) with `git submodule update --init --recursive`. - -## Configuration - -Configuration is done through environment variables or CLI flags. -You can see a list of available flags by passing `--help` to the CLI. - -The following environment variables are available to configure the `Prover`: - -- `CONFIGS_PATH`: The path where the `PROVER_CLIENT_CONFIG_FILE` is located at. -- `PROVER_CLIENT_CONFIG_FILE`: The `.toml` that contains the config for the `Prover`. -- `PROVER_ENV_FILE`: The name of the `.env` that has the parsed `.toml` configuration. -- `PROVER_CLIENT_PROVER_SERVER_ENDPOINT`: Prover Server's Endpoint used to connect the Client to the Server. - -The following environment variables are used by the ProverServer: - -- `PROVER_SERVER_LISTEN_IP`: IP used to start the Server. -- `PROVER_SERVER_LISTEN_PORT`: Port used to start the Server. -- `PROVER_SERVER_VERIFIER_ADDRESS`: The address of the account that sends the zkProofs on-chain and interacts with the `OnChainProposer` `verify()` function. -- `PROVER_SERVER_VERIFIER_PRIVATE_KEY`: The private key of the account that sends the zkProofs on-chain and interacts with the `OnChainProposer` `verify()` function. - -> [!NOTE] -> The `PROVER_SERVER_VERIFIER` account must differ from the `COMMITTER_L1` account. - -## How it works - -The prover's sole purpose is to generate a block (or batch of blocks) execution proof. For this, ethrex-prover implements a blocks execution program and generates a proof of it using different RISC-V zkVMs (SP1, Risc0). - -The prover runs a process that polls another for new jobs. The job must provide the program inputs. A proof of the program's execution with the provided inputs is generated by the prover and sent back. - -### Program inputs - -The inputs for the blocks execution program (also called program inputs or prover inputs) are: - -- the blocks to prove (header and body) -- the first block's parent header -- an execution witness -- the blocks' deposits hash -- the blocks' withdrawals Merkle root -- the blocks' state diff hash - -The last three inputs are L2 specific. - -These inputs are required for proof generation, but not all of them are committed as public inputs, which are needed for proof verification. The proof's public inputs (also called program outputs) will be: - -- the initial state hash (from the first block's parent header) -- the final state hash (from the last block's header) -- the blocks' deposits hash -- the blocks' withdrawals Merkle root -- the blocks' state diff hash - -#### Execution witness - -The purpose of the execution witness is to allow executing the blocks without having access to the whole Ethereum state, as it wouldn't fit in a zkVM program. It contains only the state values needed during the execution. - -An execution witness contains all the initial state values (state nodes, codes, storage keys, block headers) that will be read or written to during the blocks' execution. - -An execution witness is created from a prior execution of the blocks. Before proving, we need to: - -1. execute the blocks (also called "pre-execution") to identify which state values will be accessed. -2. log every initial state value accessed or updated during this execution. -3. retrieve an MPT proof for each value, linking it (or its non-existence) to the initial state root hash. - -Steps 1 and 2 are data collection steps only - no validation is performed at this stage. The actual validation happens later inside the zkVM guest program. Step 3 involves more complex logic due to potential issues when restructuring the pruned state trie after value removals. In sections [initial state validation](#step-1-initial-state-validation) and [final state validation](#step-3-final-state-validation) we explain what are pruned tries and in which case they get restructured. - -If a value is removed during block execution (meaning it existed initially but not finally), two pathological cases can occur where the witness lacks sufficient information to update the trie structure correctly: - -**Case 1** - -![Image showing restructuration for case 1](../img/execw_case1.png) - -Here, only **leaf 1** is part of the execution witness, so we lack the proof (and thus the node data) for **leaf 2**. After removing **leaf 1**, **branch 1** becomes redundant. During trie restructuring, it's replaced by **leaf 3**, whose path is the path of **leaf 2** concatenated with a prefix nibble (`k`) representing the choice taken at the original **branch 1**, and keeping **leaf 2**'s value. - -```text -branch1 = {c_1, c_2, ..., c_k, ..., c_16} # Only c_k = hash(leaf2) is non-empty -leaf2 = {value, path} -leaf3 = {value, concat(k, path)} # New leaf replacing branch1 and leaf2 + ProofCoordinator-->>-Prover: ProofData::Response(inputs) + Prover->>+zkVM: Prove(inputs) + zkVM-->>-Prover: generates zk proof + Prover->>+ProofCoordinator: ProofData::Submit(batch number, proof) + ProofCoordinator-->>-Prover: ProofData::SubmitAck(batch number) ``` -Without **leaf 2**'s data, we cannot construct **leaf 3**. The solution is to fetch the _final_ state proof for the key of **leaf 2**. This yields an exclusion proof containing **leaf 3**. By removing the prefix nibble `k`, we can reconstruct the original path and value of **leaf 2**. This process might need to be repeated if similar restructuring occurred at higher levels of the trie. - -**Case 2** - -![Image showing restructuration for case 2](../img/execw_case2.png) - -In this case, restructuring requires information about **branch/ext 2** (which could be a branch or extension node), but this node might not be in the witness. Checking the final **extension** node might seem sufficient to deduce **branch/ext 2** in simple scenarios. However, this fails if similar restructuring occurred at higher trie levels involving more removals, as the final **extension** node might combine paths from multiple original branches, making it ambiguous to reconstruct the specific missing **branch/ext 2** node. - -The solution is to fetch the missing node directly using a `debug` JSON-RPC method, like `debug_dbGet` (or `debug_accountRange` and `debug_storageRangeAt` if using a Geth node). - -> [!NOTE] -> These problems arise when creating the execution witness solely from state proofs fetched via standard JSON-RPC. In the L2 context, where we control the sequencer, we could develop a protocol to easily retrieve all necessary data more directly. However, the problems remain relevant when proving L1 blocks (e.g., for testing/benchmarking). - -### Blocks execution program - -The program leverages ethrex-common primitives and ethrex-vm methods. ethrex-prover implements a program that uses the existing execution logic and generates a proof of its execution using a zkVM. Some L2-specific logic and input validation are added on top of the basic blocks execution. - -The following sections outline the steps taken by the execution program. - -#### Prelude 1: state trie basics - -We recommend learning about Merkle Patricia Tries (MPTs) to better understand this section. - -Each executed block transitions the Ethereum state from an initial state to a final state. State values are stored in MPTs: - -1. Each account has a Storage Trie containing its storage values. -2. The World State Trie contains all account information, including each account's storage root hash (linking storage tries to the world trie). - -Hashing the root node of the world state trie generates a unique identifier for a particular Ethereum state, known as the "state hash". - -There are two kinds of MPT proofs: - -1. Inclusion proofs: Prove that `key: value` is a valid entry in the MPT with root hash `h`. -2. Exclusion proofs: Prove that `key` does not exist in the MPT with root hash `h`. - These proofs allow verifying that a value is included (or its key doesn't exist) in a specific state. - -#### Prelude 2: deposits, withdrawals and state diffs - -These three components are specific additions for ethrex's L2 protocol, layered on top of standard Ethereum execution logic. They each require specific validation steps within the program. - -For more details, refer to [Overview](./overview.md), [Withdrawals](../fundamentals/withdrawals.md), and [State diffs](../fundamentals/state_diffs.md). - -#### Step 1: initial state validation - -The program validates the initial state by converting the `ExecutionWitness` into a `GuestProgramState` and verifying that its trie structure correctly represents the expected state. This involves checking that the calculated state trie root hash matches the initial state hash (obtained from the first block's parent block header). - -The validation happens in several steps: -1. The `ExecutionWitness` (collected during pre-execution) is converted to `GuestProgramState` -2. A `GuestProgramStateWrapper` is created to provide database functionality -3. The state trie root is calculated and compared against the parent block header's state root - -This validation ensures that all state values needed for execution are properly linked to the initial state via their MPT proofs. Having the initial state proofs (paths from the root to each relevant leaf) is equivalent to having a relevant subset of the world state trie and storage tries - a set of "pruned tries". This allows operating directly on these pruned tries (adding, removing, modifying values) during execution. - -#### Step 2: blocks execution - -After validating the initial state, the program executes the blocks. This leverages the existing ethrex execution logic used by the L2 client itself. - -#### Step 3: final state validation - -During execution, state values are updated (modified, created, or removed). After execution, the program calculates the final state by applying these state updates to the initial pruned tries. - -Applying the updates results in a new world state root node for the pruned tries. Hashing this node yields the calculated final state hash. The program then verifies that this calculated hash matches the expected final state hash (from the last block header), thus validating the final state. - -As mentioned earlier, removing values can sometimes require information not present in the initial witness to correctly restructure the pruned tries. The [Execution witness](#execution-witness) section details this problem and its solution. - -#### Step 4: deposit hash calculation - -After execution and final state validation, the program calculates a hash encompassing all deposits made within the blocks (extracting deposit info from `PrivilegedL2Transaction` type transactions). This hash is committed as a public input, required for verification on the L1 bridge contract. - -#### Step 5: withdrawals Merkle root calculation - -Similarly, the program constructs a binary Merkle tree of all withdrawals initiated in the blocks and calculates its root hash. This hash is also committed as a public input. Later, L1 accounts can claim their withdrawals by providing a Merkle proof of inclusion that validates against this root hash on the L1 bridge contract. - -#### Step 6: state diff calculation and commitment - -Finally, the program calculates the state diffs (changes between initial and final state) intended for publication to L1 as blob data. It creates a commitment to this data (a Merkle root hash), which is committed as a public input. Using proof of equivalence logic within the L1 bridge contract, this Merkle commitment can be verified against the KZG commitment of the corresponding blob data. +For running the prover, see [Deploy an L2](../../l2/deployment/overview.md). +For developer-focused setup and run instructions, see [Running the Prover](../../developers/l2/prover.md). +For comprehensive details on the internals of the prover, see [ethrex-prover](../../prover/prover.md). diff --git a/docs/l2/deploy.md b/docs/l2/deploy.md deleted file mode 100644 index 1ea8b203257..00000000000 --- a/docs/l2/deploy.md +++ /dev/null @@ -1,108 +0,0 @@ -# Deploy an L2 - -## Prerequisites - -This guide assumes that you have ethrex installed. If you haven't done so, follow one of the installation methods in the [installation guide](../getting-started/installation/). - -## Deploy the contracts - -The first step is to deploy the rollup's core contracts to your chosen L1 network. - -### 1. Download the contracts - -You can get the contracts in two ways: - -- **From GitHub Releases:** - - - Download the latest release from [GitHub Releases](https://github.com/lambdaclass/ethrex/releases/download/v0.0.4-alpha/ethrex-contracts.tar.gz). - -- **From source code (latest version):** - - - Clone the repository: - ```sh - git clone https://github.com/lambdaclass/ethrex.git - cd ethrex/crates/l2/contracts/src/l1 - ``` - -### 2. Deploy the contracts - -You can deploy the contracts manually or using the built-in tool: - -```sh -ethrex l2 deploy \ - --eth-rpc-url \ - --private-key \ - --genesis-l2-path \ - --risc0.verifier-address \ - --sp1.verifier-address \ - --tdx.verifier-address \ - --aligned.aggregator-address \ - --on-chain-proposer-owner \ - --bridge-owner \ - --randomize-contract-deployment -``` - -You can find a genesis example in the [repo](https://github.com/lambdaclass/ethrex/blob/main/fixtures/genesis/l2.json). - -`--bridge-owner` must point to the address that will ultimately control the CommonBridge upgrades. -If you also need the deployer to accept the transfer on that owner’s behalf, pass `--bridge-owner-pk ` in the same command; otherwise the owner can accept later. - -Verifier addresses can be set to `0x00000000000000000000000000000000000000AA` in case you don't want to use some prover. The same applies to Aligned. - -> [!TIP] -> You can start a local [development L1](../l1/running/configuration.md#dev-mode-localnet) network with `ethrex l1 --dev` and use its RPC URL for testing. - -## Run the sequencer - -Next step is to start the sequencer. This command will start all necessary components for the L2 network except the prover. - -```sh -ethrex l2 \ - --network \ - --l1.on-chain-proposer-address \ - --l1.bridge-address \ - --rpc_url \ - --committer.l1-private-key \ - --proof-coordinator.l1-private-key \ - --block-producer.coinbase-address \ -``` - -OnChainProposer and CommonBridge addresses can be found in the `.env` file, generated during the deployment process. Committer and Proof coordinator accounts must have L1 funds, as they will need to pay for gas fees on the L1 network. - -For further configuration take a look at the [CLI document](../CLI.md#ethrex-l2) - -## Run the prover - -Lastly, you need to start the prover. This command will start the prover component for the L2 network. - -```sh -ethrex l2 prover --proof-coordinators tcp://localhost:3900 --backend exec -``` - -In this example, the `exec` backend is used, which means the prover will only execute the transactions but not generate proofs. This is fine for development as it's faster. You may look for other backends like SP1 and RISC0 in production. - -For further configuration take a look at the [CLI document](../CLI.md#ethrex-l2-prover) - -## Checking that everything is running - -After starting the sequencer and prover, you can verify that your L2 node is running correctly: - -- **Check the sequencer RPC:** - - You can request the latest block number: - - ```sh - curl http://localhost:1729 \ - -H 'content-type: application/json' \ - -d '{"jsonrpc":"2.0","method":"eth_blockNumber","id":"1","params":[]}' - ``` - - The answer should be like this, and advance every 5 seconds: - - ``` - {"id":"1","jsonrpc":"2.0","result":"0x1"} - ``` - -- **Check logs:** - - Review the terminal output or log files for any errors or warnings. - - After some time (1 minute by default) there should be a log from the L1 Committer informing a new batch is being sent to L1. diff --git a/docs/l2/deployment/README.md b/docs/l2/deployment/README.md new file mode 100644 index 00000000000..b27812f784f --- /dev/null +++ b/docs/l2/deployment/README.md @@ -0,0 +1,10 @@ +# Deploy an L2 + +This section provides step-by-step guides for deploying different types of ethrex L2 chains, including vanilla, validium, and based configurations. Each guide outlines the necessary commands and parameters to successfully deploy and start an L2 node. + +Use this section to choose the deployment method that best fits your needs and follow the instructions accordingly. + +- [Overview](./overview.md) +- [Deploying a vanilla ethrex L2](./vanilla.md) +- [Deploying a validium ethrex L2](./validium.md) +- [Deploying a based ethrex L2](./based.md) diff --git a/docs/l2/deployment/based.md b/docs/l2/deployment/based.md new file mode 100644 index 00000000000..e0bedf139e2 --- /dev/null +++ b/docs/l2/deployment/based.md @@ -0,0 +1,3 @@ +# Deploying a based ethrex L2 + +TBD diff --git a/docs/l2/deployment/overview.md b/docs/l2/deployment/overview.md new file mode 100644 index 00000000000..628141f8057 --- /dev/null +++ b/docs/l2/deployment/overview.md @@ -0,0 +1,63 @@ +# Deploying an ethrex L2 + +As outlined in the introduction, ethrex L2 offers a wide range of features to its users. The most common is a classic centralized L2 managed by an operator, which can be an individual or a DAO. ethrex L2 can also function as a Validium, which is similarly centralized and operator-managed, with the key difference that network data is not posted to L1 during settlement. + +In addition to these classic functionalities, ethrex L2 provides a novel and continually evolving feature in the industry: ethrex L2 as a based rollup. Unlike the previous options, this is a permissionless and decentralized L2 sequencer—anyone can run a node and participate in the network. + +In this section, we will cover how to deploy any of these options. + +> [!NOTE] +> This section focuses solely on the step-by-step process for deploying ethrex L2 in any of its forms. For a deeper understanding of how each mode works under the hood, refer to the Fundamentals section. To learn more about the architecture of each mode, see the Architecture section. + +Before proceeding, note that this guide assumes you have ethrex installed. If you haven't installed it yet, follow one of the methods in the [Installation Guide](../../getting-started/installation/README.md). If you're looking to build from source, don't skip this section—we'll cover that method here, as it is independent of the deployment approach you choose later. + +## Building from source (skip if ethrex is already installed) + +### Prerequisites + +Ensure you have the following installed on your system: + +- Rust and Cargo (install via [rustup](https://rustup.rs/)) +- Solidity compiler (refer to [Solidity documentation](https://docs.soliditylang.org/en/latest/installing-solidity.html)) +- SP1 Toolchain (if you plan to use SP1 proving, refer to [SP1 documentation](https://docs.succinct.xyz/docs/sp1/getting-started/install)) +- RISC0 Toolchain (if you plan to use RISC0 proving, refer to [RISC0 documentation](https://dev.risczero.com/api/zkvm/install)) +- CUDA Toolkit 12.9 (if you plan to use GPU acceleration for SP1 or RISC0 proving) + +1. Clone the official ethrex repository: + + ```shell + git clone https://github.com/lambdaclass/ethrex + cd ethrex + ``` + +2. Build the binary: + + ```shell + # For dummy proving + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql + + # For SP1 CPU proving (very slow, not recommended) + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,sp1 + + # For RISC0 CPU proving (very slow, not recommended) + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,risc0 + + # For SP1 and RISC0 CPU proving (very slow, not recommended) + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,sp1,risc0 + + # For SP1 GPU proving + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,sp1,gpu + + # For RISC0 GPU proving + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,risc0,gpu + + # For SP1 and RISC0 GPU proving + COMPILE_CONTRACTS=true cargo build --release --bin ethrex --features l2,l2-sql,sp1,risc0,gpu + ``` + +> [!WARNING] +> If you want your verifying keys generation to be reproducible, prepend `PROVER_REPRODUCIBLE_BUILD=true` to the above command: +> +> ```shell +> PROVER_REPRODUCIBLE_BUILD=true COMPILE_CONTRACTS=true cargo b -r --bin ethrex -F l2,l2-sql,sp1,risc0,gpu +> ``` diff --git a/docs/l2/deployment/prover/README.md b/docs/l2/deployment/prover/README.md new file mode 100644 index 00000000000..1b166c19975 --- /dev/null +++ b/docs/l2/deployment/prover/README.md @@ -0,0 +1,11 @@ +# Run a prover + +This section provides step-by-step guides for running an ethrex L2 prover, which is responsible for generating ZK proofs for L2 blocks. These proofs are then submitted to L1 for verification, finalizing the state of your L2. + +Use this section to choose which prover setup best fits your already deployed ethrex L2 and follow the instructions accordingly. + +- [Overview](./overview.md) +- [Running an ethrex L2 SP1 prover](./sp1.md) +- [Running an ethrex L2 RISC0 prover](./risc0.md) +- [Running an ethrex L2 TDX prover](./tee.md) +- [Running multiple provers](./multi-prover.md) diff --git a/docs/l2/deployment/prover/multi-prover.md b/docs/l2/deployment/prover/multi-prover.md new file mode 100644 index 00000000000..424396d7b5d --- /dev/null +++ b/docs/l2/deployment/prover/multi-prover.md @@ -0,0 +1,17 @@ +# Run multiple provers + +In this section, we'll guide you through the steps to run multiple ethrex L2 provers for generating ZK proofs using different backends. These proofs are essential for validating batch execution and state settlement on your ethrex L2. + +## Prerequisites + +- This guide assumes that you have already deployed an ethrex L2 with TDX enabled. If you haven't done so yet, please refer to one of the [Deploying an ethrex L2](../overview.md) guides. + +## Start multiple ethrex L2 provers + +Once you have your ethrex L2 deployed with multiple proving backends enabled (SP1, RISC0, TDX), refer to the following guides to start each prover: + +- [Run an ethrex L2 SP1 prover](./sp1.md) +- [Run an ethrex L2 RISC0 prover](./risc0.md) +- [Run an ethrex TDX prover](./tee.md) + +Each prover should be started in different machines to ensure they operate independently and efficiently. Make sure to configure each prover with the appropriate backend flag and proof coordinator URLs as specified in their respective guides. diff --git a/docs/l2/deployment/prover/overview.md b/docs/l2/deployment/prover/overview.md new file mode 100644 index 00000000000..34ae2448e83 --- /dev/null +++ b/docs/l2/deployment/prover/overview.md @@ -0,0 +1,64 @@ +# Run an ethrex prover + +Deploying the ethrex L2 contracts on L1 and starting the node isn't everything when it comes to setting up your full ethrex L2 stack. + +If you've been following the deployment guide, you should already have an ethrex L2 node running and connected to L1. If that's not the case, I recommend reviewing that guide before proceeding. + +The next step is to run the prover—the component responsible for generating ZK proofs for the L2 blocks. These proofs will then be sent to L1 for verification, finalizing the state of your L2. + +In this section, we'll cover how to run one or more ethrex L2 provers. + +> [!NOTE] +> This section focuses solely on the step-by-step process for running an ethrex L2 prover in any of its forms. For a deeper understanding of this works under the hood, refer to the Fundamentals section. To learn more about the architecture of each mode, see the Architecture section. + +Before proceeding, note that this guide assumes you have ethrex installed. If you haven't installed it yet, follow one of the methods in the [Installation Guide](../../../getting-started/installation/README.md). If you're looking to build from source, don't skip this section—we'll cover that method here, as it is independent of the deployment approach you choose later. + +## Building from source (skip if ethrex is already installed) + +### Prerequisites + +Ensure you have the following installed on your system: + +- Rust and Cargo (install via [rustup](https://rustup.rs/)) +- Solidity compiler (refer to [Solidity documentation](https://docs.soliditylang.org/en/latest/installing-solidity.html)) +- SP1 Toolchain (if you plan to use SP1 proving, refer to [SP1 documentation](https://docs.succinct.xyz/docs/sp1/getting-started/install)) +- RISC0 Toolchain (if you plan to use RISC0 proving, refer to [RISC0 documentation](https://dev.risczero.com/api/zkvm/install)) +- CUDA Toolkit 12.9 (if you plan to use GPU acceleration for SP1 or RISC0 proving) + +1. Clone the official ethrex repository: + + ```shell + git clone https://github.com/lambdaclass/ethrex + cd ethrex + ``` + +2. Build the binary: + + ```shell + # For SP1 CPU proving (very slow, not recommended) + cargo build --release --bin ethrex --features l2,l2-sql,sp1 + + # For RISC0 CPU proving (very slow, not recommended) + cargo build --release --bin ethrex --features l2,l2-sql,risc0 + + # For SP1 and RISC0 CPU proving (very slow, not recommended) + cargo build --release --bin ethrex --features l2,l2-sql,sp1,risc0 + + # For SP1 GPU proving + cargo build --release --bin ethrex --features l2,l2-sql,sp1,gpu + + # For RISC0 GPU proving + cargo build --release --bin ethrex --features l2,l2-sql,risc0,gpu + + # For SP1 and RISC0 GPU proving + cargo build --release --bin ethrex --features l2,l2-sql,sp1,risc0,gpu + ``` + +> [!WARNING] +> If you want your verifying keys generation to be reproducible, prepend `PROVER_REPRODUCIBLE_BUILD=true` to the above command. +> +> Example: +> +> ```shell +> PROVER_REPRODUCIBLE_BUILD=true COMPILE_CONTRACTS=true cargo b -r --bin ethrex -F l2,l2-sql,sp1,risc0,gpu +> ``` diff --git a/docs/l2/deployment/prover/risc0.md b/docs/l2/deployment/prover/risc0.md new file mode 100644 index 00000000000..2ba68c7f99f --- /dev/null +++ b/docs/l2/deployment/prover/risc0.md @@ -0,0 +1,25 @@ +# Run an ethrex L2 RISC0 prover + +In this section, we'll guide you through the steps to run an ethrex L2 prover that utilizes RISC0 for generating ZK proofs. These proofs are essential for validating batch execution and state settlement on your ethrex L2. + +## Prerequisites + +- This guide assumes that you have ethrex installed with the RISC0 feature and available in your PATH. If you haven't installed it yet, follow one of the methods in the Installation Guide. If you want to build the binary from source, refer to the [Building from source](./overview.md#building-from-source-skip-if-ethrex-is-already-installed) section and select the appropriate build option. +- This guide also assumes that you have already deployed an ethrex L2 with RISC0 enabled. If you haven't done so yet, please refer to one of the [Deploying an ethrex L2](../overview.md) guides. + +## Start an ethrex L2 RISC0 prover + +Once you have your ethrex L2 deployed with RISC0 enabled, you can start the RISC0 prover using the following command: + +```shell +ethrex l2 prover \ +--backend risc0 \ +--proof-coordinators http://localhost:3900 +``` + +> [!IMPORTANT] +> Cualquiera haya sido el metodo de instalacion de ethrex, asegurate de que el binario que estas utilizando tiene soporte para RISC0, y tambien para GPU si es que tu intencion es correr un prover RISC0 GPU. + +> [!NOTE] +> The flag `--proof-coordinators` is used to specify one or more proof coordinator URLs. This is so because the prover is capable of proving ethrex L2 batches from multiple sequencers. We are particularly setting it to `localhost:3900` because the command above command uses the port `3900` for the proof coordinator by default (to learn more about the proof coordinator, read the ethrex L2 sequencer and ethrex L2 prover sections). +> We choose RISC0 as the backend to indicate the prover to generate RISC0 proofs. diff --git a/docs/l2/deployment/prover/sp1.md b/docs/l2/deployment/prover/sp1.md new file mode 100644 index 00000000000..636f8b022a7 --- /dev/null +++ b/docs/l2/deployment/prover/sp1.md @@ -0,0 +1,25 @@ +# Run an ethrex L2 SP1 prover + +In this section, we'll guide you through the steps to run an ethrex L2 prover that utilizes SP1 for generating ZK proofs. These proofs are essential for validating batch execution and state settlement on your ethrex L2. + +## Prerequisites + +- This guide assumes that you have ethrex installed with the SP1 feature and available in your PATH. If you haven't installed it yet, follow one of the methods in the Installation Guide. If you want to build the binary from source, refer to the [Building from source](./overview.md#building-from-source-skip-if-ethrex-is-already-installed) section and select the appropriate build option. +- This guide also assumes that you have already deployed an ethrex L2 with SP1 enabled. If you haven't done so yet, please refer to one of the [Deploying an ethrex L2](../overview.md) guides. + +## Start an ethrex L2 SP1 prover + +Once you have your ethrex L2 deployed with SP1 enabled, you can start the SP1 prover using the following command: + +```shell +ethrex l2 prover \ +--backend sp1 \ +--proof-coordinators http://localhost:3900 +``` + +> [!IMPORTANT] +> Cualquiera haya sido el metodo de instalacion de ethrex, asegurate de que el binario que estas utilizando tiene soporte para SP1, y tambien para GPU si es que tu intencion es correr un prover SP1 GPU. + +> [!NOTE] +> The flag `--proof-coordinators` is used to specify one or more proof coordinator URLs. This is so because the prover is capable of proving ethrex L2 batches from multiple sequencers. We are particularly setting it to `localhost:3900` because the command above command uses the port `3900` for the proof coordinator by default (to learn more about the proof coordinator, read the ethrex L2 sequencer and ethrex L2 prover sections). +> We choose SP1 as the backend to indicate the prover to generate SP1 proofs. diff --git a/docs/l2/deployment/prover/tee.md b/docs/l2/deployment/prover/tee.md new file mode 100644 index 00000000000..6067cd61fe6 --- /dev/null +++ b/docs/l2/deployment/prover/tee.md @@ -0,0 +1,23 @@ +# Run an ethrex TDX prover + +In this section, we'll guide you through the steps to run an ethrex L2 TDX prover for generating TEE proofs. These proofs are essential for validating batch execution and state settlement on your ethrex L2. + +## Prerequisites + +- This guide assumes that you have already deployed an ethrex L2 with TDX enabled. If you haven't done so yet, please refer to one of the [Deploying an ethrex L2](../overview.md) guides. +- A machine with TDX support [with the required setup](https://github.com/canonical/tdx). + +## Start an ethrex L2 TDX prover + +There's no official release of our ethrex L2 TDX prover yet, so you need to build ethrex from source. To do this, clone the ethrex repository and run: + +```shell +git clone https://github.com/lambdaclass/ethrex.git + +cd ethrex/crates/l2/tee/quote-gen + +make run +``` + +> [!NOTE] +> Refer to the [TDX guide](../../architecture/tdx.md) for more information on setting up and running the quote generator. diff --git a/docs/l2/deployment/synchronous_composability_poc.md b/docs/l2/deployment/synchronous_composability_poc.md new file mode 100644 index 00000000000..f4a95ab59ef --- /dev/null +++ b/docs/l2/deployment/synchronous_composability_poc.md @@ -0,0 +1,121 @@ +# Synchronous Composability (PoC) + +## Status + +**Development branch:** `sync_comp_poc` + +| Sync | Column 1 | Status | +| -------- | ------------------------- | ------ | +| L1 -> L2 | Deposits | ✅ | +| L1 -> L2 | L2 contract calls from L1 | ✅ | +| L2 -> L1 | Withdrawals | ✅ | +| L2 -> L1 | L1 contract calls from L2 | ❌ | +| L2 -> L2 | | 🔜 | + +## Commands + +### Prerequisites + +- A fresh-cloned ethrex repository. +- `rex` installed and available in your PATH. If you haven't installed it yet, follow one of the methods in the [rex repository](https://github.com/lambdaclass/rex?tab=readme-ov-file#rex-cli). + +### Run a supernode + +The following command will: + +1. Remove both L1 and L2 dev databases (to start from scratch). +2. Start an ethrex supernode, i.e. an L1 execution client embedded with an L2 sequencer node. + +```shell +rm -rf dev_ethrex_l*; RUSTFLAGS="-Awarnings" COMPILE_CONTRACTS=true RUST_LOG=off cargo run -r -F l2,l2-sql -- l2 --supernode --block-producer.coinbase-address $(rex a -z) --committer.l1-private-key 0x850643a0224065ecce3882673c21f56bcf6eef86274cc21cadff15930b59fc8c --proof-coordinator.l1-private-key 0xf296c7802555da2a5a662be70e078cbd38b44f96f8615ae529da41122ce8db05 --eth.rpc-url http://localhost:8545 --validium --no-monitor --datadir dev_ethrex_l2 --network ./fixtures/genesis/l2.json --http.port 1729 --committer.commit-time 86400000 + +# Same but enabling logs + +rm -rf dev_ethrex_l*; RUSTFLAGS="-Awarnings" COMPILE_CONTRACTS=true RUST_LOG=info,ethrex_p2p=error,ethrex_l2::sequencer::l1_committer=debug cargo run -r -F l2,l2-sql -- l2 --supernode --block-producer.coinbase-address $(rex a -z) --committer.l1-private-key 0x850643a0224065ecce3882673c21f56bcf6eef86274cc21cadff15930b59fc8c --proof-coordinator.l1-private-key 0xf296c7802555da2a5a662be70e078cbd38b44f96f8615ae529da41122ce8db05 --eth.rpc-url http://localhost:8545 --validium --no-monitor --datadir dev_ethrex_l2 --network ./fixtures/genesis/l2.json --http.port 1729 --committer.commit-time 86400000 +``` + +### Testing L1 -> L2 synchronous composability + +#### Synchronous Deposits + +```shell +rex transfer 999999999999999999 0x67cad0d689b799f385d2ebcf3a626254a9074e12 0x41443995d9eb6c6d6df51e55db2b188b12fe0f80d32817e57e11c64acff1feb8 +``` + +#### L1 contract calling into an L2 contract + +```shell +# Deploy a Counter.sol contract in the L1 + +rex deploy --contract-path crates/l2/contracts/src/example/Counter.sol 0 0x41443995d9eb6c6d6df51e55db2b188b12fe0f80d32817e57e11c64acff1feb8 --remappings "" + +# Update that contract state by statically calling a contract in the L2 + +rex send 0x3fe21258005ca065695d205aac21168259e58155 "update(address)" 0x67cad0d689b799f385d2ebcf3a626254a9074e12 --private-key 0x41443995d9eb6c6d6df51e55db2b188b12fe0f80d32817e57e11c64acff1feb8 +``` + +### Testing L2 -> L1 synchronous composability + +#### Synchronous Withdrawals + +```shell +# Deposit +rex transfer 999999999999999999 0x67cad0d689b799f385d2ebcf3a626254a9074e12 0x41443995d9eb6c6d6df51e55db2b188b12fe0f80d32817e57e11c64acff1feb8 + +# Withdrawal +rex l2 withdraw 111111111111111111 0x41443995d9eb6c6d6df51e55db2b188b12fe0f80d32817e57e11c64acff1feb8 +``` + +## Introduction + +### L1 -> L2 Synchronous Composability + +#### Synchronous Deposits + +Deposits are the process by which L1 users can enter L2 in some form. This process begins and ends on L1 through a series of steps: + +1. **Initiate the deposit on L1**: + - A user sends a transaction to L1, either via an ETH transfer to the `CommonBridge` contract or by calling the `deposit` function on the same contract. Both actions execute the same logic, which, upon successful execution, emits a log containing the necessary information for the sequencer of the corresponding L2 to process it. + - This transaction must be included in a block, and that block must be finalized for the sequencer on the corresponding L2 to detect the log on L1. +2. **Process the deposit on L2**: + - When the sequencer processes this log, it includes a transaction in its mempool that mints the corresponding ETH to the recipient's address, thereby ensuring the recipient has funds on L2. +3. **Commit the deposit process from L2 to L1**: + - Eventually, the L2 batch that includes this mint transaction is sealed and committed to L1. This commit transaction must be included in an L1 block and finalized. + - The same batch is sent to a prover to generate a ZK proof validating the previously committed batch. +4. **Verify the deposit process from L2 to L1 (deposit finalization)**: + - Eventually, the batch execution proof is generated and returned to the sequencer, which submits it for verification on L1 via a `verify` transaction. + - The `verify` transaction, assuming it is valid, must be included in an L1 block and finalized. + +This 4-step process requires, by definition, that it occur across different L1 slots. The number of slots needed can vary based on L1's configuration, but even assuming a sufficiently fast commit time, real-time proving to generate the proof quickly, and a sufficiently fast proof submission time, this process would still require at least 2 slots: the first is always mandatory to emit the log that the sequencer listens for, and with significant luck, finalization could occur in the next slot. + +Synchronous Composability enables this entire process to happen within the same L1 slot. In other words, the transaction that initiates the deposit, the deposit processing on L2, the commit transaction for the batch that included the mint, the generation of the execution proof for that batch, and the verify transaction for the same batch all occur in the same L1 slot. + +#### L1 Contract Calling into an L2 Contract + +Another capability enabled by synchronous composability is the ability to call L2 contracts from L1. + +A simple example of this would be updating the state of a counter on L1 with the current state of another counter that resides on L2. + +Unlike deposits, which do not require synchronous composability to function normally, calling an L2 contract from L1 and using the result as part of the L1 execution is not possible without this feature. + +### L2 -> L1 Synchronous Composability + +TBD + +### Rollup Requirements for SC and How We Addressed Them in the PoC + +To achieve synchronous composability, our rollup needed to fulfill the following requirements: + +1. **Reorg with L1**: The rollup consumes unconfirmed L1 data and therefore must reorganize (reorg) with L1. +2. **Instant Settlement**: The rollup must be able to settle within one L1 slot, requiring real-time proving. +3. **Coordinated Sequencing**: The L2 proposer is the L1 proposer or works closely together (e.g., issues L1 inclusion preconfs). + +We addressed these requirements in the following manner: + +1. For this PoC, we removed reorgs from the equation. +2. Our L2 block builder would force a commit batch transaction after building a block that includes a scoped call. Assuming real-time proving by skipping verification, the commit transaction now serves as a settlement transaction. +3. We extended the ethrex functionality with a supernode mode that operates essentially as an L1 and L2 node sharing both states. This allows the L1 to insert transactions into the L2 mempool and simulate the L2 state in real time, while the L2 can insert transactions into the L1 mempool and simulate the L1 state in real time. + +## Future work + +TBD diff --git a/docs/l2/deployment/validium.md b/docs/l2/deployment/validium.md new file mode 100644 index 00000000000..92ccf615bae --- /dev/null +++ b/docs/l2/deployment/validium.md @@ -0,0 +1,84 @@ +# Deploying a validium ethrex L2 + +In this section, we'll cover how to deploy a validium ethrex L2 on a public network such as Holesky, Sepolia, or Mainnet. + +## Prerequisites + +This guide assumes that you have ethrex installed and available in your PATH. If you haven't installed it yet, follow one of the methods in the Installation Guide. If you want to build the binary from source, refer to the [Building from source](./overview.md#building-from-source-skip-if-ethrex-is-already-installed) section and select the appropriate build option. + +## 1. Deploy the Contracts + +First, deploy and initialize the contracts on L1 using the ethrex l2 deploy command (for more details on the ethrex CLI, see the ethrex CLI Reference section): + +```shell +ethrex l2 deploy \ + --validium true \ + --eth-rpc-url \ + --private-key \ + --genesis-l2-path \ + --bridge-owner \ + --on-chain-proposer-owner \ + --committer.l1-address \ + --proof-sender.l1-address \ + --env-file-path \ + --randomize-contract-deployment +``` + +> [!CAUTION] +> Ensure you control the Committer and Proof Sender accounts, as they will be authorized as sequencers. These accounts will have control over the chain state. + +> [!IMPORTANT] +> If you plan to prove your L2 using SP1, RISC0, or TEE, add the following extra arguments to the command above: +> `--sp1 true` to require SP1 proofs for validating batch execution and state settlement. +> `--sp1.verifier-address` to use an existing verifier instead of deploying one on the public network. Succinct Labs recommends their deployed canonical verifier gateways; see the list here. +> `--risc0 true` to require RISC0 proofs for validating batch execution and state settlement. +> `--risc0.verifier-address` to use an existing verifier instead of deploying one on the public network. RISC0 recommends their deployed canonical verifier gateways; see the list here. +> `--tdx true` to require TEE proofs for validating batch execution and state settlement. +> `--tdx.verifier-address` to use an existing verifier instead of deploying one on the public network. Do not pass this flag if you want to deploy a new verifier. +> Enabling multiple proving backend will require running multiple provers, one for each backend. Refer to the [Run multiple provers](./prover/multi-prover.md) section for more details. + +> [!IMPORTANT] +> Retrieve the deployed contract addresses from the console logs or the .env file generated during deployment (in the directory where you ran the command) for use in the next step. + +> [!NOTE] +> +> - Replace `L1_RPC_URL` with your preferred RPC provider endpoint. +> - Replace `PRIVATE_KEY` with the private key of an account funded on the target L1. This key will sign the transactions during deployment. +> - Replace `PATH_TO_L2_GENESIS_FILE` with the path to your L2 genesis file.A genesis example is available in the fixtures directory of the [official GitHub repository](https://github.com/lambdaclass/ethrex/blob/main/fixtures/genesis/l2.json). This file initializes the `OnChainProposer` contract with the genesis state root. +> - The `CommonBridge` and `OnChainProposer` contracts are upgradeable and ownable, with implementations behind proxies initialized during deployment. Replace `COMMON_BRIDGE_OWNER_ADDRESS` and `ON_CHAIN_PROPOSER_OWNER_ADDRESS` with the address of the account you want as the owner. The owner can upgrade implementations or perform administrative actions; for more details, see the Architecture section. +> - The sequencer components (`L1Committer` and `L1ProofSender`) require funded accounts on the target L1 to advance the network. Replace `L1_COMMITTER_ADDRESS` and `L1_PROOF_SENDER_ADDRESS` with the addresses of those accounts. +> - Replace `PATH_TO_ENV_FILE` with the path where you want to save the generated environment file. This file contains the deployed contract addresses and other configuration details needed to run the L2 node. +> - L1 contract deployment uses the `CREATE2` opcode for deterministic addresses. To deploy non-deterministically, include the `--randomize-contract-deployment` flag. + +## 2. Start the L2 node + +Once the contracts are deployed, start the L2 node: + +```shell +ethrex l2 \ + --validium \ + --l1.bridge-address \ + --l1.on-chain-proposer-address \ + --block-producer.coinbase-address \ + --proof-coordinator.l1-private-key \ + --committer.l1-private-key \ + --eth.rpc-url \ + --network \ + --no-monitor +``` + +> [!CAUTION] +> Replace `L1_COMMITTER_PRIVATE_KEY` and `L1_PROOF_SENDER_PRIVATE_KEY` with the private keys for the `L1_COMMITTER_ADDRESS` and `L1_PROOF_SENDER_ADDRESS` used in the deployment step, respectively. + +> [!IMPORTANT] +> +> The L1 Committer and L1 Proof Sender accounts must be funded for the chain to advance. + +> [!NOTE] +> +> - Replace `COMMON_BRIDGE_ADDRESS` and `ON_CHAIN_PROPOSER_ADDRESS` with the proxy addresses for the CommonBridge and OnChainProposer contracts from the deployment step. +> - Replace `L2_COINBASE_ADDRESS` with the address that will collect L2 block fees. To access these funds on L1, you'll need to withdraw them (see the Withdrawals section for details). +> - Replace `L1_PROOF_SENDER_PRIVATE_KEY` and `L1_COMMITTER_PRIVATE_KEY` with the private keys for the `L1_PROOF_SENDER_ADDRESS` and `L1_COMMITTER_ADDRESS` from the deployment step. +> - Replace `L1_RPC_URL` and `PATH_TO_L2_GENESIS_FILE` with the same values used in the deployment step. + +That's it! You now have a validium ethrex L2 up and running. However, one key component is still missing: state proving. The L2 state is considered final only after a batch execution ZK proof is successfully verified on-chain. Generating these proofs requires running a dedicated prover, which is covered in the Run an ethrex L2 Prover section. diff --git a/docs/l2/deployment/vanilla.md b/docs/l2/deployment/vanilla.md new file mode 100644 index 00000000000..8acc5afe047 --- /dev/null +++ b/docs/l2/deployment/vanilla.md @@ -0,0 +1,82 @@ +# Deploying a vanilla ethrex L2 + +In this section, we'll cover how to deploy a vanilla ethrex L2 on a public network such as Holesky, Sepolia, or Mainnet. + +## Prerequisites + +This guide assumes that you have ethrex installed and available in your PATH. If you haven't installed it yet, follow one of the methods in the Installation Guide. If you want to build the binary from source, refer to the [Building from source](./overview.md#building-from-source-skip-if-ethrex-is-already-installed) section and select the appropriate build option. + +## 1. Deploy the Contracts + +First, deploy and initialize the contracts on L1 using the ethrex l2 deploy command (for more details on the ethrex CLI, see the ethrex CLI Reference section): + +```shell +ethrex l2 deploy \ + --eth-rpc-url \ + --private-key \ + --genesis-l2-path \ + --bridge-owner \ + --on-chain-proposer-owner \ + --committer.l1-address \ + --proof-sender.l1-address \ + --env-file-path \ + --randomize-contract-deployment +``` + +> [!CAUTION] +> Ensure you control the Committer and Proof Sender accounts, as they will be authorized as sequencers. These accounts will have control over the chain state. + +> [!IMPORTANT] +> If you plan to prove your L2 using SP1, RISC0, or TDX, add the following extra arguments to the command above: +> `--sp1 true` to require SP1 proofs for validating batch execution and state settlement. +> `--sp1.verifier-address` to use an existing verifier instead of deploying one on the public network. Succinct Labs recommends their deployed canonical verifier gateways; see the list here. +> `--risc0 true` to require RISC0 proofs for validating batch execution and state settlement. +> `--risc0.verifier-address` to use an existing verifier instead of deploying one on the public network. RISC0 recommends their deployed canonical verifier gateways; see the list here. +> `--tdx true` to require TEE proofs for validating batch execution and state settlement. +> `--tdx.verifier-address` to use an existing verifier instead of deploying one on the public network. Do not pass this flag if you want to deploy a new verifier. +> Enabling multiple proving backend will require running multiple provers, one for each backend. Refer to the [Run multiple provers](./prover/multi-prover.md) section for more details. + +> [!IMPORTANT] +> Retrieve the deployed contract addresses from the console logs or the .env file generated during deployment (in the directory where you ran the command) for use in the next step. + +> [!NOTE] +> +> - Replace `L1_RPC_URL` with your preferred RPC provider endpoint. +> - Replace `PRIVATE_KEY` with the private key of an account funded on the target L1. This key will sign the transactions during deployment. +> - Replace `PATH_TO_L2_GENESIS_FILE` with the path to your L2 genesis file. A genesis example is available in the fixtures directory of the [official GitHub repository](https://github.com/lambdaclass/ethrex/blob/main/fixtures/genesis/l2.json). This file initializes the `OnChainProposer` contract with the genesis state root. +> - The `CommonBridge` and `OnChainProposer` contracts are upgradeable and ownable, with implementations behind proxies initialized during deployment. Replace `COMMON_BRIDGE_OWNER_ADDRESS` and `ON_CHAIN_PROPOSER_OWNER_ADDRESS` with the address of the account you want as the owner. The owner can upgrade implementations or perform administrative actions; for more details, see the Architecture section. +> - The sequencer components (`L1Committer` and `L1ProofSender`) require funded accounts on the target L1 to advance the network. Replace `L1_COMMITTER_ADDRESS` and `L1_PROOF_SENDER_ADDRESS` with the addresses of those accounts. +> - Replace `PATH_TO_ENV_FILE` with the path where you want to save the generated environment file. This file contains the deployed contract addresses and other configuration details needed to run the L2 node. +> - L1 contract deployment uses the `CREATE2` opcode for deterministic addresses. To deploy non-deterministically, include the `--randomize-contract-deployment` flag. + +## 2. Start the L2 node + +Once the contracts are deployed, start the L2 node: + +```shell +ethrex l2 \ + --l1.bridge-address \ + --l1.on-chain-proposer-address \ + --block-producer.coinbase-address \ + --proof-coordinator.l1-private-key \ + --committer.l1-private-key \ + --eth.rpc-url \ + --network \ + --no-monitor +``` + +> [!CAUTION] +> Replace `L1_COMMITTER_PRIVATE_KEY` and `L1_PROOF_SENDER_PRIVATE_KEY` with the private keys for the `L1_COMMITTER_ADDRESS` and `L1_PROOF_SENDER_ADDRESS` used in the deployment step, respectively. + +> [!IMPORTANT] +> +> The L1 Committer and L1 Proof Sender accounts must be funded for the chain to advance. + +> [!NOTE] +> +> - Replace `COMMON_BRIDGE_ADDRESS` and `ON_CHAIN_PROPOSER_ADDRESS` with the proxy addresses for the CommonBridge and OnChainProposer contracts from the deployment step. +> - Replace `L2_COINBASE_ADDRESS` with the address that will collect L2 block fees. To access these funds on L1, you'll need to withdraw them (see the Withdrawals section for details). +> - Replace `L1_PROOF_SENDER_PRIVATE_KEY` and `L1_COMMITTER_PRIVATE_KEY` with the private keys for the `L1_PROOF_SENDER_ADDRESS` and `L1_COMMITTER_ADDRESS` from the deployment step. +> - Replace `L1_RPC_URL` and `PATH_TO_L2_GENESIS_FILE` with the same values used in the deployment step. + +That's it! You now have a vanilla ethrex L2 up and running. However, one key component is still missing: state proving. The L2 state is considered final only after a batch execution ZK proof is successfully verified on-chain. Generating these proofs requires running a dedicated prover, which is covered in the Run an ethrex L2 Prover section. diff --git a/docs/l2/fundamentals/data_availability.md b/docs/l2/fundamentals/data_availability.md new file mode 100644 index 00000000000..9fc38cc2849 --- /dev/null +++ b/docs/l2/fundamentals/data_availability.md @@ -0,0 +1,39 @@ +# Reconstructing state or Data Availability + +Rollups, unlike validiums, need to have their state on L1. If the only thing that's published to it is the state root, then the sequencer could withhold data on the state of the chain. Because it is the one proposing and executing blocks, if it refuses to deliver certain data (like a merkle path to prove a withdrawal on L1), people may not have any place to get it from and get locked out of the chain or some of their funds. + +This is called the **Data Availability** problem. Sending the entire state of the chain on every new L2 batch is impossible; state is too big. As a first next step, what we could do is: + +- For every new L2 batch, send as part of the `commit` transaction the list of transactions in the batch. Anyone who needs to access the state of the L2 at any point in time can track all `commit` transactions, start executing them from the beginning and reconstruct the state. + +This is now feasible; if we take 200 bytes as a rough estimate for the size of a single transfer between two users (see [this post](https://ethereum.stackexchange.com/questions/30175/what-is-the-size-bytes-of-a-simple-ethereum-transaction-versus-a-bitcoin-trans) for the calculation on legacy transactions) and 128 KB as [a reasonable transaction size limit](https://github.com/ethereum/go-ethereum/blob/830f3c764c21f0d314ae0f7e60d6dd581dc540ce/core/txpool/legacypool/legacypool.go#L49-L53) we get around ~650 transactions at maximum per `commit` transaction (we are assuming we use calldata here, blobs can increase this limit as each one is 128 KB and we could use multiple per transaction). + +Going a bit further, instead of posting the entire transaction, we could just post which accounts have been modified and their new values (this includes deployed contracts and their bytecode of course). This can reduce the size a lot for most cases; in the case of a regular transfer as above, we only need to record balance updates of two accounts, which requires sending just two `(address, balance)` pairs, so (20 + 32) * 2 = 104 bytes, or around half as before. Some other clever techniques and compression algorithms can push down the publishing cost of this and other transactions much further. + +This is called `state diffs`. Instead of publishing entire transactions for data availability, we only publish whatever state they modified. This is enough for anyone to reconstruct the entire state of the chain. + +Detailed documentation on [the state diffs spec](./state_diffs.md). + +### How do we prevent the sequencer from publishing the wrong state diffs? + +Once again, state diffs have to be part of the public input. With them, the prover can show that they are equal to the ones returned by the VM after executing all blocks in the batch. As always, the actual state diffs are not part of the public input, but **their hash** is, so the size is a fixed 32 bytes. This hash is then part of the batch commitment. The prover then assures us that the given state diff hash is correct (i.e. it exactly corresponds to the changes in state of the executed blocks). + +There's still a problem however: the L1 contract needs to have the actual state diff for data availability, not just the hash. This is sent as part of calldata of the `commit` transaction (actually later as a blob, we'll get to that), so the sequencer could in theory send the wrong state diff. To make sure this can't happen, the L1 contract hashes it to make sure that it matches the actual state diff hash that is included as part of the public input. + +With that, we can be sure that state diffs are published and that they are correct. The sequencer cannot mess with them at all; either it publishes the correct state diffs or the L1 contract will reject its batch. + +### Compression + +Because state diffs are compressed to save space on L1, this compression needs to be proven as well. Otherwise, once again, the sequencer could send the wrong (compressed) state diffs. This is easy though, we just make the prover run the compression and we're done. + +## EIP 4844 (a.k.a. Blobs) + +While we could send state diffs through calldata, there is a (hopefully) cheaper way to do it: blobs. The Ethereum Cancun upgrade introduced a new type of transaction where users can submit a list of opaque blobs of data, each one of size at most 128 KB. The main purpose of this new type of transaction is precisely to be used by rollups for data availability; they are priced separately through a `blob_gas` market instead of the regular `gas` one and for all intents and purposes should be much cheaper than calldata. + +Using EIP 4844, our state diffs would now be sent through blobs. While this is cheaper, there's a new problem to address with it. The whole point of blobs is that they're cheaper because they are only kept around for approximately two weeks and ONLY in the beacon chain, i.e. the consensus side. The execution side (and thus the EVM when running contracts) does not have access to the contents of a blob. Instead, the only thing it has access to is a **KZG commitment** of it. + +This is important. If you recall, the way the L1 ensured that the state diff published by the sequencer was correct was by hashing its contents and ensuring that the hash matched the given state diff hash. With the contents of the state diff now no longer accessible by the contract, we can't do that anymore, so we need another way to ensure the correct contents of the state diff (i.e. the blob). + +The solution is to make the prover take the KZG commitment as a public input and the KZG proof as a private input, compute the state diffs after correctly executing a batch of blocks, and verify the proof to check that the commitment binds to the correct state diffs. + +Because the KZG commitment is a public input, we can use the `BLOBHASH` EVM opcode to retrieve the blob rolling hash (which is just the hash of the KZG commitment and some other constant data) and compare it to the public input KZG commitment (which needs to be hashed too). diff --git a/docs/l2/fundamentals/execution_witness.md b/docs/l2/fundamentals/execution_witness.md new file mode 100644 index 00000000000..f21d054b2f4 --- /dev/null +++ b/docs/l2/fundamentals/execution_witness.md @@ -0,0 +1,37 @@ +# Execution witness + +The purpose of the execution witness is to allow executing blocks without having access to the whole Ethereum state, as it wouldn't fit in a zkVM program. It contains only the state values needed during the execution. + +An execution witness contains all the initial state values (state nodes, codes, storage keys, block headers) that will be read or written to during the blocks' execution. + +An execution witness is created from a prior execution of the blocks. This execution can be done by a synced node, and it would expose the data to a RPC endpoint. This is the `debug_executionWitness` endpoint which is implemented by ethrex and other clients. + +If this endpoint is not available, the prover needs to do the following: + +1. execute the blocks (also called "pre-execution") to identify which state values will be accessed. +2. log every initial state value accessed or updated during this execution. +3. retrieve an MPT proof for each value, linking it (or its non-existence) to the initial state root hash, using the `eth_getProof` RPC endpoint of a synced node. + +Steps 1 and 2 are data collection steps only - no validation is performed at this stage. The actual validation happens later inside the zkVM guest program. Step 3 involves more complex logic due to potential issues when restructuring the pruned state trie after value removals. In sections [initial state validation](../../prover/guest_program.md#step-1-initial-state-validation) and [final state validation](../../prover/guest_program.md#step-3-final-state-validation) we explain what are pruned tries and in which case they get restructured. + +If a value is removed during block execution (meaning it existed initially but not finally), two pathological cases can occur where the witness lacks sufficient information to update the trie structure correctly: + +**Case 1** + +![Image showing restructuration for case 1](../img/execw_case1.png) + +Here, only **leaf 1** is part of the execution witness, so we lack the proof (and thus the node data) for **leaf 2**. After removing **leaf 1**, **branch 1** becomes redundant. During trie restructuring, it's replaced by **leaf 3**, whose path is the path of **leaf 2** concatenated with a prefix nibble (`k`) representing the choice taken at the original **branch 1**, and keeping **leaf 2**'s value. + +```text +branch1 = {c_1, c_2, ..., c_k, ..., c_16} # Only c_k = hash(leaf2) is non-empty +leaf2 = {value, path} +leaf3 = {value, concat(k, path)} # New leaf replacing branch1 and leaf2 +``` + +Without **leaf 2**'s data, we cannot construct **leaf 3**. The solution is to fetch the _final_ state proof for the key of **leaf 2**. This yields an exclusion proof containing **leaf 3**. By removing the prefix nibble `k`, we can reconstruct the original path and value of **leaf 2**. This process might need to be repeated if similar restructuring occurred at higher levels of the trie. + +**Case 2** + +![Image showing restructuration for case 2](../img/execw_case2.png) + +In this case, restructuring requires information about **branch/ext 2** (which could be a branch or extension node), but this node might not be in the witness. Checking the final **extension** node might seem sufficient to deduce **branch/ext 2** in simple scenarios. However, this fails if similar restructuring occurred at higher trie levels involving more removals, as the final **extension** node might combine paths from multiple original branches, making it ambiguous to reconstruct the specific missing **branch/ext 2** node. diff --git a/docs/l2/interacting/blockscout.md b/docs/l2/interacting/blockscout.md new file mode 100644 index 00000000000..537b8f7c002 --- /dev/null +++ b/docs/l2/interacting/blockscout.md @@ -0,0 +1,3 @@ +# Blockscout for ethrex L2 + +TDB diff --git a/docs/l2/interacting/l2_hub.md b/docs/l2/interacting/l2_hub.md new file mode 100644 index 00000000000..9cd4f96dbbb --- /dev/null +++ b/docs/l2/interacting/l2_hub.md @@ -0,0 +1,3 @@ +# L2 Hub + +TDB diff --git a/docs/l2/introduction.md b/docs/l2/introduction.md index 65f9c5606c7..76100accfdb 100644 --- a/docs/l2/introduction.md +++ b/docs/l2/introduction.md @@ -6,4 +6,4 @@ Layer 2 (L2) solutions are protocols built on top of Ethereum to increase scalab ## Get started with your L2 -Check out the [Quickstart L2 guide](../getting-started#quickstart-l2) to start your rollup in just a command, or jump right into the [Deploy an L2](./deploy.md) for more detailed instructions. +Check out the [Quickstart L2 guide](../getting-started/README.md#quickstart-l2) to start your rollup in just a command, or jump right into the [Deploy an L2](../l2/deployment/overview.md) for more detailed instructions. diff --git a/docs/prover/guest_program.md b/docs/prover/guest_program.md new file mode 100644 index 00000000000..5be18464d9a --- /dev/null +++ b/docs/prover/guest_program.md @@ -0,0 +1,88 @@ +# ethrex-prover guest program + +The guest program is the code that is compiled into a zkVM-compatible binary (e.g., RISC-V), to then generate a zero-knowledge proof of its execution. + +## Program inputs + +The inputs for the blocks execution program (also called prover inputs) are: + +- `blocks`: The blocks to be proven (header and body). +- `execution_witness`: A structure containing the necessary state data (like account and storage values with their Merkle proofs) required for the execution of the blocks. It includes the parent header of the first block. +- `elasticity_multiplier`: A parameter for block validation. +- `fee_configs`: L2-specific fee configurations for each block. +- `blob_commitment` and `blob_proof`: L2-specific data for verifying the state diff blob. + +These inputs are required for proof generation. The public values of the proof (also called program outputs), which are needed for proof verification, are: + +- `initial_state_hash`: The state root from the parent header of the first block. +- `final_state_hash`: The state root from the header of the last block. +- `l1messages_merkle_root`: The Merkle root of L1 messages (withdrawals) generated during block execution. +- `privileged_transactions_hash`: A hash representing all privileged transactions processed in the blocks. +- `blob_versioned_hash`: The versioned hash of the state diff blob, derived from its KZG commitment. +- `last_block_hash`: The hash of the last block in the batch. +- `chain_id`: The chain ID of the network. +- `non_privileged_count`: The number of non-privileged transactions in the batch. + +## Blocks execution program + +The program leverages `ethrex-common` primitives and `ethrex-vm` methods. `ethrex-prover` implements a program that uses the existing execution logic and generates a proof of its execution using a zkVM. Some L2-specific logic and input validation are added on top of the basic blocks execution. + +The following sections outline the steps taken by the execution program. + +### Prelude 1: state trie basics + +We recommend learning about Merkle Patricia Tries (MPTs) to better understand this section. + +Each executed block transitions the Ethereum state from an initial state to a final state. State values are stored in MPTs: + +1. Each account has a Storage Trie containing its storage values. +2. The World State Trie contains all account information, including each account's storage root hash (linking storage tries to the world trie). + +Hashing the root node of the world state trie generates a unique identifier for a particular Ethereum state, known as the "state hash". + +There are two kinds of MPT proofs: + +1. Inclusion proofs: Prove that `key: value` is a valid entry in the MPT with root hash `h`. +2. Exclusion proofs: Prove that `key` does not exist in the MPT with root hash `h`. + These proofs allow verifying that a value is included (or its key doesn't exist) in a specific state. + +### Prelude 2: privileged transactions, L1 messages and state diffs + +These three components are specific additions for ethrex's L2 protocol, layered on top of standard Ethereum execution logic. They each require specific validation steps within the program. + +For more details, refer to [Overview](../l2/architecture/overview.md), [Withdrawals](../l2/fundamentals/withdrawals.md), and [State diffs](../l2/fundamentals/state_diffs.md). + +### Step 1: initial state validation + +The program validates the initial state by converting the `ExecutionWitness` into a `GuestProgramState` and verifying that its trie structure correctly represents the expected state. This involves checking that the calculated state trie root hash matches the initial state hash (obtained from the first block's parent block header). + +The validation happens in several steps: + +1. The `ExecutionWitness` (collected during pre-execution) is converted to `GuestProgramState`. +2. A `GuestProgramStateWrapper` is created to provide database functionality. +3. For each state value in the database (account state and storage slots), the program verifies merkle proofs of the inclusion (or exclusion, in the case of accounts that didn't exist before this batch) of the value in the state trie +4. The state trie root is compared against the first block's parent state root. + +This validation ensures that all state values needed for execution are properly linked to the initial state via their MPT proofs. Having the initial state proofs (paths from the root to each relevant leaf) is equivalent to having a relevant subset of the world state trie and storage tries - a set of "pruned tries". This allows operating directly on these pruned tries (adding, removing, modifying values) during execution. + +### Step 2: blocks execution + +After validating the initial state, the program executes the blocks sequentially. This leverages the existing `ethrex-vm` execution logic. For each block, it performs validation checks and then executes the transactions within it. State changes from each block are applied before executing the next one. + +### Step 3: final state validation + +During execution, state values are updated (modified, created, or removed). After executing all blocks, the program calculates the final state by applying all state updates to the initial pruned tries. + +Applying the updates results in a new world state root node for the pruned tries. Hashing this node yields the calculated final state hash. The program then verifies that this calculated hash matches the expected final state hash (from the last block header), thus validating the final state. + +### Step 4: privileged transactions hash calculation + +After execution and final state validation, the program calculates a hash encompassing all privileged transactions (like L1 to L2 deposits) processed within the blocks. This hash is committed as a public input, required for verification on the L1 bridge contract. + +### Step 5: L1 messages Merkle root calculation + +Similarly, the program constructs a binary Merkle tree of all L2->L1 messages (withdrawals) initiated in the blocks and calculates its root hash. This hash is also committed as a public input. Later, L1 accounts can claim their withdrawals by providing a Merkle proof of inclusion that validates against this root hash on the L1 bridge contract. + +### Step 6: state diff calculation and commitment + +Finally, the program calculates the state diffs (changes between initial and final state) intended for publication to L1 as blob data. It then verifies the provided `blob_commitment` and `blob_proof` against the calculated state diff. The resulting `blob_versioned_hash` (derived from the KZG commitment) is committed as a public input for verification on the L1 contract. diff --git a/docs/prover/prover.md b/docs/prover/prover.md new file mode 100644 index 00000000000..9a56361d366 --- /dev/null +++ b/docs/prover/prover.md @@ -0,0 +1,52 @@ +# ethrex-prover + +The prover leverages ethrex's stateless execution to generate zero-knowledge proofs of a block (or batch of blocks) execution. Stateless execution works by asking a synced node for an execution witness (the minimal state data needed to execute that block or batch) and using the L1 client code to re-execute it. See [Stateless execution](#stateless-execution-and-execution-witness) for more details. + +The main interface to try the prover is [ethrex-replay](../ethrex_replay/ethrex_replay.md), we also use it as a component of ethrex's L2 stack, to deploy zk-rollups or zk-validiums (a rollup publishes state information to L1 to reconstruct the L2 state if sequencing were to fail, a validium does not). Because of this the prover also supports some [L2 specific checks](#l2-specific-checks). + +## How do you prove block execution? + +Now that general purpose zero-knowledge virtual machines (zkVMs) exist, most people have little trouble with the idea that you can prove execution. Just take the usual EVM code you wrote in Rust, compile to some zkVM target instead and you're mostly done. You can now prove it. + +What's usually less clear is how you prove state. Let's say we want to prove a new L2 batch of blocks that were just built. Running the `ethrex` `execute_block` function on a Rust zkVM for all the blocks in the batch does the trick, but that only proves that you ran the VM correctly on **some** previous state/batch. How do you know it was the actual previous state of the L2 and not some other, modified one? + +In other words, how do you ensure that: + +- Every time the EVM **reads** from some account state or storage slot (think an account balance, some contract's bytecode), the value returned matches the actual value present on the previous state of the chain. +- When all **writes** are done to account states or storage slots after execution, the final state matches what the (last executed) block header specified is the state at that block (the header contains the final state MPT root). + +### Stateless execution and execution witness + +Ethrex implements a way to execute a block (or a batch of blocks) without having access to the entire blockchain state, but only the necessary subset for that particular execution. This subset is called the *execution witness*, and running a block this way is called *stateless execution* (stateless in the sense that you don't need a database with hundreds of gigabytes of the entire state data to execute). + +The execution witness is composed of all MPT nodes which are relevant to the execution, so that for each read and write we have all the nodes that form a path from the root to the relevant leaf. This path is a proof that this particular value we read/wrote is part (or not) of the initial or final state MPT. + +So, before initiating block execution, we can verify each proof for each state value read from. After execution, we can verify each proof for each state value written to. After these steps we authenticated all state data to two MPT root hashes (initial and final state roots), which later can be compared against reference values to check that the execution started from and arrived to the correct state. If you were to change a single bit, this comparison would fail. + +### In a zkVM environment + +After stateless execution was done, the initial and final state roots can be committed as public values of the zk proof. By verifying the proof we know that blocks were executed from an initial state and arrived into a final state, and we know the root hashes of the MPT of each one. If the initial root is what we expected (equal to the root of the latest validated state), then we trustlessly verified that the chain advanced its state correctly, and we can authenticate the new, valid state using the final state root. + +By proving the execution of L2 blocks and verifying the zk proof (alongside with the initial state root) in an Ethereum smart contract validators attest the new state and the L2 inherits the security of Ethereum (assuming no bugs in the whole pipeline). This is the objective of an Ethereum L2. + +Validators themselves could verify L1 block execution proofs to attest Ethereum instead of re-executing. + +## L2 specific checks + +Apart from stateless execution, the prover does some extra checks needed for L2 specific features. + +### Data availability + +Rollups publish state diffs as blob data to the L1 so that users can reconstruct the L2 state and rescue their funds if the sequencing were to fail or censors data. This published data needs to be part of the zk proof the prover generated. For this it calculates the valid state diffs and verifies a KZG proof, whose commitment can later be compared to the one published to the L1 using the `BLOBHASH` EVM opcode. See [data availability](../l2/fundamentals/data_availability.md) for more details. + +### L1<->L2 messaging + +This is a fundamental feature of an L2, used mainly for bridging assets between the L1 and the L2 or between L2s using the ethrex stack. Messages need to be part of the proof to make sure the sequencer included them correctly. + +Messages are compressed into a hash or a Merkle tree root which then are stored in an L1 contract together with the rest of the L2 state data. The prover retrieves the transactions or events that the messages produced in the L2, reconstructs the message data and recomputes the hashes or Merkle tree roots, which are then committed as a public input of the zk proof. At verification we can compare these hashes with the ones stored in the L1. This is the same concept used for state data. + +For more details checkout [deposits](../l2/fundamentals/deposits.md) and [withdrawals](../l2/fundamentals/withdrawals.md) + +## See also + +[Guest program](./guest_program.md) for the detailed steps of the program that the prover generates a proof of. diff --git a/docs/roadmap.md b/docs/roadmap.md index 8f19b14bf44..5c971b4db85 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -10,16 +10,11 @@ The roadmap below outlines the remaining work required to achieve this milestone | Feature | Description | Status | |----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| -| Native Bridge | Secure and trust-minimized ERC-20 bridge between Ethereum L1 and L2 using canonical messaging and smart contracts. | In Progress | -| Based Rollup | Launch the rollup as a based permissionless rollup. Leverages Ethereum for sequencing and DA. For more information check [ethrex roadmap for becoming based](https://hackmd.io/TCa-bQisToW46enF58_3Vw?view) | In Progress | -| Aligned Integration | Optimize integration with Aligned’s aggregation mode. | In Progress | -| Risc0 Support | Integrate RISC Zero as an alternative zkVM to SP1, enabling configurable proving backends. | In Progress | +| Based Sequencing | Launch the rollup as a based permissionless rollup. Leverages Ethereum for sequencing and DA. For more information check [ethrex roadmap for becoming based](https://hackmd.io/TCa-bQisToW46enF58_3Vw?view) | In Progress | | Battle-Test the Prover | Ensure the prover (e.g., SP1, Risc0) is robust, correct, and performant under production-level conditions. | In Progress | | One-Click L2 Deployment | Deploy a fully operational rollup with a single command. Includes TDX, Prover, integrated Grafana metrics, alerting system, block explorer, bridge hub, backups and default configuration for rapid developer spin-up. | In Progress | -| Shared Bridge | Direct bridging between multiple L2s to improve UX and avoid L1 costs. | Planned | -| Custom Native Token | Define a native token (non-ETH) for gas, staking, incentives, and governance. Fully integrated into fee mechanics and bridging. | Planned | | Validiums & DACs | Enhance Validium mode with Data Availability Committees. | Planned | -| Gas & Fees | Set up a fee token model to price deposits or any forced-included transaction, including data availability costs. | Planned | +| Synchronous Composability | Enable real-time, direct calls between L1 and L2 smart contracts in a single transaction, as if on the same layer. | In Progress | ---