Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Prover CLI): status l1 command #1706

Merged
merged 73 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
17e564c
fmt
ColoCarletti Apr 8, 2024
245b0a4
wip
ColoCarletti Apr 8, 2024
8e84243
add cargo.toml cargo.lock
ColoCarletti Apr 8, 2024
17517b1
go back to anihow
ColoCarletti Apr 8, 2024
4ab7ce8
fmt
ColoCarletti Apr 8, 2024
4655201
rm tool workspace
ColoCarletti Apr 9, 2024
c32d1aa
fmt
ColoCarletti Apr 10, 2024
8e49e04
zk fmt
ColoCarletti Apr 10, 2024
80e13da
fix typo
ColoCarletti Apr 10, 2024
0f2ebd8
move query to prover_dal
ColoCarletti Apr 12, 2024
69d2104
remove --all flag
Apr 15, 2024
c9e6e5e
change command name
ColoCarletti Apr 16, 2024
83fab40
merge main
ColoCarletti Apr 16, 2024
cdbb467
fmt
ColoCarletti Apr 16, 2024
d1f3994
add sqlx query json
ColoCarletti Apr 16, 2024
4ee3ff2
Merge branch 'main' into prover_cli_status_command
ilitteri Apr 16, 2024
fff3910
update README
ColoCarletti Apr 16, 2024
ef906b9
zk fmt
ColoCarletti Apr 16, 2024
3758547
Merge branch 'prover_cli_status_command' of github.com:matter-labs/zk…
ColoCarletti Apr 16, 2024
b3051b2
remove eol
ColoCarletti Apr 16, 2024
0e4841c
command setup
ColoCarletti Apr 16, 2024
3d3d57d
add l1 contracts calls
ColoCarletti Apr 16, 2024
ee9d35b
Move jobs cmd to status cmd module
Apr 16, 2024
82faf32
Refactor status cmd
Apr 16, 2024
aa44864
Update Cargo.lock
Apr 16, 2024
3fe32b6
zk fmt
Apr 16, 2024
2a82aec
add state keeper querys
ColoCarletti Apr 16, 2024
d43dd15
Merge pull request #1705 from matter-labs/il-refactor-prover-cli-stat…
ilitteri Apr 17, 2024
70b0348
Merge branch 'main' into prover_cli_status_command
ilitteri Apr 17, 2024
fb1183e
add print
ColoCarletti Apr 17, 2024
e7c2326
small refactor
ColoCarletti Apr 17, 2024
585059b
Merge branch 'main' into prover_cli_status_command
ilitteri Apr 17, 2024
8aa01ca
add verification keys comparison
ColoCarletti Apr 17, 2024
9977e95
Refactor
Apr 18, 2024
74f6f8c
refactor contract calls
ColoCarletti Apr 18, 2024
ee88cfe
Merge branch 'main' into prover_cli_status_command
ilitteri Apr 18, 2024
0801d9e
add command name print
ColoCarletti Apr 18, 2024
14abe2d
fix print
ColoCarletti Apr 18, 2024
00fac77
rm unused import
ColoCarletti Apr 18, 2024
a29a9e8
fix print
ColoCarletti Apr 18, 2024
872edc2
merge branch prover_cli_status_command
ColoCarletti Apr 18, 2024
7cf62ff
fix status l1 command
ColoCarletti Apr 18, 2024
1e9c3ed
fix query
ColoCarletti Apr 18, 2024
b8b4582
add error handle
ColoCarletti Apr 18, 2024
e06a96f
change query return
ColoCarletti Apr 18, 2024
d2c0dc9
update README
ColoCarletti Apr 19, 2024
ad3445c
fix typos
ColoCarletti Apr 19, 2024
dba8ef1
fmt
ColoCarletti Apr 19, 2024
fabac61
zk spellcheck
ColoCarletti Apr 19, 2024
3999d59
remove batch cmd
ColoCarletti Apr 25, 2024
4348459
rm sqlx old query
ColoCarletti Apr 25, 2024
9039737
Merge branch 'main' into prover_cli_status_l1
ColoCarletti Apr 25, 2024
62ee2d3
Merge branch 'main' into prover_cli_status_l1
ilitteri Apr 30, 2024
720e69b
Merge branch 'main' into prover_cli_status_l1
ilitteri Apr 30, 2024
586161b
Merge branch 'main' of github.com:matter-labs/zksync-era into prover_…
ilitteri Apr 30, 2024
8605afe
zk lint rust
ilitteri Apr 30, 2024
66d232f
Fix import
ilitteri Apr 30, 2024
21c4741
fix README
ColoCarletti Apr 30, 2024
202e04e
Merge branch 'main' into prover_cli_status_l1
ilitteri May 2, 2024
bec81f3
Merge branch 'main' into prover_cli_status_l1
ilitteri May 2, 2024
03a843a
Merge branch 'main' into prover_cli_status_l1
ilitteri May 2, 2024
fa3e0b2
Merge branch 'main' into prover_cli_status_l1
EmilLuta May 3, 2024
5f30839
fix hyperchain contract call
ColoCarletti May 3, 2024
73854f1
merge main
ColoCarletti May 3, 2024
cc9ec3c
fix merge error
ColoCarletti May 3, 2024
485b81b
add tracing
ColoCarletti May 3, 2024
32d781f
Use anyhow instead of thiserror
ilitteri May 6, 2024
a45ba48
merge main
ColoCarletti May 7, 2024
a0d6f70
Merge branch 'main' into prover_cli_status_l1
ilitteri May 8, 2024
33eef67
Add Cargo.lock
ilitteri May 8, 2024
7821eda
fix eth_client
ColoCarletti May 8, 2024
c2a4a07
Merge branch 'main' into prover_cli_status_l1
EmilLuta May 8, 2024
fa2f370
Merge branch 'main' into prover_cli_status_l1
ilitteri May 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
505 changes: 482 additions & 23 deletions prover/Cargo.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions prover/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ zksync_state = { path = "../core/lib/state" }
zksync_system_constants = { path = "../core/lib/constants" }
zksync_types = { path = "../core/lib/types" }
zksync_utils = { path = "../core/lib/utils" }
zksync_eth_client = { path = "../core/lib/eth_client" }
zksync_contracts = { path = "../core/lib/contracts" }

# for `perf` profiling
[profile.perf]
Expand Down
13 changes: 7 additions & 6 deletions prover/prover_cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,10 @@ keywords.workspace = true
categories.workspace = true

[dependencies]
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
env_logger = "0.10"
log = "0.4"
colored = "2.1.0"

tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
clap = { workspace = true, features = ["derive"] }
tracing.workspace = true
tracing-subscriber = { workspace = true, features = ["env-filter"] }
tracing.workspace = true
bincode.workspace = true
hex.workspace = true
anyhow.workspace = true
Expand All @@ -29,4 +25,9 @@ zksync_types.workspace = true
zksync_prover_fri_types.workspace = true
zksync_prover_interface.workspace = true
prover_dal.workspace = true
zksync_eth_client.workspace = true
zksync_contracts.workspace = true
zksync_dal.workspace = true
strum.workspace = true
colored.workspace = true
sqlx.workspace = true
32 changes: 32 additions & 0 deletions prover/prover_cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,38 @@ Scheduler: In progress ⌛️
> Compressor job not found 🚫
```

## Status

### Status l1

Retrieve information about the state of the batches sent to L1 and compare the contract hashes in L1 with those stored
in the prover database.

Example:

```bash
$ zk f run --release -- status l1

====== L1 Status ======
State keeper: First batch: 0, recent batch: 10
L1 state: block verified: 7, block committed: 9
Eth sender is 1 behind. Last block committed: 9. Most recent sealed state keeper batch: 10.
-----------------------
Verifier key hash matches: 0x063c9c1e9d39fc0b1633c78a49f1905s65ee0982ad96d97ef7fe3d4f1f1a72c7
-----------------------
Verification node hash in DB differs from the one in contract.
Contract hash: 0x1186ec268d49f1905f8d9c1e9d39fc33e98c74f91d91a21b8f7ef78bd09a8db8
DB hash: 0x5a3ef282b21e12fe1f4438e5bb158fc5060b160559c5158c6389d62d9fe3d080
-----------------------
Verification leaf hash in DB differs from the one in contract.
Contract hash: 0x101e08b00193e529145ee09823378ef51a3bc8966504064f1f6ba3f1ba863210
DB hash: 0x400a4b532c6f072c00d1806ef299300d4c104f4ac55bd8698ade78894fcadc0a
-----------------------
Verification circuits hash in DB differs from the one in contract.
Contract hash: 0x18c1639094f58177409186e8c48d9f577c9410901d2f1d486b3e7d6cf553ae4c
DB hash: 0x0000000000000000000000000000000000000000000000000000000000000000
```

## File-Info

Displays the information about a given file:
Expand Down
177 changes: 177 additions & 0 deletions prover/prover_cli/src/commands/status/l1.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
use anyhow::Context;
use prover_dal::{Prover, ProverDal};
use zksync_basic_types::{
protocol_version::{L1VerifierConfig, VerifierParams},
L1BatchNumber, H256, U256,
};
use zksync_config::{ContractsConfig, EthConfig, PostgresConfig};
use zksync_dal::{ConnectionPool, Core, CoreDal};
use zksync_env_config::FromEnv;
use zksync_eth_client::{clients::QueryClient, CallFunctionArgs};

pub(crate) async fn run() -> anyhow::Result<()> {
println!(" ====== L1 Status ====== ");
let postgres_config = PostgresConfig::from_env().context("PostgresConfig::from_env")?;
let contracts_config = ContractsConfig::from_env().context("ContractsConfig::from_env()")?;
let eth_config = EthConfig::from_env().context("EthConfig::from_env")?;
let query_client = QueryClient::new(eth_config.web3_url)?;

let total_batches_committed: U256 = CallFunctionArgs::new("getTotalBatchesCommitted", ())
.for_contract(
contracts_config.diamond_proxy_addr,
&zksync_contracts::hyperchain_contract(),
)
.call(&query_client)
.await?;

let total_batches_verified: U256 = CallFunctionArgs::new("getTotalBatchesVerified", ())
.for_contract(
contracts_config.diamond_proxy_addr,
&zksync_contracts::hyperchain_contract(),
)
.call(&query_client)
.await?;

let connection_pool = ConnectionPool::<Core>::builder(
postgres_config
.replica_url()
.context("postgres_config.replica_url()")?,
postgres_config
.max_connections()
.context("postgres_config.max_connections()")?,
)
.build()
.await
.context("ConnectionPoolBuilder::build()")?;

let mut conn = connection_pool.connection().await?;

// Using unwrap() safely as there will always be at least one block.
let first_state_keeper_l1_batch = conn
.blocks_dal()
.get_earliest_l1_batch_number()
.await?
.unwrap();
let last_state_keeper_l1_batch = conn
.blocks_dal()
.get_sealed_l1_batch_number()
.await?
.unwrap();

pretty_print_l1_status(
total_batches_committed,
total_batches_verified,
first_state_keeper_l1_batch,
last_state_keeper_l1_batch,
);

let node_verification_key_hash: H256 = CallFunctionArgs::new("verificationKeyHash", ())
.for_contract(
contracts_config.verifier_addr,
&zksync_contracts::verifier_contract(),
)
.call(&query_client)
.await?;

let node_verifier_params: VerifierParams = CallFunctionArgs::new("getVerifierParams", ())
.for_contract(
contracts_config.diamond_proxy_addr,
&zksync_contracts::hyperchain_contract(),
)
.call(&query_client)
.await?;

let node_l1_verifier_config = L1VerifierConfig {
params: node_verifier_params,
recursion_scheduler_level_vk_hash: node_verification_key_hash,
};

let prover_connection_pool = ConnectionPool::<Prover>::builder(
postgres_config
.prover_url()
.context("postgres_config.replica_url()")?,
postgres_config
.max_connections()
.context("postgres_config.max_connections()")?,
)
.build()
.await
.context("ConnectionPoolBuilder::build()")?;

let mut conn = prover_connection_pool.connection().await.unwrap();

let db_l1_verifier_config = conn
.fri_protocol_versions_dal()
.get_l1_verifier_config()
.await?;

pretty_print_l1_verifier_config(node_l1_verifier_config, db_l1_verifier_config);

Ok(())
}

fn pretty_print_l1_status(
total_batches_committed: U256,
total_batches_verified: U256,
first_state_keeper_l1_batch: L1BatchNumber,
last_state_keeper_l1_batch: L1BatchNumber,
) {
println!(
"State keeper: First batch: {}, recent batch: {}",
first_state_keeper_l1_batch, last_state_keeper_l1_batch
);

println!(
"L1 state: block verified: {}, block committed: {}",
total_batches_verified, total_batches_committed
);

let eth_sender_lag = U256::from(last_state_keeper_l1_batch.0) - total_batches_committed;
if eth_sender_lag > U256::zero() {
println!(
"Eth sender is {} behind. Last block committed: {}. Most recent sealed state keeper batch: {}.",
eth_sender_lag,
total_batches_committed,
last_state_keeper_l1_batch
);
}
}

fn print_hash_comparison(name: &str, contract_hash: H256, db_hash: H256) {
println!(" ----------------------- ");
if contract_hash != db_hash {
println!("{name} hash in DB differs from the one in contract.");
println!("Contract hash: {contract_hash:?}");
println!("DB hash: {db_hash:?}");
} else {
println!("{name} hash matches: {contract_hash:}");
}
}

fn pretty_print_l1_verifier_config(
node_l1_verifier_config: L1VerifierConfig,
db_l1_verifier_config: L1VerifierConfig,
) {
print_hash_comparison(
"Verifier key",
node_l1_verifier_config.recursion_scheduler_level_vk_hash,
db_l1_verifier_config.recursion_scheduler_level_vk_hash,
);
print_hash_comparison(
"Verification node",
node_l1_verifier_config.params.recursion_node_level_vk_hash,
db_l1_verifier_config.params.recursion_node_level_vk_hash,
);
print_hash_comparison(
"Verification leaf",
node_l1_verifier_config.params.recursion_leaf_level_vk_hash,
db_l1_verifier_config.params.recursion_leaf_level_vk_hash,
);
print_hash_comparison(
"Verification circuits",
node_l1_verifier_config
.params
.recursion_circuits_set_vks_hash,
db_l1_verifier_config.params.recursion_circuits_set_vks_hash,
);
}
3 changes: 3 additions & 0 deletions prover/prover_cli/src/commands/status/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,20 @@ use clap::Subcommand;
use crate::cli::ProverCLIConfig;

pub(crate) mod batch;
pub(crate) mod l1;
mod utils;

#[derive(Subcommand)]
pub enum StatusCommand {
Batch(batch::Args),
L1,
}

impl StatusCommand {
pub(crate) async fn run(self, config: ProverCLIConfig) -> anyhow::Result<()> {
match self {
StatusCommand::Batch(args) => batch::run(args, config).await,
StatusCommand::L1 => l1::run().await,
}
}
}
9 changes: 3 additions & 6 deletions prover/prover_cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,13 @@ use prover_cli::cli;

#[tokio::main]
async fn main() {
env_logger::builder()
.filter_module("zksync_db_connection", log::LevelFilter::Off)
.filter_module("sqlx", log::LevelFilter::Off)
.filter_level(log::LevelFilter::Debug)
tracing_subscriber::fmt()
.with_max_level(tracing::Level::ERROR)
.init();

match cli::start().await {
Ok(_) => {}
Err(err) => {
log::error!("{err:?}");
tracing::error!("{err:?}");
std::process::exit(1);
}
}
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 34 additions & 0 deletions prover/prover_dal/src/fri_protocol_versions_dal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,38 @@ impl FriProtocolVersionsDal<'_, '_> {
),
})
}

pub async fn get_l1_verifier_config(&mut self) -> Result<L1VerifierConfig, sqlx::Error> {
let result = sqlx::query!(
ilitteri marked this conversation as resolved.
Show resolved Hide resolved
r#"
SELECT
recursion_scheduler_level_vk_hash,
recursion_node_level_vk_hash,
recursion_leaf_level_vk_hash,
recursion_circuits_set_vks_hash
FROM
prover_fri_protocol_versions
ORDER BY
id DESC
LIMIT 1
"#,
)
.fetch_one(self.storage.conn())
.await?;

let params = VerifierParams {
recursion_node_level_vk_hash: H256::from_slice(&result.recursion_node_level_vk_hash),
recursion_leaf_level_vk_hash: H256::from_slice(&result.recursion_leaf_level_vk_hash),
recursion_circuits_set_vks_hash: H256::from_slice(
&result.recursion_circuits_set_vks_hash,
),
};

Ok(L1VerifierConfig {
params,
recursion_scheduler_level_vk_hash: H256::from_slice(
&result.recursion_scheduler_level_vk_hash,
),
})
}
}
Loading