Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(proof_data_handler): add new endpoints to the TEE prover interface API #1993

Merged
merged 42 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
63214e8
Introduce initial version of TEE request processor
pbeza May 15, 2024
20dee9c
WIP: simplify
pbeza May 16, 2024
7ad4563
Refine the TEE proof submission endpoint
pbeza May 17, 2024
90bef14
Added the initial unit test for the prover interface
pbeza May 20, 2024
377a0ba
Fix the unit test for the /tee_proof_generation_data endpoint
pbeza May 21, 2024
9412d7e
Move tests to a separate file
pbeza May 22, 2024
75cee28
Add a unit test for the /submit_tee_proof endpoint
pbeza May 22, 2024
d952f8a
Create a new SQL table to track TEE proof generation details
pbeza May 23, 2024
e5edaa3
Add .sqlx `query-*.json` files
pbeza May 23, 2024
b468c30
Add one more serialization test to cover both SubmitProofRequest and …
pbeza May 23, 2024
c140233
Utilize mocked SQL table in unit tests for the /tee_proof_generation_…
pbeza May 23, 2024
9ea9cfe
Revise database schema and update dependent code
pbeza May 24, 2024
71a263c
Fix request_tee_proof_generation_data unit test by
pbeza May 27, 2024
aca8810
Refine /submit_tee_proof endpoint unit test
pbeza May 27, 2024
d763c9c
Address CI code formatting complaints
pbeza May 27, 2024
ac83fa5
Reformat SQL queries using `zk fmt`
pbeza May 28, 2024
f985d62
Simplify unit tests
pbeza May 28, 2024
c0a8d34
Rename /tee_proof_generation_data endpoint to /tee_proof_inputs
pbeza May 31, 2024
8b83be4
Rename `attestation` to `proof`
pbeza May 31, 2024
dc24fb5
Fix test_proof_request_serialization unit test
pbeza May 31, 2024
143abaf
Add tee_attestations SQL table
pbeza Jun 3, 2024
a445762
Add valid_until column to the attestations table
pbeza Jun 4, 2024
0bfe0c1
Get rid of mime dependency
pbeza Jun 5, 2024
d07b649
Remove println! from the production code
pbeza Jun 5, 2024
b893618
Remove println! from tests
pbeza Jun 5, 2024
0d21e6e
Propagate errors instead of using unwrap()
pbeza Jun 5, 2024
1277ae1
Replace ObjectSoreFactory with MockObjectStore after rebase
pbeza Jun 5, 2024
3e8013c
Remove unused dependency
pbeza Jun 5, 2024
3a5aef6
Use workspace dependencies only
pbeza Jun 6, 2024
d2dbd95
Don't use deprecated hyper::body::to_bytes
pbeza Jun 6, 2024
5bbba0c
Remove valid_until column
pbeza Jun 6, 2024
7b092ac
Refactor TEE endpoints and make them optional
pbeza Jun 7, 2024
c753a46
Add a comment to clarify the role of the L1BatchTeeProofForL1 struct
pbeza Jun 10, 2024
76af19d
Remove redundant dependency (vm_utils)
pbeza Jun 10, 2024
9a22bb3
Remove redundant dependency (zksync_utils)
pbeza Jun 10, 2024
10d0c8a
Don't create TEE objects unless TEE support is enabled
pbeza Jun 10, 2024
46d0631
Update Cargo.lock
pbeza Jun 10, 2024
9de0bec
Create tee_proof_generation_job once the verifier input is produced
pbeza Jun 10, 2024
7d73747
Rework error handling as suggested by Igor
pbeza Jun 10, 2024
c48b07d
`zk fmt` fixes
pbeza Jun 11, 2024
6c1c448
Add missing config option to /etc/env/base/proof_data_handler.toml
pbeza Jun 11, 2024
eb145f4
Remove 'skip proof generation' feature from the TEE prover
pbeza Jun 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 15 additions & 16 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ google-cloud-storage = "0.15.0"
governor = "0.4.2"
hex = "0.4"
http = "0.2.9"
hyper = "0.14.29"
iai = "0.1"
insta = "1.29.0"
itertools = "0.10"
Expand Down
8 changes: 8 additions & 0 deletions checks-config/era.dic
Original file line number Diff line number Diff line change
Expand Up @@ -973,3 +973,11 @@ uncached
untrimmed
UNNEST
semver
TeeRequestProcessor
l1_batch_number
RequestProcessorError
map_err
proof_inputs
submit_proofs
ready_to_be_proven
privkey
1 change: 1 addition & 0 deletions core/lib/config/src/configs/proof_data_handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use serde::Deserialize;
pub struct ProofDataHandlerConfig {
pub http_port: u16,
pub proof_generation_timeout_in_secs: u16,
pub tee_support: bool,
}

impl ProofDataHandlerConfig {
Expand Down
1 change: 1 addition & 0 deletions core/lib/config/src/testonly.rs
Original file line number Diff line number Diff line change
Expand Up @@ -626,6 +626,7 @@ impl Distribution<configs::ProofDataHandlerConfig> for EncodeDist {
configs::ProofDataHandlerConfig {
http_port: self.sample(rng),
proof_generation_timeout_in_secs: self.sample(rng),
tee_support: self.sample(rng),
}
}
}
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
DROP TABLE IF EXISTS tee_attestations;
DROP TABLE IF EXISTS tee_proof_generation_details;

DROP INDEX IF EXISTS idx_tee_proof_generation_details_status_prover_taken_at;
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
CREATE TABLE IF NOT EXISTS tee_attestations
(
pubkey BYTEA PRIMARY KEY,
attestation BYTEA
);

CREATE TABLE IF NOT EXISTS tee_proof_generation_details
(
l1_batch_number BIGINT PRIMARY KEY REFERENCES tee_verifier_input_producer_jobs (l1_batch_number) ON DELETE CASCADE,
status TEXT NOT NULL,
signature BYTEA,
pubkey BYTEA REFERENCES tee_attestations (pubkey) ON DELETE SET NULL,
proof BYTEA,
tee_type TEXT,
created_at TIMESTAMP NOT NULL,
updated_at TIMESTAMP NOT NULL,
prover_taken_at TIMESTAMP
);

CREATE INDEX IF NOT EXISTS idx_tee_proof_generation_details_status_prover_taken_at
ON tee_proof_generation_details (prover_taken_at)
WHERE status = 'picked_by_prover';
9 changes: 8 additions & 1 deletion core/lib/dal/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ use crate::{
snapshot_recovery_dal::SnapshotRecoveryDal, snapshots_creator_dal::SnapshotsCreatorDal,
snapshots_dal::SnapshotsDal, storage_logs_dal::StorageLogsDal,
storage_logs_dedup_dal::StorageLogsDedupDal, storage_web3_dal::StorageWeb3Dal,
sync_dal::SyncDal, system_dal::SystemDal,
sync_dal::SyncDal, system_dal::SystemDal, tee_proof_generation_dal::TeeProofGenerationDal,
tee_verifier_input_producer_dal::TeeVerifierInputProducerDal, tokens_dal::TokensDal,
tokens_web3_dal::TokensWeb3Dal, transactions_dal::TransactionsDal,
transactions_web3_dal::TransactionsWeb3Dal, vm_runner_dal::VmRunnerDal,
Expand Down Expand Up @@ -50,6 +50,7 @@ pub mod storage_logs_dedup_dal;
pub mod storage_web3_dal;
pub mod sync_dal;
pub mod system_dal;
pub mod tee_proof_generation_dal;
pub mod tee_verifier_input_producer_dal;
pub mod tokens_dal;
pub mod tokens_web3_dal;
Expand Down Expand Up @@ -111,6 +112,8 @@ where

fn proof_generation_dal(&mut self) -> ProofGenerationDal<'_, 'a>;

fn tee_proof_generation_dal(&mut self) -> TeeProofGenerationDal<'_, 'a>;

fn system_dal(&mut self) -> SystemDal<'_, 'a>;

fn snapshots_dal(&mut self) -> SnapshotsDal<'_, 'a>;
Expand Down Expand Up @@ -213,6 +216,10 @@ impl<'a> CoreDal<'a> for Connection<'a, Core> {
ProofGenerationDal { storage: self }
}

fn tee_proof_generation_dal(&mut self) -> TeeProofGenerationDal<'_, 'a> {
TeeProofGenerationDal { storage: self }
}

fn system_dal(&mut self) -> SystemDal<'_, 'a> {
SystemDal { storage: self }
}
Expand Down
Loading
Loading