Periodic L1 synchronization #96

koxu1996 · 2024-05-09T10:37:36Z

This PR adds periodic L1 synchronization 🔃, based on casper-event-toolkit.

Details

The core component is EventManager, which allows tracking events from a specified smart contract. The process begins with the setup of CES-related data - schemas, toolkit fetcher, and the number of events - using initialize(). Subsequently, process_new_events() checks for new events based on its internal counter, next_event_id.

L1SyncService operates the event manager as a Tokio task and provides a safe API for shared access within an Axum server.

Future work

Once the deposit contract is merged into main, we will be able to parse deposit data from events - currently, this data is mocked - and push it to the batch manager (trie).
Right now synchronization is called periodically - it will be replaced with SSE trigger in the future.

Demo 🎥

l1-sync-demo5-cropped.mp4

If you want to reproduce it locally, please see the next section.

Test environment

I used dockerized NCTL with bunch of shell scripts to run this demo - all available in this gist.

1. Cleanup old resources and compile example smart contract

$ ./cleanup.sh
$ ./prepare.sh

2. Run NCTL and deploy WASM

$ ./setup.sh
> Running NCTL container.
9e4fd0c153e63765ea13b565c70e2d2986ad929a2e74e74cb5e37938e3fe1edf

> Copying user keys.
Successfully copied 37.4kB to /tmp/casper-users

> Deploying session code.
- hash: f9531e06d8f533aa033106b40030ff6d4a9f6b63dc542ff3177d6d29412313ee

> Waiting for execution.
- attempt: 1/10
- attempt: 2/10
> Getting state root hash.
 - d196121dc9613867e228fb807c798fee4e9d1d7d84a2ba6e1a9067494a63c231

> Getting account hash.
 - c6ec39ce20e3bda02cd061e5c91698b0be8a5d5f9abf2b95b8a2a639ae88d060

> Getting contract hash.
 - 9397c7dbfe9ffd60fd03774d8b570bcb3a3a482c5eb3202c97779cb5c0dac008

At this point you have contract hash that is ready to be used in the next steps.

3. Initialize Casper Event Standard

$ ./call-init-entrypoint.sh 9397c7dbfe9ffd60fd03774d8b570bcb3a3a482c5eb3202c97779cb5c0dac008
> Calling 'init' entrypoint.
 - 24c05e37de1c453dd85ef9d93384583c3bbc7106dc916b219e02335cfd3a78ff

> Waiting for execution.
- attempt: 1/20
- attempt: 2/20
- attempt: 3/20
- attempt: 4/20

*-------*
| DONE! |
*-------*

4. Call deposit/batch entrypoint to emit test events

./call-batch-entrypoint.sh 9397c7dbfe9ffd60fd03774d8b570bcb3a3a482c5eb3202c97779cb5c0dac008
> Calling 'batch' entrypoint.
 - 63342404ed21eab1d3a29703a76f81834a0e8bb3bdf3ad1caa53357bf2bd4c38

> Waiting for execution.
- attempt: 1/20
- attempt: 2/20
- attempt: 3/20
- attempt: 4/20
- attempt: 5/20

*-------*
| DONE! |
*-------*

Additional changes for demo

In order to record nice and quick demo, I updated subscribed to include relevant logs:

diff --git a/kairos-server/src/main.rs b/kairos-server/src/main.rs
index 107e421..7b8f1ce 100644
--- a/kairos-server/src/main.rs
+++ b/kairos-server/src/main.rs
@@ -5,6 +5,11 @@ use kairos_server::config::ServerConfig;
 async fn main() {
     let subscriber = tracing_subscriber::fmt()
         .with_max_level(tracing::Level::INFO)
+        .with_env_filter(
+            "kairos_server::l1_sync=debug,kairos_server=warn"
+                .parse::<tracing_subscriber::EnvFilter>()
+                .unwrap(),
+        )
         .finish();

     tracing::subscriber::set_global_default(subscriber).expect("Failed to set subscriber");

I also reduced synchronization interval from 30 to 10 seconds:

diff --git a/kairos-server/src/l1_sync/interval_trigger.rs b/kairos-server/src/l1_sync/interval_trigger.rs
index 419f13c..08acc65 100644
--- a/kairos-server/src/l1_sync/interval_trigger.rs
+++ b/kairos-server/src/l1_sync/interval_trigger.rs
@@ -5,7 +5,7 @@ use std::sync::Arc;
 use super::service::L1SyncService;

 pub async fn run(sync_service: Arc<L1SyncService>) {
-    let mut interval = time::interval(Duration::from_secs(30));
+    let mut interval = time::interval(Duration::from_secs(10));

     loop {
         interval.tick().await;

github-actions · 2024-05-09T11:09:39Z

File	Coverage
All files	`51%`	❌
kairos-crypto/src/implementations/casper.rs	`6%`	❌
kairos-server/src/config.rs	`0%`	❌
kairos-server/src/errors.rs	`12%`	❌
kairos-server/src/lib.rs	`87%`	✅
kairos-server/src/utils.rs	`22%`	❌
kairos-server/src/routes/deposit.rs	`88%`	✅
kairos-server/src/routes/transfer.rs	`90%`	✅
kairos-server/src/state/transactions.rs	`57%`	❌
kairos-server/src/state/trie.rs	`35%`	❌
kairos-server/src/l1_sync/event_manager.rs	`12%`	❌
kairos-server/src/l1_sync/interval_trigger.rs	`0%`	❌
kairos-server/src/l1_sync/service.rs	`56%`	❌
kairos-test-utils/src/cctl.rs	`87%`	✅
kairos-server/src/state/transactions/batch_state.rs	`42%`	❌
kairos-test-utils/src/cctl/parsers.rs	`66%`	✅
kairos-tx/src/asn.rs	`48%`	❌
kairos-tx/src/error.rs	`0%`	❌
kairos-server/tests/transactions.rs	`85%`	✅

Minimum allowed coverage is 60%

Generated by 🐒 cobertura-action against 70c7057

…server' into feature/l1-synchronization

Rom3dius

solid work

kairos-server/src/l1_sync/event_manager.rs

Avi-D-coder · 2024-05-14T14:52:24Z

kairos-server/src/lib.rs

+            .initialize(config.casper_rpc.to_string(), config.casper_contract_hash)
+            .await
+        {
+            panic!("Event manager failed to initialize: {}", e);


Let's propagate this error up.

There is no "up" this is basically before we start the server

Also simply do a map_err

map_err introduced in 326b147.

Avi-D-coder · 2024-05-14T15:03:16Z

kairos-server/src/l1_sync/service.rs

+        }
+        SyncCommand::TriggerSync(completion_ack) => {
+            em.process_new_events().await?;
+            let _ = completion_ack.send(());


what happens on failure?

Fixed in f437b2e.

kairos-server/src/lib.rs

marijanp · 2024-05-15T17:51:44Z

kairos-server/src/lib.rs

+            .initialize(config.casper_rpc.to_string(), config.casper_contract_hash)
+            .await
+        {
+            panic!("Event manager failed to initialize: {}", e);


There is no "up" this is basically before we start the server

marijanp · 2024-05-15T17:53:42Z

kairos-server/src/lib.rs

+            .initialize(config.casper_rpc.to_string(), config.casper_contract_hash)
+            .await
+        {
+            panic!("Event manager failed to initialize: {}", e);


Also simply do a map_err

marijanp · 2024-05-15T17:56:58Z

kairos-server/src/lib.rs

+        {
+            panic!("Event manager failed to initialize: {}", e);
+        }
+        l1_sync::interval_trigger::run(l1_sync_service).await;


Why are there 3 steps to run this service? Just make run take the server config and do all of it in run or introduce three parameters i.e. batch_state, contract_hash and the casper_rpc url.

Service must be created and initialized, then periodic trigger spawned - all that happens in run_l1_sync().

Btw. initialization is done directly now - 1b6cae2.

Initialization steps removed in 50a3654 and 0d4fb13.

marijanp · 2024-05-15T18:06:53Z

kairos-server/src/l1_sync/service.rs

+        match handle_command(command, event_manager.clone()).await {
+            Ok(()) => {}
+            Err(L1SyncError::UnexpectedError(e)) => panic!("Unrecoverable error: {}", e),
+            Err(e) => tracing::error!("Transient error: {}", e),
+        }
+    }


can be replaced with map_err and then just match on the error type.

Done in 1fac715.

marijanp · 2024-05-15T18:10:15Z

kairos-server/src/l1_sync/interval_trigger.rs

+        let result = sync_service.trigger_sync().await;
+
+        if let Err(e) = result {
+            tracing::error!("Unable to trigger sync: {}", e);
+        }


Nit: use map_err

Done in be86df8.

marijanp · 2024-06-10T09:29:03Z

kairos-server/src/l1_sync/event_manager.rs

+pub struct EventManager {
+    next_event_id: u32,
+    fetcher: Option<Fetcher>,
+    schemas: Option<Schemas>,
+    server_state: Arc<ServerStateInner>,
+}
+
+impl EventManager {
+    pub fn new(server_state: Arc<ServerStateInner>) -> Self {
+        EventManager {
+            next_event_id: 0,
+            fetcher: None,
+            schemas: None,
+            server_state,
+        }
+    }


I don't see where it makes sense to model the fetcher and schemas attributes as optionals and introduce a new type that basically sets up the user for using the api in a wrokng way, and moreover forces a user to have a two-step setup process ending up with these two possible use scenarios:
(Notice that the case fetcher == Some && schema == None and fetcher == None && schema Some is not even possible)

// correct let event_manager = EventManager::new(); // fetcher == None && schema == None event_manager.initialize(); // fetcher == Some(..) && schema == Some(..) event_manager.process_new_events(); // wrong usage, but possible without any compiler warning let event_manager = EventManager::new(); // fetcher == None && schema == None event_manager.process_new_events(); // This will fail and I think I should not even be able to call this and a strong indicator is a missing fetcher and schema, which can both be used to prevent an eventuall call // better: process_new_events is now just a function in the server crate and can belong semantically to the server // the ServerState contains all the information like the `rpc_url`, `contract_hash`, `server_state`, even the `next_event_id` pub fn process_new_events(server_state: &ServerState, callback: FnOnce -> Result) -> Result { // initialize the fethcer. Could fail, hence result return type // initialize the schema. Could fail, hence result return type // call callback which injects any behavior a user might want to use which could potentially fail, hence Result return type of the callback }

The latter being a single function call, which deals with all the details a user does not care about. The user only cares about what should happen when new events are observed, reducing the amount of knowledge to just implement the callback logic.

I removed partial construction pattern, so fetcher and schemas are no longer optional: 50a3654.

marijanp · 2024-06-11T10:39:02Z

kairos-server/src/l1_sync/event_manager.rs

+        rpc_url: &str,
+        contract_hash: &str,


We can get the rpc_url from the server state. i.e. server_state.server_config.casper_rpc same goes for the contract hash.

Fixed in 736a540 and 3b52fb1.

kairos-server/src/lib.rs

marijanp · 2024-06-11T10:46:33Z

kairos-server/src/lib.rs

@@ -38,6 +40,35 @@ pub fn app_router(state: ServerState) -> Router {
        .with_state(state)
 }

+pub async fn run_l1_sync(config: ServerConfig, server_state: Arc<ServerStateInner>) {


Don't take ownership here.

config: ServerConfig removed in 8c5cc97.

marijanp · 2024-06-11T10:55:02Z

kairos-server/src/l1_sync/service.rs

+        rpc_url: String,
+        contract_hash: String,


Same goes here as commented for the EventManager both these are in the server_state.server_config

Fixed in 736a540 and 3b52fb1.

marijanp · 2024-06-11T11:07:15Z

kairos-server/src/l1_sync/event_manager.rs

+                .enqueue_transaction(txn)
+                .await
+                .map_err(|e| L1SyncError::UnexpectedError(format!("unable to batch tx: {}", e)))?;
+            self.next_event_id = i + 1;


This reads odd to me, I would move this outside of the for loop, and set it to fetch_events_count. We are not benefiting from updating that value immediately.

I want to keep EventManager state - next_event_id - in sync with enqueued transactions. ~~In other words if some transaction fails to batch, then manager does not consider event as processed.~~

Update: Counter gets incremented with every successfully processed event.

Avi-D-coder · 2024-06-11T16:24:06Z

kairos-server/src/l1_sync/event_manager.rs

+            // TODO: Parse full transaction data from event, then push it to Data Availability layer.
+
+            // TODO: Once we have ASN transaction, it should be converted and pushed into batch.
+            let recipient: Vec<u8> = "cafebabe".into();


why are we mocking the deposit here?

I was hoping that #88 will get merged, and then it could be used for parsing. ~~Since kairos-tx in the contract is postponed for later discussion, I will implement basic parsing here.~~ Blocked on #121.

#121 got merged, so I introduced Deposit event parsing in ff7b6c6.

However, recipient field is still mocked. It is not currently possible to obtain depositor public key from event, as it contains account hash (unnecessarily wrapped in Key)... This was already solved in kairos-tx for contract PR - validation if explicitly given public key matches with caller account hash - so maybe we should reconsider merging it before demo?

Avi-D-coder

please fix the mocked recipient.

NOTE: We get `amount`, but handling `recipient` is not possible yet.

koxu1996 self-assigned this May 9, 2024

koxu1996 added 6 commits May 10, 2024 14:45

Add casper-event-toolkit dependency.

27f2c82

Add reqwest dependency.

b9f2713

Add thiserror dependency.

fbf99e5

Prototype of state synchronization.

03544d2

Fix deadlock.

a60cbb9

Use amount greater than 0 for mocked tx.

6897e36

koxu1996 force-pushed the feature/l1-synchronization branch from 80a2329 to 6897e36 Compare May 10, 2024 12:45

koxu1996 mentioned this pull request May 13, 2024

Add Casper RPC url to server configuration #100

Merged

koxu1996 added 7 commits May 13, 2024 16:07

Merge remote-tracking branch 'origin/feature/configure-casper-rpc-in-…

413c599

…server' into feature/l1-synchronization

Move L1 sync into separate function.

8806f1b

Skip synchronization if contract hash not configured.

a4b0434

Use scoped ENV for contract hash.

88aa500

Merge branch 'main' into feature/l1-synchronization

70c7057

Fix default contract hash value.

f57b840

Add default contract hash to nix setup.

308577b

koxu1996 marked this pull request as ready for review May 14, 2024 10:39

koxu1996 requested review from Avi-D-coder, marijanp, Rom3dius and jonas089 May 14, 2024 10:39

koxu1996 added the demo Required for our first demo label May 14, 2024

Rom3dius approved these changes May 14, 2024

View reviewed changes

Avi-D-coder requested changes May 14, 2024

View reviewed changes

marijanp requested changes May 15, 2024

View reviewed changes

koxu1996 added 4 commits June 6, 2024 20:02

Bump casper-event-toolkit to v0.1.3.

c2ccfdf

Remove into() that is no longer required.

c09d3fd

Replace unused Result match with map_err.

1fac715

Replace another unused Result with map_err.

be86df8

Use empty contract hash in test utils.

e5f9f8b

koxu1996 force-pushed the feature/l1-synchronization branch from 7a69feb to e5f9f8b Compare June 7, 2024 16:48

Avi-D-coder approved these changes Jun 7, 2024

View reviewed changes

marijanp requested changes Jun 10, 2024

View reviewed changes

koxu1996 mentioned this pull request Jun 10, 2024

L1 state synchronization #75

Open

5 tasks

koxu1996 added 4 commits June 11, 2024 11:07

Initialize EventManager directly in constructor.

50a3654

Remove unusued InitializationError.

dd3bc22

Initialize L1SyncService directly in constructor.

0d4fb13

Merge branch 'main' into feature/l1-synchronization

c89f14d

koxu1996 requested a review from marijanp June 11, 2024 09:19

marijanp reviewed Jun 11, 2024

View reviewed changes

Avi-D-coder reviewed Jun 11, 2024

View reviewed changes

Avi-D-coder requested changes Jun 12, 2024

View reviewed changes

koxu1996 added 3 commits June 12, 2024 10:15

Take RPC url from server config.

736a540

Take contract hash from server config.

3b52fb1

Update default contract hash check.

8c5cc97

koxu1996 force-pushed the feature/l1-synchronization branch from a639bde to 8c5cc97 Compare June 12, 2024 10:26

koxu1996 mentioned this pull request Jun 12, 2024

Factor out contract utils #121

Merged

koxu1996 added 4 commits June 13, 2024 09:32

Merge branch 'main' into feature/l1-synchronization

f26f045

Add contract-utils to server workspace.

5b7934b

Build deposit transaction from parsed event.

ff7b6c6

NOTE: We get `amount`, but handling `recipient` is not possible yet.

Fixes for Nix build.

7c35fd0

koxu1996 requested review from Avi-D-coder and marijanp June 13, 2024 08:33

koxu1996 added 2 commits June 14, 2024 14:58

Merge branch 'main' into feature/l1-synchronization

9f90399

Merge branch 'main' into feature/l1-synchronization

59b91af

Avi-D-coder approved these changes Jun 17, 2024

View reviewed changes

marijanp approved these changes Jun 17, 2024

View reviewed changes

koxu1996 merged commit 0c244c0 into main Jun 17, 2024
4 checks passed

koxu1996 deleted the feature/l1-synchronization branch June 17, 2024 15:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Periodic L1 synchronization #96

Periodic L1 synchronization #96

koxu1996 commented May 9, 2024 •

edited

Loading

github-actions bot commented May 9, 2024 •

edited

Loading

Rom3dius left a comment

Avi-D-coder May 14, 2024

marijanp May 15, 2024

marijanp May 15, 2024

koxu1996 Jun 6, 2024

Avi-D-coder May 14, 2024

koxu1996 Jun 7, 2024

marijanp May 15, 2024

marijanp May 15, 2024

marijanp May 15, 2024

koxu1996 Jun 7, 2024

koxu1996 Jun 11, 2024

marijanp May 15, 2024

koxu1996 Jun 6, 2024

marijanp May 15, 2024

koxu1996 Jun 6, 2024

marijanp Jun 10, 2024

koxu1996 Jun 11, 2024

marijanp Jun 11, 2024

koxu1996 Jun 12, 2024

marijanp Jun 11, 2024

koxu1996 Jun 12, 2024 •

edited

Loading

marijanp Jun 11, 2024

koxu1996 Jun 12, 2024

marijanp Jun 11, 2024

koxu1996 Jun 12, 2024 •

edited

Loading

Avi-D-coder Jun 11, 2024

koxu1996 Jun 12, 2024 •

edited

Loading

koxu1996 Jun 13, 2024

Avi-D-coder left a comment

Periodic L1 synchronization #96

Periodic L1 synchronization #96

Conversation

koxu1996 commented May 9, 2024 • edited Loading

Details

Future work

Demo 🎥

Test environment

1. Cleanup old resources and compile example smart contract

2. Run NCTL and deploy WASM

3. Initialize Casper Event Standard

4. Call deposit/batch entrypoint to emit test events

Additional changes for demo

github-actions bot commented May 9, 2024 • edited Loading

Rom3dius left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koxu1996 Jun 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koxu1996 Jun 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koxu1996 Jun 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Avi-D-coder left a comment

Choose a reason for hiding this comment

koxu1996 commented May 9, 2024 •

edited

Loading

github-actions bot commented May 9, 2024 •

edited

Loading

koxu1996 Jun 12, 2024 •

edited

Loading

koxu1996 Jun 12, 2024 •

edited

Loading

koxu1996 Jun 12, 2024 •

edited

Loading