Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[event_v2] migrate the rest files in aptos-framework #11688

Merged
merged 1 commit into from
Apr 8, 2024

Conversation

lightmark
Copy link
Contributor

@lightmark lightmark commented Jan 18, 2024

Description

migrate other files

Test Plan

Copy link

trunk-io bot commented Jan 18, 2024

⏱️ 7h 11m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-unit-tests 3h 9m 🟩🟥🟥🟩🟩 (+1 more)
windows-build 1h 45m 🟩🟩🟩🟩🟩 (+1 more)
rust-lints 49m 🟩🟩🟩🟥🟩 (+1 more)
run-tests-main-branch 31m 🟩🟩🟩🟩🟥 (+2 more)
check 25m 🟩🟩🟩🟩🟩 (+1 more)
general-lints 15m 🟩🟩🟩🟩🟩 (+1 more)
check-dynamic-deps 12m 🟩🟩🟩🟩🟩 (+1 more)
semgrep/ci 2m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 59s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 28s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 18s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 17s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 15s 🟩🟩🟩🟩🟩 (+1 more)

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
run-tests-main-branch 6m 4m +67%
windows-build 15m 20m -25%

settingsfeedbackdocs ⋅ learn more about trunk.io

@lightmark lightmark changed the title [event_v2] migrate 3 files [event_v2] migrate the rest files in aptos-framework Jan 19, 2024
@lightmark lightmark force-pushed the event_v2_migration_2 branch 3 times, most recently from da0e98b to e03a9c4 Compare January 20, 2024 04:49
@lightmark lightmark force-pushed the event_v2_migration_2 branch 2 times, most recently from 549a928 to 22f159f Compare February 28, 2024 23:17
Copy link

codecov bot commented Feb 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.6%. Comparing base (c807593) to head (f8d9432).

❗ Current head f8d9432 differs from pull request most recent head 356731a. Consider uploading reports for the commit 356731a to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##             main   #11688       +/-   ##
===========================================
- Coverage    69.2%    62.6%     -6.7%     
===========================================
  Files        2296      821     -1475     
  Lines      438064   184173   -253891     
===========================================
- Hits       303482   115391   -188091     
+ Misses     134582    68782    -65800     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lightmark lightmark force-pushed the event_v2_migration_2 branch 2 times, most recently from 6087009 to e695fc1 Compare March 28, 2024 14:00
Copy link
Contributor

@davidiw davidiw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be good to test this with an updated indexer processor and verify it is correct before we end up shipping this.

also might be worth adding the disabling of v1 in there as well for that reason.

Comment on lines +193 to +203
let new_block_event_v2 = NewBlock {
hash,
epoch,
round,
height: block_metadata_ref.height,
previous_block_votes_bitvec,
proposer,
failed_proposer_indices,
time_microseconds: timestamp,
};
emit_new_block_event(vm, &mut block_metadata_ref.new_block_events, new_block_event, new_block_event_v2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we're doing this, we might incur a performance impact on each block before we even migrate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more event per block? should not be an issue regarding perf.

Comment on lines 144 to +150
spec create_staking_contract(
staker: &signer,
operator: address,
voter: address,
amount: u64,
commission_percentage: u64,
contract_creation_seed: vector<u8>,
staker: &signer,
operator: address,
voter: address,
amount: u64,
commission_percentage: u64,
contract_creation_seed: vector<u8>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something weird

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intellij move plugin does this automatically and I have no way to disable it...

@lightmark
Copy link
Contributor Author

I will rebase once #11224 lands

@lightmark
Copy link
Contributor Author

@grao1991 do you need to change the new block event tracking after this PR?

@grao1991
Copy link
Contributor

@grao1991 do you need to change the new block event tracking after this PR?

Yes. It's protected by the sharding flag which hasn't been enabled. You go first and I can try to make the change later.

FYI, you need to double check if other things (e.g. consensus) that depending on the new block event works correctly after your change.

@lightmark lightmark force-pushed the event_v2_migration_2 branch 3 times, most recently from 3c0145c to 24f567d Compare April 5, 2024 16:18
@bowenyang007
Copy link
Contributor

I trust you. Didn't carefully review.

@bowenyang007
Copy link
Contributor

When are we removing the old events though?

@lightmark
Copy link
Contributor Author

When are we removing the old events though?

Let's do it after 1-2 release.

add feature flags

use feature_flag to switch
@lightmark lightmark enabled auto-merge (squash) April 8, 2024 14:48
@lightmark lightmark disabled auto-merge April 8, 2024 14:48
@lightmark lightmark enabled auto-merge (rebase) April 8, 2024 14:48

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Apr 8, 2024

✅ Forge suite compat success on aptos-node-v1.10.1 ==> 356731aebc7fff11edbd07106cb30615830acbd1

Compatibility test results for aptos-node-v1.10.1 ==> 356731aebc7fff11edbd07106cb30615830acbd1 (PR)
1. Check liveness of validators at old version: aptos-node-v1.10.1
compatibility::simple-validator-upgrade::liveness-check : committed: 6756 txn/s, latency: 4893 ms, (p50: 4800 ms, p90: 7800 ms, p99: 9300 ms), latency samples: 236480
2. Upgrading first Validator to new version: 356731aebc7fff11edbd07106cb30615830acbd1
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1708 txn/s, latency: 17239 ms, (p50: 19100 ms, p90: 22600 ms, p99: 22800 ms), latency samples: 88860
3. Upgrading rest of first batch to new version: 356731aebc7fff11edbd07106cb30615830acbd1
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1432 txn/s, latency: 20899 ms, (p50: 19100 ms, p90: 30600 ms, p99: 30900 ms), latency samples: 88820
4. upgrading second batch to new version: 356731aebc7fff11edbd07106cb30615830acbd1
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3606 txn/s, latency: 8702 ms, (p50: 9800 ms, p90: 12400 ms, p99: 12600 ms), latency samples: 144240
5. check swarm health
Compatibility test for aptos-node-v1.10.1 ==> 356731aebc7fff11edbd07106cb30615830acbd1 passed
Test Ok

Copy link
Contributor

github-actions bot commented Apr 8, 2024

✅ Forge suite realistic_env_max_load success on 356731aebc7fff11edbd07106cb30615830acbd1

two traffics test: inner traffic : committed: 8114 txn/s, latency: 4831 ms, (p50: 4500 ms, p90: 5700 ms, p99: 11700 ms), latency samples: 3505400
two traffics test : committed: 100 txn/s, latency: 1850 ms, (p50: 1800 ms, p90: 2100 ms, p99: 6100 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.213, avg: 0.204", "QsPosToProposal: max: 0.429, avg: 0.262", "ConsensusProposalToOrdered: max: 0.454, avg: 0.428", "ConsensusOrderedToCommit: max: 0.356, avg: 0.325", "ConsensusProposalToCommit: max: 0.811, avg: 0.752"]
Max round gap was 1 [limit 4] at version 1489611. Max no progress secs was 5.032854 [limit 15] at version 1489611.
Test Ok

@lightmark lightmark merged commit 728bf9b into main Apr 8, 2024
109 of 147 checks passed
@lightmark lightmark deleted the event_v2_migration_2 branch April 8, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants