Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLI][E2E] Wait for reconfiguration in staking tests, use latest Python SDK #13389

Merged
merged 1 commit into from
May 24, 2024

Conversation

banool
Copy link
Contributor

@banool banool commented May 22, 2024

Description

Fixing broken staking / vesting tests. More context: https://aptos-org.slack.com/archives/C03N83P7QUC/p1715712296029719.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

See CLI E2E CI.

Testing this locally is proving difficult as usual, I just can't run a localnet from a container on my system for some reason:

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --pull always --detach --name aptos-tools-devnet -p 8080:8080 -p 9101:9101 -p 8081:8081 aptoslabs/tools:devnet aptos node run-local-testnet --with-faucet

Let's see how CI goes. If this doesn't work I'll try test on Windows.

Key Areas to Review

N/A

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@banool banool requested a review from gregnazario as a code owner May 22, 2024 18:44
Copy link

trunk-io bot commented May 22, 2024

⏱️ 19h 43m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-smoke-tests 3h 14m 🟩🟩🟩 (+2 more)
rust-unit-tests 2h 52m 🟩🟩🟩 (+2 more)
windows-build 2h 45m 🟩🟩🟩🟩
forge-e2e-test / forge 1h 25m 🟩🟩🟩🟩 (+1 more)
forge-compat-test / forge 1h 18m 🟩🟩🟩🟩 (+1 more)
rust-targeted-unit-tests 56m 🟩🟩🟩🟩 (+4 more)
rust-images / rust-all 51m 🟩🟩🟩🟩 (+2 more)
rust-lints 40m 🟩🟩🟩🟩 (+4 more)
cli-e2e-tests / run-cli-tests 39m 🟩🟩🟩🟥 (+1 more)
forge-framework-upgrade-test / forge 38m 🟩🟩
run-tests-main-branch 38m 🟩🟩🟩🟩🟩 (+4 more)
rust-move-tests 31m 🟩🟩🟩🟩 (+4 more)
execution-performance / test-target-determinator 30m 🟩🟩🟩🟩🟩 (+4 more)
rust-build-cached-packages 30m 🟩🟩🟩🟩 (+2 more)
check 26m 🟩🟩🟩🟩 (+3 more)
test-target-determinator 22m 🟩🟩🟩🟩 (+2 more)
general-lints 15m 🟩🟩🟩🟩 (+4 more)
rust-unit-coverage 15m 🟥
rust-smoke-coverage 15m 🟥
check-dynamic-deps 14m 🟩🟩🟩🟩🟩 (+6 more)
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose 10m 🟩🟩🟩🟩🟩 (+1 more)
node-api-compatibility-tests / node-api-compatibility-tests 5m 🟩🟩🟩🟩🟩 (+1 more)
semgrep/ci 4m 🟩🟩🟩🟩🟩 (+6 more)
file_change_determinator 2m 🟩🟩🟩🟩🟩 (+4 more)
file_change_determinator 2m 🟩🟩🟩🟩🟩 (+4 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
execution-performance / single-node-performance 1m 🟩🟩🟩🟩🟩 (+4 more)
permission-check 33s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 31s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 28s 🟩🟩🟩🟩🟩 (+6 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+5 more)
permission-check 20s 🟩🟩🟩🟩🟩 (+2 more)
determine-docker-build-metadata 14s 🟩🟩🟩🟩🟩 (+2 more)
upload-to-codecov 14s 🟥🟥

🚨 5 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
forge-framework-upgrade-test / forge 14m 25m -43%
rust-targeted-unit-tests 9m 18m -52%
rust-move-tests 3m 9m -63%
rust-images / rust-all 30s 13m -96%
execution-performance / single-node-performance 9s 20m -99%

settingsfeedbackdocs ⋅ learn more about trunk.io

@banool banool marked this pull request as draft May 22, 2024 18:44
@banool banool added CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-coverage run tests with test coverage instrumentation CICD:run-all-unit-tests Runs all unit tests labels May 22, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@banool
Copy link
Contributor Author

banool commented May 23, 2024

Should be ready for review. Not 100% sure this fixes it, it's hard to tell since it only fails sometimes. But when we go to land it it should run once again.

@banool banool marked this pull request as ready for review May 23, 2024 09:55

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@banool banool enabled auto-merge (squash) May 23, 2024 14:12

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

@perryjrandall perryjrandall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

@banool banool enabled auto-merge (squash) May 23, 2024 17:23

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 29c8bb9150b99b7f6d599b404e5b383d688b655c

two traffics test: inner traffic : committed: 7696.697193896433 txn/s, latency: 5086.508815957657 ms, (p50: 5000 ms, p90: 6600 ms, p99: 10500 ms), latency samples: 3332820
two traffics test : committed: 99.9648001603874 txn/s, latency: 1928.7191011235955 ms, (p50: 1900 ms, p90: 2200 ms, p99: 2600 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.214, avg: 0.206", "QsPosToProposal: max: 0.291, avg: 0.258", "ConsensusProposalToOrdered: max: 0.466, avg: 0.428", "ConsensusOrderedToCommit: max: 0.401, avg: 0.385", "ConsensusProposalToCommit: max: 0.838, avg: 0.812"]
Max round gap was 1 [limit 4] at version 1685168. Max no progress secs was 4.698497 [limit 15] at version 1685168.
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c (PR)
1. Check liveness of validators at old version: 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411
compatibility::simple-validator-upgrade::liveness-check : committed: 6497.591075646208 txn/s, latency: 4789.245384922217 ms, (p50: 4800 ms, p90: 5400 ms, p99: 8300 ms), latency samples: 250700
2. Upgrading first Validator to new version: 29c8bb9150b99b7f6d599b404e5b383d688b655c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3403.9528569721856 txn/s, latency: 9112.785480838756 ms, (p50: 9200 ms, p90: 13900 ms, p99: 14200 ms), latency samples: 138300
3. Upgrading rest of first batch to new version: 29c8bb9150b99b7f6d599b404e5b383d688b655c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3321.043530571822 txn/s, latency: 9326.93037956621 ms, (p50: 9400 ms, p90: 13900 ms, p99: 14400 ms), latency samples: 140160
4. upgrading second batch to new version: 29c8bb9150b99b7f6d599b404e5b383d688b655c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4336.765834785434 txn/s, latency: 7433.30248191569 ms, (p50: 7200 ms, p90: 13200 ms, p99: 17200 ms), latency samples: 160360
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c passed
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c (PR)
Upgrade the nodes to version: 29c8bb9150b99b7f6d599b404e5b383d688b655c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1163.6558711518426 txn/s, submitted: 1165.956495649968 txn/s, failed submission: 2.3006244981254302 txn/s, expired: 2.3006244981254302 txn/s, latency: 2662.29943653618 ms, (p50: 1800 ms, p90: 4800 ms, p99: 11000 ms), latency samples: 101160
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1071.0342313146389 txn/s, submitted: 1074.1704955351797 txn/s, failed submission: 3.1362642205406703 txn/s, expired: 3.1362642205406703 txn/s, latency: 2847.5886739175903 ms, (p50: 2100 ms, p90: 5700 ms, p99: 10800 ms), latency samples: 95620
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 29c8bb9150b99b7f6d599b404e5b383d688b655c passed
Upgrade the remaining nodes to version: 29c8bb9150b99b7f6d599b404e5b383d688b655c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1060.0850587399107 txn/s, submitted: 1062.5698095701127 txn/s, failed submission: 2.48475083020222 txn/s, expired: 2.48475083020222 txn/s, latency: 3181.4930534839123 ms, (p50: 2400 ms, p90: 6000 ms, p99: 12700 ms), latency samples: 93860
Test Ok

@banool banool merged commit 67ed746 into main May 24, 2024
98 of 99 checks passed
@banool banool deleted the banool/cli-tests branch May 24, 2024 22:20
@banool banool mentioned this pull request May 29, 2024
21 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-all-unit-tests Runs all unit tests CICD:run-coverage run tests with test coverage instrumentation CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants