Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smoke test with mithril #1252

Merged
merged 8 commits into from
Jan 17, 2024
Merged

Smoke test with mithril #1252

merged 8 commits into from
Jan 17, 2024

Conversation

ch1bo
Copy link
Collaborator

@ch1bo ch1bo commented Jan 15, 2024

  • Adds --use-mithril to hydra-cluster which makes it use mithril-client to download the cardano network state before starting cardano-node.

  • Adds mithril-client to our development and CI shells.

  • Adds integration tests to ensure that mithril-client is invoked correctly on all known networks.

A mithril-bootstrapped smoke test run on preview is here: https://github.com/input-output-hk/hydra/actions/runs/7540117340/job/20524247050

Not in scope / moved to red bin:


  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

@ch1bo ch1bo requested a review from a team January 15, 2024 17:57
@ch1bo ch1bo self-assigned this Jan 15, 2024
Copy link

github-actions bot commented Jan 15, 2024

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-01-17 13:25:03.373182896 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial 985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb 4072
νCommit 7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd 2043
νHead 7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772 8816
μHead 8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463* 3851
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4376 10.63 4.11 0.46
2 4575 12.85 4.95 0.49
3 4777 14.73 5.65 0.52
5 5179 19.33 7.40 0.59
10 6184 30.20 11.51 0.75
41 12418 99.00 37.59 1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 531 11.37 4.44 0.30
2 724 15.04 6.07 0.35
3 911 18.85 7.75 0.40
5 1284 26.90 11.27 0.51
10 2224 49.55 20.97 0.80
19 3907 99.43 41.75 1.43

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 480 21.12 8.27 0.40
2 114 590 34.12 13.43 0.55
3 170 700 46.10 18.35 0.69
4 227 810 62.78 25.08 0.87
5 282 920 77.27 31.12 1.04
6 339 1031 98.06 39.58 1.27

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 489 15.16 6.44 0.34
2 686 18.04 8.97 0.39
3 927 20.13 10.92 0.43
5 889 20.68 9.40 0.42
10 2167 32.41 22.10 0.67
50 9020 99.70 84.54 1.96

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 575 20.22 8.96 0.40
2 758 22.21 10.66 0.44
3 926 24.15 12.34 0.48
5 1317 28.11 15.77 0.55
10 2151 37.51 23.76 0.72
44 7892 99.32 78.02 1.86

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4307 18.85 8.11 0.55
2 4493 32.05 13.97 0.71
3 4667 47.75 20.95 0.89
4 4795 66.40 29.20 1.11
5 4946 80.16 35.20 1.27

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4205 8.26 3.46 0.43
5 1 57 4239 9.07 4.04 0.44
5 5 283 4374 14.40 7.27 0.51
5 10 569 4544 20.13 10.90 0.59
5 20 1136 4881 32.64 18.62 0.76
5 30 1706 5224 45.15 26.34 0.93
5 40 2277 5564 57.46 33.97 1.10
5 50 2848 5905 69.35 41.43 1.27
5 74 4206 6712 99.22 59.90 1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-01-17 13:27:45.038161291 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 24.363659856
P99 117.83298538000003ms
P95 33.556506899999995ms
P50 21.4216755ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 4.791108343
P99 7.830356819999968ms
P95 6.164305149999999ms
P50 4.6173965ms
Number of Invalid txs 0

@ch1bo ch1bo force-pushed the smoke-test-with-mithril branch 2 times, most recently from b6a177e to 84e22cb Compare January 16, 2024 09:35
Calls mithril-client to download and verify cardano-node db snapshots.
This is particularly useful for running scenarios on public (test)
networks.
This can be used to speed up smoke tests by downloading the latest
snapshot and have the cardano-node synchronize less.
This will also delete the cardano-node db/ on the runner to download it
fresh from mithril.
, ["--download-dir", dir]
]
where
genesisKeyURLForNetwork = \case
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we can query these url's since I think I saw them change once.

}
deriving stock (Show)

data PublishOrReuse = Publish | Reuse TxId
deriving stock (Show)

-- TODO: Provide an option to use mithril aggregated snapshots to bootstrap the testnet
data UseMithril = NotUseMithril | UseMithril
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe NoMithril or DontUseMithril?

@ch1bo ch1bo added this to the 0.15.0 milestone Jan 16, 2024
This keeps the log output of tests clean and consistent with outputs of
cardano-node and hydra-node processes.
EOF errors are expected when reading all stdout (when the process exits)
and we can safely ignore them.
This makes the test quicker and asserts correct invocation.
Copy link

Test Results

408 tests  +3   401 ✅ +3   15m 55s ⏱️ +15s
136 suites +1     7 💤 ±0 
  5 files   ±0     0 ❌ ±0 

Results for commit 40d948c. ± Comparison against base commit 7a19916.

@ch1bo ch1bo merged commit 2a68a74 into master Jan 17, 2024
21 checks passed
@ch1bo ch1bo deleted the smoke-test-with-mithril branch January 17, 2024 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants