Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a test suite for testing consecutive close/contest transactions #1385

Merged
merged 29 commits into from
Apr 8, 2024

Conversation

ch1bo
Copy link
Collaborator

@ch1bo ch1bo commented Apr 5, 2024

cabal test hydra-cluster --test-options '-m TxTrace' # passes
git revert 85a9b36eb3a6b2dbdb2bad8aa94f5ffe21a8bdc1
cabal test hydra-cluster --test-options '-m TxTrace' # fails
  • 💚 Adds a new stateful property based test suite in TxSpec, which hooks into Hydra.Chain.Direct.State transaction creation (i.e. close and contest) functions and ensures created transactions are valid and can be observed (by observeHeadTx)
    • The model is currently geared toward close and contest transactions
    • Head id, parameters, parties (alice, bob and carol) and close / contest times are all constant
    • Only correctly signed snapshots of a constant UTxO are used
    • Hence, variables are only the Actor and SnapshotNumber of Close and Contest actions.

  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

@ch1bo ch1bo requested review from a team and abailly-iohk April 5, 2024 08:22
Copy link

github-actions bot commented Apr 5, 2024

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-04-08 07:05:02.432395484 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial bccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611 4110
νCommit 56b0f0b597150e619c76bed60683f3b1e42d7bc0685ed951b882bfc5 1975
νHead 86bff95ba20e9d1d1b34899a56d86bbacc9fed999260b27dcc92d128 9351
μHead 88f533cf67cd0fc93d7d9ccf0a8b1d69ffd1208a825efbebbc1d36ba* 4213
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4794 8.89 3.40 0.46
2 4993 10.95 4.20 0.49
3 5196 12.64 4.83 0.52
5 5597 16.64 6.37 0.58
10 6605 26.33 10.09 0.73
48 14245 99.96 38.36 1.86

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 594 10.24 4.04 0.29
2 778 13.88 5.64 0.34
3 969 17.66 7.29 0.39
5 1341 25.66 10.74 0.49
10 2279 48.19 20.30 0.79
19 3975 97.83 40.79 1.41

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 544 16.93 6.68 0.36
2 113 658 26.88 10.71 0.47
3 171 764 37.64 15.16 0.60
4 225 874 53.94 21.76 0.78
5 281 984 69.94 28.34 0.96
6 339 1095 86.17 35.12 1.15

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 599 8.29 4.59 0.28
2 688 8.55 5.31 0.29
3 983 10.43 7.37 0.33
5 1178 11.44 9.14 0.36
10 2071 16.84 15.92 0.49
50 8291 51.99 64.48 1.40

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 646 8.65 4.74 0.28
2 757 9.42 5.82 0.30
3 914 10.15 6.91 0.32
5 1220 11.84 9.31 0.37
10 1966 16.25 15.38 0.48
50 8110 53.13 64.40 1.40

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4699 17.09 7.40 0.55
2 4766 24.86 10.68 0.64
3 4899 41.35 18.08 0.83
4 5132 59.09 26.04 1.04
5 5236 78.25 34.43 1.26
6 5153 88.73 38.75 1.37

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4627 8.06 3.37 0.44
5 1 57 4661 9.00 3.99 0.46
5 5 284 4797 13.40 6.75 0.52
5 10 567 4964 19.79 10.58 0.60
5 20 1139 5307 31.11 17.62 0.76
5 30 1706 5645 42.43 24.67 0.92
5 40 2276 5985 53.96 31.81 1.08
5 50 2843 6323 65.80 39.07 1.24
5 79 4500 7313 99.00 59.68 1.70

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-04-08 07:07:07.132510371 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 21.424022681
P99 112.64878053000001ms
P95 30.683390099999976ms
P50 18.713602ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 3.773575788
P99 5.182355469999989ms
P95 4.48480115ms
P50 3.6629695ms
Number of Invalid txs 0

ch1bo added 14 commits April 8, 2024 09:01
Also adds cardano payment key fixtures.
This is using the Hydra.Chain.Direct.State.close as a first draft, but
direct usage of Hydra.Chain.Direct.Tx.closeTx should be possible as
well.

Let's see what is better further down the road.
The fact that all snapshots will be arbitrarily generated and correctly
signed, makes it natural to push that out into the perform stage and
only keep the snapshot number in the model.
Hemmingway bridge for next time..
Surprisingly this seems to be also wrong without reverting our latest fix!
While both approaches would be fine (and we had it different in the
past), this matches the implementation and specification.
Copy link

github-actions bot commented Apr 8, 2024

Test Results

427 tests  +2   419 ✅ +2   14m 38s ⏱️ +19s
139 suites +1     8 💤 ±0 
  5 files   ±0     0 ❌ ±0 

Results for commit c7171e2. ± Comparison against base commit 4e1e1ed.

@ch1bo ch1bo merged commit 0fbcffe into master Apr 8, 2024
21 checks passed
@ch1bo ch1bo deleted the tx-trace-spec branch April 8, 2024 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Write a test exposing observed contesters bug
2 participants