Switch bench-e2e to JSON datasets again

[ch1bo]

This keeps the cardano-cli style transaction envelopes for transactions
with essentially base16 encoded tx data, but uses JSON otherwise.

Having JSON encoded datasets allows easier introspection (title, description, number of txs) of
individual parts of a dataset.

Note that this also fixes the broken titles in https://hydra.family/head-protocol/unstable/benchmarks/end-to-end-benchmarks/ as each dataset has a title.

---

• CHANGELOG updated or not needed
• Documentation updated or not needed
• Haddocks updated or not needed
• No new TODOs introduced or explained herafter

[ch1bo]

@v0d1ch Created this to also ensure that #1409 does not mess with our transaction decoding unexpectedly.

[github-actions]

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-05-13 15:59:58.716529766 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	bccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611	4110
νCommit	56b0f0b597150e619c76bed60683f3b1e42d7bc0685ed951b882bfc5	1975
νHead	86bff95ba20e9d1d1b34899a56d86bbacc9fed999260b27dcc92d128	9351
μHead	88f533cf67cd0fc93d7d9ccf0a8b1d69ffd1208a825efbebbc1d36ba*	4213

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4795	8.72	3.33	0.46
2	4996	10.91	4.18	0.49
3	5196	12.79	4.90	0.52
5	5600	16.72	6.41	0.58
10	6604	26.12	10.01	0.72
47	14043	98.21	37.70	1.83

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	559	10.24	4.04	0.29
2	746	13.88	5.64	0.34
3	937	17.66	7.29	0.39
5	1313	25.66	10.74	0.49
10	2249	48.19	20.30	0.78
19	3923	97.83	40.79	1.41

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	544	16.93	6.68	0.36
2	114	654	26.88	10.71	0.47
3	169	764	38.13	15.34	0.60
4	226	878	54.16	21.83	0.78
5	282	984	69.01	28.03	0.95
6	338	1095	82.56	33.83	1.11

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	597	8.30	4.60	0.28
2	734	8.91	5.62	0.30
3	890	9.75	6.75	0.32
5	1215	11.35	9.14	0.36
10	2026	16.80	15.87	0.49
50	7841	50.05	62.59	1.35

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	577	8.24	4.39	0.27
2	746	9.34	5.78	0.30
3	959	10.48	7.21	0.33
5	1251	12.42	9.69	0.38
10	2054	17.37	16.10	0.50
50	8233	53.18	64.66	1.41

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4666	17.09	7.39	0.55
2	4782	27.79	12.11	0.67
3	4917	40.55	17.77	0.82
4	5027	56.51	24.77	1.01
5	5100	67.27	29.30	1.13
6	5293	95.74	42.14	1.46

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4627	8.06	3.37	0.44
5	1	57	4661	9.00	3.99	0.46
5	5	285	4798	13.82	6.93	0.52
5	10	568	4965	18.95	10.22	0.60
5	20	1138	5306	30.69	17.44	0.76
5	30	1704	5644	42.01	24.49	0.91
5	40	2277	5987	53.96	31.81	1.08
5	50	2845	6325	65.29	38.86	1.24
5	79	4498	7310	99.00	59.68	1.70

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-05-13 16:02:50.021147497 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	3.636125948
P99	5.497634109999991ms
P95	4.28879875ms
P50	3.513503ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	20.895142725
P99	63.165024110000125ms
P95	28.330992649999995ms
P50	18.568966000000003ms
Number of Invalid txs	0

[github-actions]

Test Results

427 tests  +1   419 ✅ +1   15m 15s ⏱️ -21s
139 suites +1     8 💤 ±0 
  2 files   ±0     0 ❌ ±0 

Results for commit 101b770. ± Comparison against base commit d8d2f93.