Incorrect arguments passed for running bench

[abailly-iohk]

fix #1164

[github-actions]

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-11-16 08:30:29.190329318 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	57dd0a3aba3f4477afed74e7d12b6e86d7abad22ee51612ff81b4194	4059
νCommit	171a1e6bdbc8aa96d957a65b3f505517386af06ba265e3f784741f67	2050
νHead	e89b0c4a6155bac2434d1e500bd49c155b2b56744ccf5a0efa72a82e	9092
μHead	92368af5649999c05211407d9a837cdd10ecf98c5829b385e89a3648*	4062

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4594	10.92	4.33	0.47
2	4798	13.30	5.25	0.51
3	5004	15.19	5.96	0.54
5	5418	19.67	7.67	0.60
10	6444	30.97	11.99	0.77
41	12794	99.61	38.18	1.79

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	536	12.23	4.81	0.31
2	723	15.93	6.48	0.36
3	908	19.77	8.20	0.41
5	1285	27.87	11.80	0.52
10	2221	50.58	21.69	0.81
18	3721	94.20	40.16	1.36

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	483	22.88	9.12	0.42
2	114	593	34.05	13.74	0.55
3	170	703	48.26	19.60	0.71
4	226	813	62.92	25.73	0.88
5	281	923	83.94	34.32	1.12

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	540	17.80	8.36	0.38
2	600	17.68	7.89	0.38
3	932	21.18	11.59	0.45
5	1296	24.70	14.85	0.51
10	2199	32.92	22.63	0.68
50	9176	97.83	84.50	1.95

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	565	21.29	9.68	0.42
2	721	22.79	11.05	0.45
3	957	25.07	13.05	0.49
5	1296	28.41	16.08	0.55
10	2207	38.20	24.61	0.73
44	7769	96.48	77.28	1.83

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4531	20.14	8.76	0.58
2	4712	33.68	14.85	0.74
3	4864	50.20	22.25	0.93
4	4965	63.10	27.84	1.08
5	5039	78.26	34.39	1.25

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4417	8.98	3.78	0.44
5	1	57	4453	10.51	4.67	0.46
5	5	285	4597	15.70	7.84	0.54
5	10	570	4777	22.18	11.81	0.62
5	20	1136	5133	34.92	19.66	0.80
5	30	1708	5497	47.77	27.55	0.98
5	40	2278	5857	61.12	35.65	1.16
5	50	2844	6213	73.88	43.51	1.33
5	69	3931	6900	98.80	58.73	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2023-11-16 08:22:39.912383909 UTC

3-nodes Scenario

A rather typical setup, with 3 nodes forming a Hydra head.

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	28.581770044
P99	264.29460303ms
P95	37.98876714999991ms
P50	20.137029ms
Number of Invalid txs	0

Baseline Scenario

This scenario represents a minimal case and as such is a good baseline against which to assess the overhead introduced by more complex setups. There is a single hydra-node d with a single client submitting single input and single output transactions with a constant UTxO set of 1.

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	4.508013236
P99	7.051921419999986ms
P95	5.3609962ms
P50	4.236398ms
Number of Invalid txs	0

[github-actions]

Test Results

367 tests  ±0   362 ✔️ ±0   20m 35s ⏱️ +31s
124 suites ±0       5 💤 ±0 
    5 files   ±0       0 ❌ ±0 

Results for commit 0bf0f3f. ± Comparison against base commit 092f445.

♻️ This comment has been updated with latest results.

[ch1bo]

Not sure I understand the LoggingSpec change. Should we do that also to other invocations of prop_validateJSONSchema if it's better?

[abailly-iohk]

Yes, I think the use of conjoin is not generally helpful when the generators are independent anyhow so I would separate the other uses of these spec checker tests, but I agree this is confusing and unrelated to this PR.