Fix logs schema #1244

ch1bo · 2024-01-10T10:28:44Z

🐞 Fixes several broken JSON schemas from stateless observation changes (have not been released, hence no CHANGELOG update here)

🐞 Fixes and improves output of prop_validateJSONSchema such that it allowed me to find those schema mismatches each with 1-2 runs of --match "/Hydra.Logging/Validates logs.yaml schema/". The 5 successes and 100 sized test vectors seem to be a good trade-off.

CHANGELOG updated or not needed
Documentation updated or not needed
Haddocks updated or not needed
No new TODOs introduced or explained herafter

github-actions · 2024-01-10T10:37:35Z

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-01-11 09:55:49.243020446 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb	4072
νCommit	7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd	2043
νHead	7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772	8816
μHead	8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463*	3851

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4372	10.46	4.04	0.46
2	4576	12.57	4.83	0.49
3	4781	14.94	5.74	0.52
5	5177	19.22	7.35	0.59
10	6189	30.30	11.55	0.75
41	12417	99.02	37.59	1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	534	11.37	4.44	0.30
2	718	15.04	6.07	0.35
3	906	18.85	7.75	0.40
5	1282	26.90	11.27	0.51
10	2225	49.55	20.97	0.80
19	3909	99.43	41.75	1.43

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	484	21.78	8.51	0.41
2	113	590	34.21	13.47	0.55
3	170	700	45.75	18.21	0.68
4	226	810	63.78	25.44	0.88
5	283	920	82.73	33.14	1.10
6	339	1031	93.58	37.96	1.23

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	567	16.64	7.68	0.37
2	739	18.45	9.31	0.40
3	917	20.27	10.96	0.43
5	1208	23.18	13.77	0.49
10	2123	32.32	22.04	0.66
50	8685	97.47	82.72	1.92

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	564	20.18	8.94	0.40
2	774	22.30	10.71	0.44
3	927	24.23	12.38	0.48
5	1233	27.58	15.36	0.54
10	2101	37.58	23.82	0.72
43	7537	97.88	76.11	1.82

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4339	19.02	8.19	0.55
2	4506	31.87	13.90	0.71
3	4545	41.85	18.14	0.82
4	4765	59.73	26.10	1.03
5	5005	88.01	38.80	1.36

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4206	8.05	3.37	0.42
5	1	57	4240	9.28	4.13	0.44
5	5	285	4376	14.40	7.27	0.51
5	10	567	4542	20.55	11.08	0.60
5	20	1138	4883	32.85	18.71	0.77
5	30	1708	5225	44.94	26.25	0.93
5	40	2276	5563	57.25	33.88	1.10
5	50	2845	5902	69.35	41.43	1.27
5	74	4215	6721	99.13	59.87	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-01-11 09:58:40.037315429 UTC

Baseline Scenario

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	23.345994873
P99	113.38022407000005ms
P95	32.9747366ms
P50	20.7097945ms
Number of Invalid txs	0

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.053925358
P99	7.303820499999983ms
P95	4.8601696ms
P50	3.9001314999999996ms
Number of Invalid txs	0

github-actions · 2024-01-10T10:43:25Z

Test Results

401 tests ±0 394 ✅ ±0 16m 26s ⏱️ +18s
134 suites ±0 7 💤 ±0
5 files ±0 0 ❌ ±0

Results for commit 516f761. ± Comparison against base commit ff2182a.

♻️ This comment has been updated with latest results.

ghost

You mention 5 successes and lists of size 100 but it seems this comment is not true anymore? It's one success with a list of size 1000.

ghost · 2024-01-11T09:44:16Z

hydra-node/json-schemas/api.yaml

@@ -1555,7 +1555,7 @@ components:
            tag:
              type: string
              enum: ["InvalidHeadId"]
-            headSeed:
+            headId:


Good catch!

Using mapSize was preventing shrinking and produced very big counter examples. Adding -v allows to debug non-matching 'oneOf' schemas better.

This also bumps up the number of generated values to only invoke the external program once (if it passes). Shrinking will pick apart the big list of values again.

ch1bo added this to the 0.15.0 milestone Jan 10, 2024

ch1bo requested a review from a team January 10, 2024 10:28

ch1bo force-pushed the fix-logs-schema branch 3 times, most recently from a0adf26 to 84251ae Compare January 10, 2024 14:18

ghost approved these changes Jan 11, 2024

View reviewed changes

hydra-node/json-schemas/api.yaml

@@ -1555,7 +1555,7 @@ components:

tag:

type: string

enum: ["InvalidHeadId"]

headSeed:

headId:

Copy link

ghost Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

ch1bo added 3 commits January 11, 2024 10:53

Fix shrinking in prop_validateJSONSchema and enable verbos output

26420d6

Using mapSize was preventing shrinking and produced very big counter examples. Adding -v allows to debug non-matching 'oneOf' schemas better.

Fix several schemas in api.yaml and logs.yaml

66c770e

Make prop_validateJSONSchema run with a fixed maxSuccess=1

516f761

This also bumps up the number of generated values to only invoke the external program once (if it passes). Shrinking will pick apart the big list of values again.

ghost force-pushed the fix-logs-schema branch from 84251ae to 516f761 Compare January 11, 2024 09:53

ch1bo merged commit b0b8563 into master Jan 11, 2024
21 checks passed

ch1bo deleted the fix-logs-schema branch January 11, 2024 10:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix logs schema #1244

Fix logs schema #1244

ch1bo commented Jan 10, 2024

github-actions bot commented Jan 10, 2024 •

edited

Loading

github-actions bot commented Jan 10, 2024 •

edited

Loading

ghost left a comment

ghost Jan 11, 2024

Fix logs schema #1244

Fix logs schema #1244

Conversation

ch1bo commented Jan 10, 2024

github-actions bot commented Jan 10, 2024 • edited Loading

Transactions Costs

Script summary

Cost of Init Transaction

Cost of Commit Transaction

Cost of CollectCom Transaction

Cost of Close Transaction

Cost of Contest Transaction

Cost of Abort Transaction

Cost of FanOut Transaction

End-To-End Benchmark Results

Baseline Scenario

Baseline Scenario

github-actions bot commented Jan 10, 2024 • edited Loading

Test Results

ghost left a comment

Choose a reason for hiding this comment

ghost Jan 11, 2024

Choose a reason for hiding this comment

github-actions bot commented Jan 10, 2024 •

edited

Loading

github-actions bot commented Jan 10, 2024 •

edited

Loading