API Server as event sink

[v0d1ch]

This PR refactors the API server to remove persistence argument and instead pass in event source in order to make the API server just another event sink.
In turn we are able to reduce the memory footprint of the hydra-node just by streaming events from disk instead of loading them all in memory. On top of this hydra-node does not serve history by default anymore.

Notable changes that came out of this PR:
❄️ API server doesn't use persistence any more and is now just another event sink
❄️ PersistenceIncremental/source doesn't check anymore for the authorized thread since we made the implementation work across threads.
❄️ API server now serves the history only on demand ?history=yes -> breaking change
❄️ We store StateEvent on disk instead of ServerOutput
❄️ Our projections now work with StateChanged type.
❄️ Some of the messages like Greetings, PostTxOnChainFailed are served directly from the api without time and seq fields so they need to be decoded separately (more work on the client side)

---

• CHANGELOG updated or not needed
• Documentation updated or not needed
• Haddocks updated or not needed
• No new TODOs introduced or explained herafter

[github-actions]

Transaction cost differences

Script summary

Name	Size (Bytes)
νInitial	-
νCommit	-
νHead	-
μHead	-
νDeposit	-

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	-	-	-
10	-	-	-	-
40	-	-	-	-

Commit transaction costs

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	-	-	-
10	-	-	-	-
54	-	-	-	-

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-	-
2	-	-	-	-	-
3	-	-	-	-	-
4	-	-	-	-	-
5	-	-	-	-	-
6	-	-	-	-	-
7	-	-	-	-	-
8	-	-	-	-	-

Cost of Increment Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	+0.39	+0.09	-
10	-	$${\color{green}-0.39}$$	$${\color{green}-0.09}$$	$${\color{green}-0.01}$$
37	-	-	-	-

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	-	-	-
10	-	-	-	-
40	-	-	-	-

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	-	-	-
10	-	-	-	-
34	-	-	-	-

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	-	-	-	-
5	-	-	-	-
10	-	-	-	-
27	-	-	-	-

FanOut transaction costs

UTxO, Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
(0, 10)	-	-	-	-	-
(1, 10)	-	-	-	-	-
(5, 10)	-	-	-	-	-
(10, 10)	-	-	-	-	-
(20, 10)	-	-	-	-	-
(37, 10)	-	-	-	-	-

[github-actions]

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2025-03-11 16:41:21.035959626 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045	2652
νCommit	61458bc2f297fff3cc5df6ac7ab57cefd87763b0b7bd722146a1035c	685
νHead	0e35115a2c7c13c68ecd8d74e4987c04d4539e337643be20bb3274bd	14756
μHead	57166715eadb8d3135964325c016eea546c21e1c0aae974ca67df9a5*	5541
νDeposit	ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c	1102

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	6091	10.98	3.42	0.53
2	6295	13.35	4.15	0.57
3	6496	15.50	4.80	0.60
5	6898	20.07	6.21	0.66
10	7903	31.36	9.67	0.82
40	13936	98.18	30.14	1.77

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	559	2.44	1.16	0.20
2	739	3.38	1.73	0.22
3	920	4.36	2.33	0.24
5	1283	6.41	3.60	0.28
10	2171	12.13	7.25	0.40
54	10059	98.61	68.52	1.88

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	525	25.64	7.39	0.43
2	114	636	34.89	9.98	0.53
3	170	747	42.28	12.15	0.61
4	226	858	56.68	16.00	0.76
5	282	969	58.74	16.86	0.79
6	341	1081	67.65	19.38	0.88
7	395	1192	91.04	25.45	1.12
8	449	1303	97.78	27.41	1.20

Cost of Increment Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	1789	25.11	8.24	0.49
2	1928	27.12	9.50	0.52
3	2113	29.83	11.07	0.56
5	2368	32.81	13.33	0.62
10	3101	41.91	19.56	0.77
38	7229	95.05	55.37	1.64

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	601	23.99	7.61	0.43
2	688	23.80	8.20	0.43
3	940	28.45	10.17	0.49
5	1178	30.77	12.12	0.54
10	1967	40.89	18.25	0.70
39	6128	94.66	52.41	1.56

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	687	29.15	9.21	0.48
2	774	30.32	10.13	0.50
3	948	32.86	11.66	0.54
5	1270	39.86	15.25	0.64
10	2026	50.38	21.88	0.80
33	5428	94.20	51.34	1.52

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	678	35.95	11.00	0.55
2	801	38.11	12.31	0.58
3	938	40.36	13.65	0.62
5	1204	44.59	16.24	0.68
10	1994	56.66	23.40	0.86
26	4378	95.88	46.47	1.45

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5896	24.08	7.82	0.66
2	6098	37.31	12.30	0.81
3	6210	46.63	15.38	0.91
4	6355	53.62	17.70	0.99
5	6564	67.65	22.37	1.15
6	6574	72.12	23.76	1.19
7	6912	89.40	29.62	1.39

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTXO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	0	0	6091	19.66	6.46	0.62
10	5	283	6259	30.31	10.59	0.75
10	10	569	6430	41.42	14.88	0.88
10	20	1140	6771	61.34	22.68	1.11
10	30	1708	7111	83.11	31.09	1.36
10	37	2105	7347	98.02	36.88	1.53

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2025-03-11 16:43:51.513716302 UTC

Baseline Scenario

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	4.399231906
P99	6.385587119999987ms
P95	5.2380849000000005ms
P50	4.2334635ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2025-03-11 16:42:36.475121441 UTC	888M	2964M
2025-03-11 16:42:41.475002363 UTC	1005M	2838M
2025-03-11 16:42:46.474958036 UTC	994M	2848M
2025-03-11 16:42:51.474954228 UTC	995M	2847M
2025-03-11 16:42:56.474992877 UTC	1011M	2830M
2025-03-11 16:43:01.475009701 UTC	1011M	2830M

Three local nodes

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	28.032962362
P99	42.53910716ms
P95	37.3937734ms
P50	26.9057805ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2025-03-11 16:43:14.504742874 UTC	923M	2928M
2025-03-11 16:43:19.505700049 UTC	1180M	2668M
2025-03-11 16:43:24.506053643 UTC	1231M	2547M
2025-03-11 16:43:29.504788508 UTC	1238M	2498M
2025-03-11 16:43:34.505078508 UTC	1242M	2493M
2025-03-11 16:43:39.505102446 UTC	1242M	2492M
2025-03-11 16:43:44.504938704 UTC	1242M	2492M
2025-03-11 16:43:49.504969891 UTC	1259M	2475M

[ch1bo]

The scope is more-or-less fine, but we can avoid some breaking changes (if we want).

Biggest issue is that this still loads all events into memory and the api server can be represented as an EventSink much more easy.

The diff contains at least one bug and a lot of commented code, outdated comments and at lest one FIXME.

[ch1bo]

The model thread delays are weird.. let's look at them another time.