Use transaction ids in snapshots

[abailly-iohk]

This PR is a first step towards addressing the issue outlined in #904, as it replaces the use of the full transaction with the transaction id in the Snapshot body and therefore in SnapshotConfirmed messages sent to the client.

It remains to implement the replacement of full tx with txid in the ReqSn message within the protocol.

---

• CHANGELOG updated or not needed
• Documentation updated or not needed
• Haddocks updated or not needed
• No new TODOs introduced or explained herafter

[github-actions]

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-06-19 12:23:16.884203672 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2212a4ee618434b9b2f366d7c330dbdfb5c7072e793a850fd0de6ddd	4294
νCommit	69e1ccf9ad73dc6d37a5bc8de5aec86f3c4c1710fe5fd334e0e16b18	2124
νHead	8ae095dca4d14a1b8edffb37faa6c84ec60340fbf389a62f027e0b76	9355
μHead	33642a45c7fbb955ce1704ef09229bb211bf9af9980530db28c6aafe*	4148

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4739	13.44	5.27	0.51
2	4951	16.19	6.32	0.54
3	5152	17.17	6.64	0.56
5	5559	21.45	8.24	0.63
10	6584	35.39	13.57	0.82
37	12121	98.64	37.40	1.75

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	599	14.89	5.70	0.34
2	787	19.66	7.73	0.40
3	975	24.66	9.84	0.46
5	1346	36.07	14.56	0.61
10	2283	71.73	28.85	1.04
13	2852	98.11	39.18	1.35

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	815	27.89	10.83	0.49
2	113	1135	43.42	16.99	0.67
3	171	1456	61.54	24.24	0.89
4	226	1774	82.78	32.76	1.13

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	639	18.88	8.42	0.39
2	804	20.27	9.68	0.42
3	969	21.37	10.82	0.44
5	964	20.73	8.83	0.42
10	2124	31.12	19.62	0.64
50	8725	87.17	69.99	1.74

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	675	24.34	10.47	0.45
2	841	26.06	11.84	0.48
3	995	28.20	13.38	0.52
5	1337	31.20	15.95	0.58
10	2160	39.77	22.80	0.73
44	7773	98.46	69.53	1.79

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4856	22.45	9.40	0.61
2	5175	36.68	15.56	0.79
3	5498	53.72	22.99	0.99
4	5818	73.46	31.59	1.23
5	6141	96.13	41.49	1.50

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4764	8.66	3.57	0.46
5	1	57	4800	10.06	4.39	0.47
5	5	285	4944	15.64	7.69	0.55
5	10	569	5124	22.61	11.82	0.64
5	20	1137	5483	36.56	20.07	0.83
5	30	1707	5840	50.52	28.33	1.02
5	40	2277	6204	64.49	36.60	1.21
5	50	2849	6567	78.46	44.87	1.40
5	65	3703	7105	99.42	57.28	1.68

[github-actions]

Test Results

312 tests   - 11   306 ✔️  - 11   20m 17s ⏱️ -14s
106 suites  -   4       6 💤 ±  0 
    5 files    -   1       0 ❌ ±  0 

Results for commit 95948ae. ± Comparison against base commit 4a27e4c.

This pull request removes 11 tests.

Hydra.TUI.Options ‑ parses --cardano-signing-key option
Hydra.TUI.Options ‑ parses --connect option
Hydra.TUI.Options ‑ parses --node-socket option
Hydra.TUI.Options ‑ parses --testnet-magic option
Hydra.TUI/end-to-end smoke tests ‑ display feedback long enough
Hydra.TUI/end-to-end smoke tests ‑ doesn't allow multiple initializations
Hydra.TUI/end-to-end smoke tests ‑ starts & renders
Hydra.TUI/end-to-end smoke tests ‑ supports the full Head life cycle
Hydra.TUI/end-to-end smoke tests ‑ supports the init & abort Head life cycle
Hydra.TUI/text rendering errors ‑ should show not enough fuel message and suggestion
…

♻️ This comment has been updated with latest results.

[pgrange]

We need to update the TUI if we want it to work with this change:

hydra-tui: ExceptionInLinkedThread (ThreadId 6) (ClientJSONDecodeError "Error in $.snapshot.confirmedTransactions[0]: parsing TxId failed, expected String, but encountered Object" "{\"headId\":\"b2e49bacf4bd815db4dab65ee9591e918cedf3c3adf24f6add212360\",\"seq\":9,\"signatures\":{\"multiSignature\":[\"fa91a6bd45e2805d4722deac4564d0a1a2077a0506538843edb235596721ddfd2733c916eeaaddddabb4d530043473341efc071086e87a94d77e62d89f6ba502\",\"323298b0c3b13a4f7c856440eefe4b1380519e86fc8ab6edac7d825e03ce0e2ed3614f233c982ba34dc581f8fe922cbb7c8dfd538c7754264e0e84e272fa380a\",\"2e70bfbf59368c1eac077e13f25bab60f6e2b8b2c5990b34b2045aed3789c36be19d9b5ee893880b907d712fd71c315e192a89b31a2d9032232544ad04924a0c\"]},\"snapshot\":{\"confirmedTransactions\":[{\"body\":{\"fees\":0,\"inputs\":[\"ddf1db5cc1d110528828e22984d237b275af510dc82d0e7a8fc941469277e31e#0\"],\"outputs\":[{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":1000000000}}]},\"id\":\"2ee2cb485d7ceda00a1f9f8fc9eac0be46cc03011ec0177c008160389f6145d8\",\"isValid\":true,\"witnesses\":{\"keys\":[\"8200825820eb94e8236e2099357fa499bfbc415968691573f25ec77435b7949f5fdfaa5da0584091d7f0e233c15745a3f793e35bf29f6cd2ed767f1f51032ae8202d7117b839d98d84858607796fbb583933b653911170a6b81b5122cabcdb2651e3d89b6e0001\"]}}],\"snapshotNumber\":1,\"utxo\":{\"2ee2cb485d7ceda00a1f9f8fc9eac0be46cc03011ec0177c008160389f6145d8#0\":{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":1000000000}},\"5c05106772f97eacce0a31ac215ef87aa0746279008bdb3cd07e3abece6d3985#0\":{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":500000000}},\"c4c4804879236033b77292c1cfe44c750a32ac88b998a3ec16533392990daeb6#0\":{\"address\":\"addr_test1vqa25t3aayfmpad20elswmsj94ehmdfjnhc64yz3jg5yl6skf5cck\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":250000000}}}},\"tag\":\"SnapshotConfirmed\",\"timestamp\":\"2023-06-12T15:53:18.840950541Z\"}")

[abailly-iohk]

@pgrange How is this failing? I am running all the tests and they pass, and I can't find where this issue comes from. Are you trying to run a TUI from this PR against a hydra-node from an earlier version? It of course can't work out of the box because the change in SnapshotConfirmed message is a breaking change.

[ch1bo]

Change looks good and "feels right". Only the tx transformatin on the API is now not needed and yes, the TUI likely needs updating. The TUI should be fine as it is re-using the types from hydra-node. Users just need to ensure to use the same version as the node.

(I also just realized that the hydra-tui tests built via nix now have weird and maybe problematic output as it emulates an interactive terminal already for the nix build)

[ch1bo]

Should: make this now only a encodedResponse

This and the other ConfirmedSnapshot cases now don't contain transactions and hence do not need to be transformed.

[pgrange]

@abailly-iohk I'm running (hopefully) this PR tui against this PR hydra-node and get the following error:

hydra-tui: ExceptionInLinkedThread (ThreadId 6) (ClientJSONDecodeError "Error in $.snapshot.confirmedTransactions[0]: parsing TxId failed, expected String, but encountered Object" "{\"headId\":\"b2e49bacf4bd815db4dab65ee9591e918cedf3c3adf24f6add212360\",\"seq\":9,\"signatures\":{\"multiSignature\":[\"fa91a6bd45e2805d4722deac4564d0a1a2077a0506538843edb235596721ddfd2733c916eeaaddddabb4d530043473341efc071086e87a94d77e62d89f6ba502\",\"323298b0c3b13a4f7c856440eefe4b1380519e86fc8ab6edac7d825e03ce0e2ed3614f233c982ba34dc581f8fe922cbb7c8dfd538c7754264e0e84e272fa380a\",\"2e70bfbf59368c1eac077e13f25bab60f6e2b8b2c5990b34b2045aed3789c36be19d9b5ee893880b907d712fd71c315e192a89b31a2d9032232544ad04924a0c\"]},\"snapshot\":{\"confirmedTransactions\":[{\"body\":{\"fees\":0,\"inputs\":[\"ddf1db5cc1d110528828e22984d237b275af510dc82d0e7a8fc941469277e31e#0\"],\"outputs\":[{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":1000000000}}]},\"id\":\"2ee2cb485d7ceda00a1f9f8fc9eac0be46cc03011ec0177c008160389f6145d8\",\"isValid\":true,\"witnesses\":{\"keys\":[\"8200825820eb94e8236e2099357fa499bfbc415968691573f25ec77435b7949f5fdfaa5da0584091d7f0e233c15745a3f793e35bf29f6cd2ed767f1f51032ae8202d7117b839d98d84858607796fbb583933b653911170a6b81b5122cabcdb2651e3d89b6e0001\"]}}],\"snapshotNumber\":1,\"utxo\":{\"2ee2cb485d7ceda00a1f9f8fc9eac0be46cc03011ec0177c008160389f6145d8#0\":{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":1000000000}},\"5c05106772f97eacce0a31ac215ef87aa0746279008bdb3cd07e3abece6d3985#0\":{\"address\":\"addr_test1vqg9ywrpx6e50uam03nlu0ewunh3yrscxmjayurmkp52lfskgkq5k\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":500000000}},\"c4c4804879236033b77292c1cfe44c750a32ac88b998a3ec16533392990daeb6#0\":{\"address\":\"addr_test1vqa25t3aayfmpad20elswmsj94ehmdfjnhc64yz3jg5yl6skf5cck\",\"datum\":null,\"datumhash\":null,\"inlineDatum\":null,\"referenceScript\":null,\"value\":{\"lovelace\":250000000}}}},\"tag\":\"SnapshotConfirmed\",\"timestamp\":\"2023-06-12T15:53:18.840950541Z\"}")

I will triple check my configuration. It might well be the case that I'm not running the good tui. Let's see...

[pgrange]

@abailly-iohk I was, indeed, not using the right version of the TUI. So it works, my bad. (we should add a --version option to the tui, by the way, as it saved my life before with the hydra-node).

[ch1bo]

As pascal pointed out, this will be a significant change on the client-side api and we should get feedback from some early adopting users.

@uhbif19 @matiwinnetou @Yasuke @hSloan @luigy This PR is removing transactions from SnapshotConfirmed and only leaves transaction ids in this message. To see the transactions submitted by other hydra nodes, a client would need to resort to the TxValid message or keep around their own transactions they submit via NewTx. Would this change be okay with you?

[matiwinnetou]

For sure this is a big change but we can adjust to the API, of course breaking changes will be made for us and app developers.

Keeping own transactions is easy, the tricker part is keeping others transactions (also doable).

I don't know if any of this is negotiable, if there is an issue and it needs to be done - it needs to be done. I believe you are not breaking the client API contract "for the fun of it" :)

[abailly-iohk]

@pgrange ;)

[abailly-iohk]

@matiwinnetou If you think this makes more sense, we could keep the API as is.

[abailly-iohk]

But I think it generally goes in the sense of a leaner API that we only pass the confirmed txs ids instead of the full transaction: The transactions are observed by clients anyway through the TxValid/TxInvalid messages

[uhbif19]

@ch1bo

We only use utxo and Tx IDs from Snapshot for now.
But overall I think there are situations, where this would require complication of introducing statefullness in Hydra client.
So, if it is only change in API, it would be better to have option for full Txs in answers.

Also I still do not understand which responses to client are broadcasted, and which are not. So TxValid/TxInvalid is broadcasted across all Hydra nodes?

[abailly]

@uhbif19 This is a fair point that should be addressed by #686. There should exist an API that allows clients to retrieve (part of) the state of the Hydra Head without having to follow the stream of ServerOutput from the beginning, to better support statelessness of clients.

[uhbif19]

There should exist an API that allows clients to retrieve (part of) the state of the Hydra Head without having to follow the stream of ServerOutput from the beginning

That is a different issue. That requires to do some query ClientInput and either block until answer or once again introduce state.

[Yasuke]

@ch1bo I think this change makes sense, the API said that transactions (in a SnapshotConfirmed) could be in a bunch of different shapes originally, and I think the Id is really all that matters here, it is consistent across all forms and is likely more flexible for people who build and submit their transactions in those different shapes.

Being able to get a SnapshotConfirmed and check it against the txids that have been submitted sounds like it will 'just work'.

So I am for this change.