ReqSn only sends transaction ids

[ch1bo]

Why

Users expect high throughput of transactions in the Hydra Head.

Transactions are communicated between Hydra participants through the layer two network. Currently the transactions are submitted twice, once in a ReqTx message and gain in the ReqSn message.

The protocol is defined with both kinds of network messages precisely to increase concurrency with minimal increase in bandwidth demands - which is lost if we submit the transactions twice.

What

• Not send transactions with ReqSn, but only the transaction ids (hashes of transaction bodies).

  This requires additional book-keeping in the Head protocol:

  • Keep a mapping of TxId -> Tx and extend it on every ReqTx
  • ReqSn should only include a [TxId]
  • Resolve TxId -> Tx before checking their applicability on receiving ReqSn

• Update specification with new behavior Make the spec consistent with #904 #974

• Do benchmarks of the off-chain performance before and after this change with a discussion / conclusion on this issue or the respective PR. We have a benchmark on a local cluster of hydra-nodes in hydra-cluster.

• Test with Hydraw which currently leads to deadlocks when it conflicts

• Create a documentation page about the L2 consensus and how it behaves in prose (core concepts section). This should touch on topics like liveness, fairness, front-running, etc. Add a FAQ entry about:

  • What happens when a hydra-node is offline? (explain liveness constraint)
  • Does the Hydra L2 consensus prevent front-running?

  And reference the specification section 6.4.2 (currently) for details. Could also use the motivation of the PR Proposed snapshot should contain only seen transactions #904 (comment) as input.

How

• Likely requires not only a seenTxs, but a allTxs :: Map TxId Tx analogous to what is in the spec
• We want to only use the seenTxs to resolve transactions, no allTxs as this would be an ever growing set (we might explore having this all txs set separately)
• Ok, definitely use a allTxs :: Map TxId Tx analogous to what is in the spec otherwise the head is very easily stuck in case of conflicting transactions
• transactions included in a snapshot should be removed from allTxs

[pgrange]

We need to run benchmark with/without this improvement and compare performances to ensure it makes a difference.

[ch1bo]

We need to also make sure that we keep track of all transactions (T) to properly handle a ReqSn which requests a snapshot using transactions that are conflicting with our local view (T hat).

Roughly the red items from this draft:

[ch1bo]

Marked as an idea again as it

• requires a clear need by a concrete use case
• we need to measure the status quo and identify transaction submission to be impacting performance.

[ch1bo]

After having found a reason to wanting a more "complete" set of requested transactions with #904, we decided to continue on this item.

[abailly-iohk]

Ran a couple benchmarks on 0.11.0 baseline and branch from #904 and the results are similar so there is no apparent improvement to be expected now with this change.

[ch1bo]

Ran a couple benchmarks on 0.11.0 baseline and branch from #904 and the results are similar so there is no apparent improvement to be expected now with this change.

An improvement is only expected on networked setups. Our benchmarks usually run local nodes, where there is not really a limited network bandwidth. But it's good that there is no regression.

[abailly]

Sure, but we could have expected some slight speedup due to less work being done in serialisation.

[ch1bo]

Also tested with hydraw where two users spammed to spend the same UTxO with different colors/positions which leads to conflicting transactions (metadata is different at least). The Head just worked fine with this: