Misleading error feedback when several clients fanout at the same time

[pgrange]

Context & versions

0.10.0 unstable

Steps to reproduce

1. Open a head with several parties
2. Close the head
3. Wait for the contestation period to appear
4. from all the parties, try to fanout the head simultaneously

Actual behavior

The head fanout does success but all but one party will return to the client: PostTxOnChainFailed which makes the user confused about what happened.

#832 contains more details with error logs from one node.

Expected behavior

If the fanout succeed, no party should see no error.

Extra Information

The fanout transaction needs to spend the output of the close transaction (or the last contest). There is only one such output so if we have several parties trying to spend this output in a fanout transaction, of course, only one of these transaction will succeed and the other transactions will fail.

The hydra-node whose transactions fail would receive an error of the form:

HardForkApplyTxErrFromEra
    S
    ( S
        ( S
            ( S
                ( S
                    ( Z
                        ( WrapApplyTxErr
                            { unwrapApplyTxErr =
                                ApplyTxError
                                  [ UtxowFailure
                                      ( FromAlonzoUtxowFail
                                          ( NonOutputSupplimentaryDatums
                                              ( fromList
                                                  [SafeHash "94823cbee8d16b5154da8f6e57cbd6a9de165ffef1299fa8520d3ca6acd36da5"]
                                              )
                                              (fromList [])
                                          )
                                      )
                                  , UtxowFailure (FromAlonzoUtxowFail (ExtraRedeemers [RdmrPtr Spend 0]))
                                  , UtxowFailure (UtxoFailure (FromAlonzoUtxoFail (UtxosFailure (CollectErrors [BadTranslation (TranslationLogicMissingInput (TxIn (TxId{_unTxId = SafeHash "0b7ada4617bf2a7180f9eac4ea60a7a95f5c90c853af8bb309f37b4d2a2092c7"}) (TxIx 0)))]))))
                                  , UtxowFailure
                                      ( UtxoFailure
                                          ( FromAlonzoUtxoFail
                                              ( ValueNotConservedUTxO
                                                  ( Value
                                                      96471200
                                                      ( fromList
                                                          [
                                                            ( PolicyID{policyID = ScriptHash "6b4a78a641be2867925749129f97f8a5965631b9c80a222f82188355"}
                                                            , fromList
                                                                [ (14 b045cc21d4316f6ac896a438ac194fc19573619181c6a44a540215, -1)
                                                                , (4879647261486561645631, -1)
                                                                , (c6813d435dfed1f0d6dda71ebf97285ad1214a6dfd3abcfa4b740ab1, -1)
                                                                , (ddde64bb6ae682b34b69ad651375b6b23abfa9860d46329f4444fd31, -1)
                                                                ]
                                                            )
                                                          ]
                                                      )
                                                  )
                                                  ( Value
                                                      856471200
                                                      ( fromList
                                                          [
                                                            ( PolicyID{policyID = ScriptHash "2035da628491aba52c1ab37ff68124d40fd49c0e5f258992229f0da7"}
                                                            , fromList [(566 f7563686572, 1)]
                                                            )
                                                          ]
                                                      )
                                                  )
                                              )
                                          )
                                      )
                                  , UtxowFailure
                                      ( UtxoFailure
                                          ( FromAlonzoUtxoFail
                                              ( BadInputsUTxO
                                                  ( fromList
                                                      [TxIn (TxId{_unTxId = SafeHash "0b7ada4617bf2a7180f9eac4ea60a7a95f5c90c853af8bb309f37b4d2a2092c7"}) (TxIx 0)]
                                                  )
                                              )
                                          )
                                      )
                                  ]
                            }
                        )
                    )
                )
            )
        )
    )

Are we able to figure out that this is an ok error and, either ignore it, or raise a AlreadyFanedOut error to the client?

[ch1bo]

Had a look on the user-provided logs in #832 and it seems we get the PostTxOnChainFailed before seeing any OnFanoutTx. This would likely come just after and the hydra-node hence has not enough information to not report the PostTxOnChainFailed right-away, i.e. stateless handling of this is not possible and we would need to "wait for a bit" in case we see an OnFanoutTx observation event "shortly after"?

This applies to any state transitions via chain transactions (Collect, Close, Contest) which consume the same UTxO to advance the head life cycle on-chain.

[ch1bo]

Our options are not great:

• The chain layer does not "really understand" the state transitions
• The head logic does not understand details about transactions (txin already spent)
  • The ledger errors are very annoying to work with and unpack.
• Maybe there is the option to do interconnect the tx submission with the chain sync portions a bit better to "have the observation" before the error. Because the node already saw the transaction (otherwise the inputs would not be already spent), but errors propagate faster than observations out of our chain layer.

Besides: it would be perfectly fine to ignore this if we see the head advance correctly into the next state. Should we "tone down" the message of this error as it is something which is not so severe?

Also: This also reminds us from the "pending transaction" situation where we made the TUI block before submitting another transaction.

---

Next step: Spend 1-2 hours to analyze this situation (either on Fanout or Collect) in our e2e tests (controlled environment) to find out what the cardano-node behavior is exactly in this case. i.e. does it already have the block with the fanout/collect tx in it and just doesn't tell us?