Use CPFP to resolve stuck transactions from too low miner fees

[chimp1984]

This is a Bisq Network proposal. Please familiarize yourself with the submission and review process.

Problem

Fee estimation is a conceptual problem lacking a "perfect" solution. As relevant transactions in Bisq are created in an interactive context, some possible solutions like RBF [1] are not feasible. A stuck tx due too low miner fee which never gets confirmed can cause additional problems if the change outputs have been used for other offers and trades, leading to a cascade of failed trades. We had that experience in the past when fees have been spiking and sustained on a high level for longer period. The situation today might be a bit better as we have a better fee estimation service but it can happen again any time.

Background

The miner fee for trade txs in Bisq is used from fee estimation services. Those give the best estimation at the current time but cannot look into the future. It can happen that just after the tx is published the fee increases a lot and sustain on a high level and mempool gets filled up for weeks. The settings how many txs are kept in the mempool is a local setting of any node but average I think it is about 2 weeks to get dropped from mempool if the tx was still not confirmed.
If the change outputs of such txs have been used for other offers and trades the problem escalates further as all those will never get confirmed as well, even if they might have used a better fee.

To use RBF comes with the problem that the trade txs would need to be recreated with the peer, requiring that both are online at the same time which is problematic. Furthermore any folllow up tx would need to be recreated as well if the parent tx change (tx, miner fee -> diff. change output). Beside that BitcoinJ has not implemented RBF ;-(.

We have the same problematic for attempts to create multiple txs with diff. fee levels and pre-sign all and broadcast the next higher fee tx in case the former does not get confirmed in time. That would also make change outputs basically un-usable.

To not allow to spend unconfirmed change outputs is for UX a no-go. One would need to wait each time after creating an offer until the tx is confirmed before being able to spend the outputs.

Note, that there are no security risk in the context of Bisq by spending unconfirmed outputs. The start of the trade requires a confirmation and that guaratees that the txs cannot be double spent. Double spent of any tx earlier would render the trade invalid.

Possible solution

We could use CPFP [2] and add mandatory small outputs which gets reserved until the tx is confirmed. In case of a stuck tx we can use those change outputs for boosting the tx chain with CPFP.

The good thing with CPFP is that it can be applied after the fact without any prior requirements (beside that an output is available). We could add a mandatory output to each trader to the deposit tx and keep that utxo in a reserved state similar as we do it with reserved balance for the amount needed for an offer, but we should use a new model based on tx outputs not on addresses (as with the AddressEntry model) as that was a bad design choice.
Once the tx is confirmed that utxo gets released and can be used for any future tx.

If a tx gets stuck because of too low fees the app calculates the required fee to boost the tx chain to have more realistic chances to get confirmed. A tx spending the utxo is created and sufficient miner fee added so that miners take the whole tx chain to be able to earn the premium fee.

Who pays for it?

An open problem is who of the traders is willing to make that tx and carries the costs for it.
Another problem is that one of the traders could have a very long chain of unconfirmed txs used for funding the trade fee tx. It would be unfair to the peer to pay for that.

So the protocol should detect who caused more cost to the chain. That user need to do the boost tx, it could/should be done automatically in the background.
The part of the cost caused from the peers side will be deducted at the payout tx, so the other peer will receive less than the normal security deposit. In case the peer is the seller and the deposit would not cover those costs (I guess that is unlikely) we could communicate it before the fiat or altcoin is sent so we deduct the cost from the transferred fiat/altcoin amount.
As this case seems to be rather rare, I think its acceptable to let the trade fail in such a case and leave it to mediation to resolve such a situation.
For the seller has done the tx and the costs exceed the deposit the buyer would receive less of the trade amount. The case that the trade amount is not sufficient as well will be very rare and can be treated as well manually via mediation.

Cost calucaltion

Maker:

• Cost for maker fee tx
• All depending txs of inputs of the maker fee tx

Taker:

• Cost for taker fee tx
• All depending txs of inputs of the taker fee tx
• Cost for deposit tx (as it is now)
• Cost for payout tx (as it is now)

It could be discussed if the costs for the deposit tx and payout tx should be shared in that case. The reason the taker pays for that in the current protocol is because the maker cannot know the required fee at maker offer time when he reserves the funds, so it has pure protocol technical reasons and was not intended conceptually.

Limitation

We are only concerned about the deposit tx not the payout tx. The payout tx could be boosted anyway from any user and if it becomes inputs of a future offer/trade and that get stuck it would become a part of boosting that new trades deposit tx.

Downsides

• The deposit tx is getting a bit larger so miner fees will be a bit higher. I think that will not be significant as inputs are usually the larger part.
• It adds complexity to the protocol.

Implementation

I think it can be implemented without breaking backward compatibility and the effort is not that high overall.
Here a rough overview of the main aspects:

• Add mandatory extra outputs to both trades to the deposit tx.
• Add persitable data structure for managing the utxos. Those are tagged by the trade ID and released once the deposit tx is confirmed.
• Add API support for requesting all the unconfirmed txs not known in our wallet (the peers txs as well as foreign funding txs, as our own wallet txs are known).
• Define threshold when a deposit tx is considered stuck (in blocks).
• If that threshold is reached execute the boost tx publishing
• Re-adjust the payout distrubution for such a case.

We should consider how that proposal fit into the ideas at #279
and maybe postpone implementation and do one big protocol change instead of that intermediary improvement.

Alternative idea

There might be an alternative idea to the above.
We could let the traders create an cancel-trade tx which spend the output from the deposit tx by refunding both traders and the extra fee costs are deducted from each sides.
The downside of that is the both traders need another interactive tx signing process. It could be done without requiring that both are online at the same time, but by sending the signature as mailbox msg.
It might be overall an easier approach which does not interfere with existing protocol and would be a pure additional sub protocol.

I made another proposal for that idea: #287
I think that would be superior as it is less intrusive and less effort.

[1] https://bitcoinops.org/en/topics/replace-by-fee/
[2] https://bitcoinops.org/en/topics/cpfp/

[LeadManPL]

@chimp1984 : you wrote BitcoinJ does not support RBF - According to this issue this might be possible. Please revise, and tell us if this is possible, according to mentioned issue.

Also looking at the version 0.14 release notes mention some methods for detecting RBF:
TransactionInput.isOptInFullRBF() and Transaction.isOptInFullRBF() tell if a transactions opt into being replaced
... maybe there is more?

Unfortunately I wasn't able to find anything more RBF related in their documentation

[chimp1984]

The issue does not show anything that they implemented RBF, quite the opposite as it got closed by the maintainer without comment. Detecting RBF is probablt done for their risk management code.