Allow all mempool txs to be replaced after a configurable timeout (default 6h)

[greenaddress]

This PR' aim is to improve user experience around stuck transactions without
affecting users of zero conf transactions.

tldr: Allow transaction replacement for transactions sitting in mempool for
longer than timeout (default 6h configurable) regardless of opt-in replacement
flag.

This PR affects policy/relay only.

Stuck transactions have been a problem for users recently. While wallets
are improving (opt in replacement, Child Pays For Parent, etc) there are some
cases which find users with transactions stuck for days that can't be solved
easily/reliably by wallet developers, especially when the user creates the
stuck transaction with old software or for some reason disabled available
features countering stuck transactions.

For the purpose of the below I will ignore transactions created by the core
wallet when talking about transaction expiration/eviction and focus on policy.

Bitcoin 0.12 introduced (or in a way re-introduced) opt-in transaction
replacement (BIP125), allowing people to more explicitly flag that their
transaction can be replaced (such that users of zero conf transactions can
immediately recognize them).

At the same time mempool limiting (configurable) was introduced, making the
individual mempool drop transactions at the bottom (low fee) when full.

Both before and after these changes any transaction in mempool would be
automatically evicted after 72 hours (configurable).

Recently 0.14.0 increased the eviction from 72 hours to 2 weeks. These changes
allows users of the system to aim for lower fees but at the same time makes it
frustrating for users that disable opt-in transaction replacement or that use
software that doesn't support it in first place to bump the fee at a later time
or to revert the payment as they have to wait for a while or use ad-hoc
software.

A number of miners will mine transactions regardless of opt-in flags (5-10%
maybe) and while core nodes won't propagate those transactions, a well
connected user can generally get replacement transactions mined within a
reasonable amount of time without opt-in transaction replacement flags set.

This may be convenient for attackers or ad-hoc expert use but
not ideal for wallet developers, or at least until core merges full transaction
replacement because using this functionality would requires wallets to use
preferential peering and/or forks of bitcoin core.

Until then a compromise solution that doesn't impact zero conf use and that
improves user experience would be to allow transactions to be replaced after
sitting for a timeout in mempool (thus unconfirmed).

The timeout should be high enough that allows current use of zero conf and at
the same time allows same working day solution for users.
I suggest a 6 hours timeout and to have it configurable for testing and ability
for user to change.

The changes continue to support disabling entirely transaction replacement
(-mempoolreplacement) and introduces a new command line parameter
(-mempoolreplacementtimeout) which allows to pass the number of seconds after
which a transaction can be unconditionally replaced and setting this parameter
to two weeks will keep the original behavior.

If you want to test the changes using @petertodd Replace-by-Fee tools build
core with this PR applied and wallet enabled and run with
-mempoolreplacementtimeout=10 and use doublespend.py (with and without
-b 1) from https://github.com/petertodd/replace-by-fee-tools

[jonasschnelli]

I think such policy changes should first be discussed on the bitcoin-dev mailing list and eventually deserve a BIP.

[jonasschnelli]

Maybe use SignalsOptInRBF()

[greenaddress]

I only moved the code but indeed that's a good idea (addressed in 38f4c85) - thanks!

[TheBlueMatt]

Yea, agreed that this should get a BIP (sadly probably means endlessly trolled), but does seem awesome to me. Does this need a new option? We dont currently have an option for opt-in-rbf, why not just leave this as hardcoded policy?

[greenaddress]

@TheBlueMatt replacement can be disabled (as a node policy afaik rather than wallet) and the new option i added allows for easier testing and for people to run values that they like different from the default (either to be same as previous behavior or to get the other end without running some fork/patched core).

@TheBlueMatt @jonasschnelli I am happy to do a BIP and discuss in the dev mailing list as needed. I thought the changes may be borderline like the change from 72h to 2 weeks for mempool expiry but happy (minus potential trolling) to learn otherwise and or try to reduce the impact of the changes if this is an option.

[petertodd]

I disagree that this needs a BIP.

Opt-in RBF added a new way to interpret a transaction, which just barely qualified as something you might want to do a BIP for.

This however makes an existing behavior - transactions being replaceable in spite of them not signalling opt-in RBF - happen a little sooner in some circumstances, just like adding expiration did in the first place. We didn't create a BIP for expiry, so why does this need a BIP?

Secondly, writing a BIP for such a trivial change gives the misleading impression that you could rely on the opposite behavior. We've had continual problems with people misunderstanding the security properties of zeroconf; there's no reason to add to that problem.

[luke-jr]

Indeed, mere policies are not BIP material...

[gmaxwell]

People might want to know about it... and a BIP might be a good way to communicate it... but we certainly didn't write a BIP about the expiration time changes over time, and this is strictly a narrower change.

Dunno how ideal 6 hours is though.

[EagleTM]

I'd suggest to put the timeout at 72h - at least when introducing the functionality. This way the behviour is similar to pre 0.14.x code. It's less likely to create havoc / confusion for operators who are still used to regularly see 0-conf transactions being re-spent after that time-frame (but not earlier).

[petertodd]

@gmaxwell I suggested six hours because it's more than long enough (36 blocks) that if you wanted that tx mined relatively quickly, you're probably getting annoyed that it isn't.

@EagleTM Try actually doublespending some time - it's a lot easier than you think it is.