[wallet] Use destination groups instead of coins in coin select

[kallewoof]

This PR adds an optional (off by default) -avoidpartialspends flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.

It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).

For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).

Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.

Example: a node has four outputs linked to two addresses A and B:

• 1.0 btc to A
• 0.5 btc to A
• 1.0 btc to B
• 0.5 btc to B

The node sends 0.2 btc to C. Without -avoidpartialspends, the following coin selection will occur:

• 0.5 btc to A or B is picked
• 0.2 btc is output to C
• 0.3 - fee is output to (unique change address)

With -avoidpartialspends, the following will instead happen:

• Both of (0.5, 1.0) btc to A or B is picked (one or the other pair)
• 0.2 btc is output to C
• 1.3 - fee is output to (unique change address)

As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.

This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in #10386 (comment) and #10386 (comment).

Together with -avoidreuse, this fully addresses the concerns in #10065 I believe.

[luke-jr]

It doesn't avoid reuse (you still need to sign for each input), and has security issues in the event of QC (but that's already a hypothetical big issue for us anyway), but seems useful anyway.

Is this affected by the current coin control grouping defect, where it includes change in the same group as unrelated outputs? That would be a serious privacy risk, I think, since it would basically tell the world which output was change.

[kallewoof]

@luke-jr:

It doesn't avoid reuse (you still need to sign for each input), and has security issues in the event of QC (but that's already a hypothetical big issue for us anyway), but seems useful anyway.

Together with -avoidreuse in #10386 it avoids reuse, I think. Are there any cases I'm missing?

Is this affected by the current coin control grouping defect, where it includes change in the same group as unrelated outputs? That would be a serious privacy risk, I think, since it would basically tell the world which output was change.

I'm not sure what this defect is. Is it described somewhere?

[ziggamon]

Does this also keep track of the change outputs in case more coins arrive later to A?

Imagine your example, where the coins from A go to C, with a change output to what we now call D.

If more coins arrive at A later, will they be spent together with D?

[kallewoof]

@ziggamon Change outputs will go to a different destination so they will be considered separate from their inputs. In other words, D in your case will not be considered related to A in this case, and will not be associated with any coins arriving at A afterwards.

[NicolasDorier]

I think this is avoidable complexity, because avoidpartial will be always higher fees.
I think it is OK, you pay fee for privacy, while giving incentive to privacy over fee by default.

[kallewoof]

Actually avoidpartial gives the same fee a lot of the times, from what I've seen so far. It depends on your utxo set, of course.

[NicolasDorier]

Some notes I am leaving here but to think about:

• Making it default is the right way, but we should keep in mind that people relying on address reuse will pay way higher fee, so they need to be notified appropriately.
  Virtually all japanese exchanges are using address reuse (1 address per customer), this change would impact them a lot. I am unsure if they depend on Bitcoin Core Wallet though.
• I am unsure if Bitcoin Core filter uneconomical outputs (filtering outputs which are more costly than their value to spend), but if bitcoin core implements this, we need to make sure this rule is always enforced at the Output level and not at the OutputGroup level. (Need to be enforced before you create the group)

[NicolasDorier]

I think coinControl->m_avoid_partial_spends should be a boolean, and coinControl should set its value based on the arg avoidpartialspends just after being created. (At worse, just before this call for now)

This will make the code easier to reason about.

[kallewoof]

Unfortunately that means either adding a coincontrol.cpp file and moving the constructor there, or adding a dependency on the gArg thing in coincontrol.h, which seems a bit overkill just for this. I'm very open to suggestions though; the short thing is an undesired hack.

[NicolasDorier]

what about std::optional<bool> ?

[maflcko]

(since C++17) 😛

[NicolasDorier]

Ah I forgot we were using boost for that... I would still prefer boost than a short though.

[kallewoof]

Changed to using a boost::optional.

[kallewoof]

@NicolasDorier

Making it default is the right way, but we should keep in mind that people relying on address reuse will pay way higher fee, so they need to be notified appropriately.

I think keeping it off by default for now is reasonable, until we get enough of an idea of the general consensus among people. I suspect, though, that most people won't care, as long as it doesn't give overly huge fee increases.

Virtually all japanese exchanges are using address reuse (1 address per customer), this change would impact them a lot. I am unsure if they depend on Bitcoin Core Wallet though.

Also some people have a "permanent" address that they receive payments to over a long period of time (e.g. charities). They also care little about privacy, usually, as their address is publicly tied to them anyway.

I am unsure if Bitcoin Core filter uneconomical outputs (filtering outputs which are more costly than their value to spend), but if bitcoin core implements this, we need to make sure this rule is always enforced at the Output level and not at the OutputGroup level. (Need to be enforced before you create the group)

Good point. I think the dust limit check can be added to the grouping method fairly easily.

[NicolasDorier]

I am unsure yet, but I think you can make the diff way smaller by keeping the old signature of this method and building the OutputGroup at the scope (local variable) of this function only.

[kallewoof]

This method doesn't know whether you want single or grouped as it doesn't have coincontrol. Would have to add a bool for that at least. Not sure it'll become smaller though. It'll just move code around.

[NicolasDorier]

So the way I did it with NBitcoin is to keep the concept of group internal to only the method which does coin packing.

If grouping is deactivated, I group by random, that way there is a single code path for everything.

I think it is a way smaller code change because all consumer of SelectCoinsMinConf do not have to change (you can make the bool default to false to keep original behavior)
The concept of group does not make lot's of sense anywhere outside of this method.

[kallewoof]

I discussed with @NicolasDorier and we concluded that the current way this is done is okay (NBitcoin does it differently so it made sense there).

[NicolasDorier]

would say Privacy is improved as an address...

[kallewoof]

Changed