Clients leak IPs if they are recipients of a transaction

[vessenes]

So, this paper http://ifca.ai/fc14/papers/fc14_submission_71.pdf got me thinking about the current rules for transaction rebroadcasting:

Once a transaction has been broadcast, you stop rebroadcasting. Unless you own txins or txouts in the transaction.

So, you use the paper's techniques. But you can be much more speculative than they are, and get a low-likelihood but possible IP match for an address, connect your client up, and issue a transaction paying yourself and a small amount to the address you're interested in, just over the dust amount.

The transaction should be constructed so it's unlikely to be mined.

The transaction traverses the network, then it stops being rebroadcast, except by the recipient and you. If your client is connected to the wallet that owns the address, it will see rebroadcasting for some time, providing a very strong link between the two.

This seems like a bad outcome.

I speculate that the 1Sochi transactions may have this motivation -- mapping addresses to determine IPs at large in Bitcoin.

It seems like the simplest thing would be to not re-re-broadcast to clients you've already spoken to, but I'll wait for smarter people than me to work out the right fix.

[laanwj]

This can be avoided by using Tor or another anonymity overlay network for transaction (re)broadcast.

Not re-broadcasting to clients you have already spoken does not sound like a solution, as an attacker can easily have a lot of IPs available. Also when mempool expiration is implemented this will make it harder to rebroadcast transactions (at a certain point all the reliable nodes will be marked).

[vessenes]

I'm just saying I watched U Penn researchers talk very knowledgeably about exact nodes rebroadcasting repeatedly; they had very pretty graphs. I don't think most people expect that having the qt wallet turned on means they are broadcasting IP-Address linkages to the world.

I'm not enough of an expert on Tor to comment as to whether repeatedly issuing the same packets out over and over breaks some assumptions Tor makes as to anonymity, but I am suspicious that it's not an ideal communication pattern even there.

[laanwj]

Well at least when using Tor none of your peers can see what your IP is.

Sure, someone monitoring all the world's communication lines could make educated guesses based on timing correlations and such, but that's not the kind of threat scenario most people have to deal with. Or that we can even hope to protect against.

For directly hiding your IP (for example from criminals), using Tor works well, and many people are using bitcoin behind Tor for that reason alone. From what I've heard, BitcoinJ is even going to make operating over Tor the default (then again, it's more urgent there as an SPV client gives away much more information to peers).

[vessenes]

I know there's a long list - o - bugs, just wanted to put this out there; it seemed surprising that you could induce a very long period of 'beaconing' from the qt client just by sending it an unlikely to be mined transaction in which it has an economic stake.

[leo-bogert]

@laanwj : I think outsourcing anonymity to Tor is very wrong for algorithmic reasons:
The foundation of anonymous P2P networks is that data is not directly transferred from sender to receiver, but relayed among multiple people, where each of the relayers does not tell the others for whom the data is. So its difficult to determine who wants/sends the data as multiple people seem to want/send it. Each of them can claim to only be a relay - plausible deniability.
Unfortunately, there are two things in an optimization conflict: The more people you relay data through, the more anonymous it is - but also slower. So the level of anonymity has to be limited by relaying through a limited amount of peers for performance reasons.
With Bitcoin, we do something on the whole network which would be ABSOLUTELY unwanted on a regular P2P network: Flooding. Every transaction is sent to everyone, because thats the only way of preventing double-spending. This is horribly inefficient, because for each single transaction between 2 peers, the whole network has to talk.
Luckily, this also is the ideal version of an anonymous P2P network as I described it above: Instead of limiting the anonymization to a certain level of relays, Bitcoin relays everything to everyone. This is as much noise to hide sender/receiver as you can possibly get.
So Bitcoin needs to buy the horribly expensive mathematical foundation for perfect anonymization anyway - it would be a shame to not use it by having such glitches as this bug describes.

[laanwj]

But no matter what, as soon as a node is seen broadcasting a previously-seen transaction a second time (say, more than an hour later) you know he's either the original sender or recipient of the transaction.

One way to avoid this would be to avoid rebroadcasting of transactions completely (after the initial flurry). The sender would have to make sure he's really well-connected the first time to avoid the transaction from getting lost. The receiver would never (re-)broadcast the transaction at all. If your transaction gets stuck, there's nothing that can be done about it.

Another way to hide would be for nodes to randomly re-broadcast other people's transactions from the mempool. But I think this won't interact well with memory pool expiration.

[leo-bogert]

@laanwj :
Yes you are right, rebroadcasting your own transactions only of course reveals that they belong to you. But why are we doing that anyway?
You already mentioned a valid replacement for this functionality: Re-broadcast random transactions, not your own. You said we don't do this because it messes with the mempool expiration, I don't understand that. If I understand the mempool correctly, it stores transactions in memory which have not been seen in a block yet. They have a timeout, and if it expires before they are seen in a block, they are removed from the mempool. Right? Now for the mempools which store the transactions, how is it relevant to them WHO re-broadcasted them? Their timeout is initialized when they are first seen I suppose. It won't be reset no matter how often the transactions is received again from someone else. So how does it matter that the transactions are being re-broadcast by other peers than their original senders? The timeout cannot be reset anyway.
(In addition, I see a non-privacy related bonus feature of random re-broadcast: Without doing any math, intuitively this still sounds to me as it would also cause a better spreading of the transactions over the network because it removes sender/receiver as a single point of failure. Even if they terminate their client after sending the transactions, the rest of the network will still take care of re-broadcast.)

[laanwj]

I explained why we ware doing that! Rebroadcasting your own transactions avoids them from getting stuck if your set of peers at the time was flaky. If you have another channel to send the transaction to your recipient (like the payment protocol) it becomes less important to broadcast it yourself, but we cannot assume that.

As for mempool expiration, different peers receive the transaction at slightly different times, and thus expire them at different times. Let's say Node B received a transaction sooner than node A. B is just about the expire the transaction, and then Node A rebroadcasts it randomly. B will see it as a new transaction... and after A expired it, it may be that B rebroadcasts it again. Bouncing around it will be kept in the mempool forever.
(this is with just two nodes, imagine with a whole forest of them...)

This is how it would go right now. You propose to persistently remember transaction expired from the mempool (for a certain duration) so that they don't reenter, but such functionaltiy doesn't exist at this point. I also don't think it would work. The point of expiring transactions is not to avoid them from being re-broadcasted, in the contrary, it's just to make sure transactions that the sender/receiver don't care about (by not re-broadcasting them) don't keep cluttering the mempool.

[leofidus]

@laanwj The general target for expiring a transaction from the mempool seems to be 2-3 days. If random transactions would be broadcasted for a significatnly shorter time, let's say half the expiration time, that seems to make it unlikely that another peer has already expired it when it's randomly rebroadcasted.

[leo-bogert]

@laanwj Lets discuss the two issues separately:

1. Transactions getting lost due to lack of own broadcasting: I think this is easy to prevent by having the client issue a warning when the user wants to shutdown but a transactions has not broadcast through less than lets say 4 peers. And given the fact that Bitcoin uses a flooding algorithm, i.e. every peer sends the transaction to all other peers, it should be very difficult for transactions to get lost even with such small amounts of peers. The amount of peers a transactions has been broadcast through could also be stored persistently among restarts, so the user can shutdown even if the transaction has not been sent yet. Further, it should be noticed that random re-broadcasts do also include the user's own transactions, just with a lower (=random) priority.
2. Transactions getting stuck in mempools forever: To prevent this, when sending transactions to our peers, we can include our own mempool expiration time of the transactions. The peers then adopt this as this their own expiration time for the transaction (unless its farther in the future than current_time + default_timeout would be, then the expiration time is reset to that). This would be similar to what existing networks do: For example IP packets have a "time to live" counter which is decremented by each transfer of the packet, and the packet is not sent any further when it reaches zero. In fact, this could be used as an alternate solution to sending timeouts. [Notice that the TTL field of IP was actually meant as a timeout before it was changed to be just decremented].

[leo-bogert]

Addition to 2: The expiration time must of course be modified by a random bias before sharing it, otherwise it gives away too much information about how close to the origin of the transactions the sender is.

[laanwj]

I think it makes sense to move this to the mailing list. This bug tracker is intended for concrete issues specific to this implementation ("bitcoin core"). General protocol improvements should be discussed over there, with more visibility.

[jwilkins]

Outsourcing to tor is a major problem, it creates an unexpected binding between the two. People are likely to be de-anonymized when they fire up bitcoind and tor isn't running or if the tor process ever dies while bitcoind is running or any number of other scenarios.

[Leviathn]

Is this closeable? @laanwj @vessenes

[sipa]

As long as we don't have mempool reconciliation, I think there is no real solution to this problem.

[laanwj]

At least we now have an option to disable the default broadcast: -walletbroadcast=0. This allows being creating about alternative ways to do the initial transaction broadcast. Not a structural solution, that would be in the domain of adding some kind of onion layer to the P2P protocol itself (I vaguely remember there has been a proposal in that direction, but can't find it right now), or mempool reconciliation as @sipa says.

The problem still exists so might as well keep open the issue as a reminder.

[leo-bogert]

At least we now have an option to disable the default broadcast: -walletbroadcast=0. This allows being creating about alternative ways to do the initial transaction broadcast. Not a structural solution, that would be in the domain of adding some kind of onion layer to the P2P protocol itself (I vaguely remember there has been a proposal in that direction, but can't find it right now), or mempool reconciliation as @sipa says.

I do appreciate the availability of this option but let's be honest:
Achieving privacy is a very difficult thing even in an environment fully controlled by software.
Requiring the user to manually add command line options, which 90% of users won't be capable of already, and then doing even more complex things like sending raw transactions with a different software, is a usability nightmare in that context.

The problem still exists so might as well keep open the issue as a reminder.

I would please strongly ask to keep it open - storage of issues costs you nothing, but the value of resolving this someday is very high for the large amount of users who want privacy.
If you don't want the issue to bloat your view of the bugtracker then filter by priority, it is set to low priority already anyway.

[naumenkogs]

Just randomly stumbled upon this issue.
The issue is being addressed here.

[adamjonas]

ref #21061