Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New BIP 351: Private Payments #1349

Merged

Conversation

alfred-hodler
Copy link
Contributor

I am submitting a new BIP following a discussion on the mailing list.

Link: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-July/020812.html

@clarkmoody
Copy link
Contributor

@luke-jr @kallewoof We're ready for a BIP number assignment.

{| class="wikitable"
! Address Type !! Flag !! Flag Value !! Ordinal Value
|-
| P2PKH || <code>1 << 0</code> || <code>0x0001</code> || 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Gleaning from the fact that BIP32 requires it and the payment code also requires it, most people will assume that this is a P2PKH of the compressed key.

However, in past BIPs I have seen many people completely miss things that aren't explicitly spelled out for them, so I would feel better if "compressed" was somewhere around here (a note or something)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe P2PKH<sub>compressed</sub>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is actually not possible to have an uncompressed key at any point because BIP32 is utilized, which works exclusively with compressed keys.

But I agree that we can make this more explicit in order to prevent confusion 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been addressed.

* The BIP uses a notification mechanism that relies on publicly known per-recipient notification addresses. If Alice wants to send funds to Bob, she has to use the same notification address that everyone else uses to notify Bob. If Alice is not careful with coin selection, i.e. ensuring that her notification UTXO is not linked to her, she will publicly expose herself as someone who is trying to send funds to Bob and their relationship becomes permanently visible on the blockchain.

* The BIP does not say anything about address types. Receiving wallets therefore have to watch all address types that can be created from a single public key. Even then, a sender could send to a script that a receipient cannot spend from.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about sharing a BOLT12?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you expand on this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the drawbacks of BOLT12, benefits of this over BOLT12, etc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that BOLT12 is specific to Lightning and lives one layer above this, hence I didn't consider it worthwhile to try to compare it with static addresses and payment codes, which all work on-chain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it is debatable in the case of static addresses, but payment codes are definitely a layer above the chain.

Regardless, they serve the same goals of the BIP, so IMO should be addressed here.


===Public Key Derivation Path===

The derivation path for this BIP follows BIP44. The following BIP32 path levels are defined:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't it be derivation-neutral so non-BIP44 wallets can use it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can still use it, they just have to follow the prescribed BIP32 derivation path. It's simply a matter of convention for the sake of wallet compatibility.

We could relax this requirement by stating that one can start the process with any extended private key, but then each wallet may end up implementing its own incompatible derivation path that other wallets won't know how to import (see Electrum derivation path mess).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wallets are not intended to be compatible between different software.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that BIP39 and BIP44/49/84 exist and are ubiquitous attests to the fact that wallets indeed try to keep a degree of compatibility.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're not ubiquitous, and even wallets that do use them, do not claim to be compatible with others that do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Survey of wallet derivation paths for reference: https://walletsrecovery.org/


===Notifications===

Notifications are performed by publishing transactions that contain a 40-byte <code>OP_RETURN</code> output. The value of the <code>OP_RETURN</code> is constructed using the following formula:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought part of the point of this BIP was to avoid such spam?

Why can't this stay out of band?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notifications can indeed be performed out of band (the original BIP was trying to use Bitmessage), but then we lose the ability to deterministically restore a wallet from seed simply by rescanning the blockchain.

So if we were to get rid of OP_RETURNs, we'd either need a centralized directory of notifications that's vulnerable to takedowns, or each user would have to back up their set of notifications (losing it would mean losing funds forever).

I tried my best to cut the size of these notifications from 80 bytes in BIP47 to half that amount.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This older draft didn't require notifications at all?

https://gist.github.com/RubenSomsen/c43b79517e7cb701ebf77eec6dbb46b8

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silent Payments is a different standard by different people that I had no part in making. It requires a full node and doesn't work with light clients such as mobile wallets.

Private Payments requires these notifications in order to be compatible with light wallets.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm showing OP_RETURN at 2.8 GB right now, which is 0.6% of the total chain size and is prunable if full nodes need that space.

The history of total OP_RETURN size looks like a very large user came online at one point but then stopped after a while.

nulldata-totals-graph

This protocol does not need to continually place anchors into the chain like other protocols, so the usage should have much less impact.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bitcoin's blockchain is not for storing private backups, or distributing information between users.

Instead of lazily abusing OP_RETURN, it would be preferable to have an external central directory, but there's no reason the directory couldn't be p2p either. It doesn't require any of the properties of the blockchain (double spend protection).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expanded on out of band notifications.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Far better to just require a full node, than to spam Bitcoin for the benefit of light wallets (ie, non-Bitcoin users).

(FWIW, I would discourage implementation of this BIP it it keeps OP_RETURN abuse.)

@alfred-hodler
Copy link
Contributor Author

Feedback is always welcome but ideally we'd get a BIP number at this point because it'll change the test vectors. Not having one also blocks us from publishing the reference implementation.

@luke-jr
Copy link
Member

luke-jr commented Aug 18, 2022

For a number assignment, this needs a section addressing backward compatibility.

- fix out of range indexing
- add a note about compressed keys
- add a note about of-of-band notifications
- add a backward compatibility section
@alfred-hodler
Copy link
Contributor Author

Backward compatibility has been addressed.


===Address Types===

Address type flags determine which address types a payment code accepts. This is represented by big-endian ordered 16 bits. For instance, a hypothetical payment code that handles all address types will have all defined bits set to 1 (<code>0xffff</code>).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "payment code" here is the address. Script templates and scripts are not addresses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking you're correct, but "address type" is a commonly used and understood term. Calling "p2wpkh" an address type is easier to understand than calling it a "script template". Example from a prominent library: https://docs.rs/bitcoin/latest/bitcoin/util/address/enum.AddressType.html

@luke-jr luke-jr changed the title New BIP: Private Payments New BIP 351: Private Payments Aug 19, 2022
@luke-jr
Copy link
Member

luke-jr commented Aug 19, 2022

Assigned BIP 351

@alfred-hodler
Copy link
Contributor Author

Added a link to reference implementation.

@clarkmoody
Copy link
Contributor

It looks like this is ready for merge.

@alfred-hodler
Copy link
Contributor Author

@luke-jr we are ready for merge. We'll keep the draft status for now.

@michaelfolkson
Copy link
Contributor

I'm not sure if @RubenSomsen's feedback in his mailing list response has been addressed. He is listed as a co-author so just checking he's happy with this.

@RubenSomsen
Copy link

Briefly, my thoughts:

  • I think the Prague discussion and its authors deserve mentioning, seeing as the protocol is the same except for one change (blinding the payment code)

  • I'm not convinced this was a good change, given that it introduces a scanning requirement (albeit more modest than Silent Payments) which complicates light client integration, though on the other hand it does remove the complexity of outsourcing the on-chain notification

  • I don't think any of this matters for BIP assignment

@alfred-hodler
Copy link
Contributor Author

Added a reference to the Prague discussion.

@clarkmoody
Copy link
Contributor

ACK bffc84f

@junderw
Copy link
Contributor

junderw commented Sep 8, 2022

I'm not convinced this was a good change, given that it introduces a scanning requirement

I agree here. This definitely does not prevent this becoming a BIP... but the reason why Dark Wallet's stealth address protocol and BIP47 never really caught on was the various scanning requirements increasing the implementation difficulty.

Not saying that there's an "easily implemented alternative"... they all have their pros and cons from a wallet dev perspective.

ACK bffc84f

@clarkmoody
Copy link
Contributor

clarkmoody commented Sep 8, 2022

scanning requirement

The required data is the OP_RETURN payload, which can be outsourced in a privacy-preserving way to the service described in Appendix B. Such a service would publish bulk OP_RETURNs matching the format described here, and wallets would scan against them (ECDH + SHA256 for each) to find matches and discover the stealth addresses. The reference implementation function detect_notification helps here as well.

So this is a lighter weight alternative to Silent Payments (no full node required) and a more private approach than BIP47 (no reused address).

@RubenSomsen
Copy link

The required data is the OP_RETURN payload, which can be outsourced in a privacy-preserving way [...] this is a lighter weight alternative to Silent Payments (no full node required)

Note that a very similar mitigation strategy (with a few more steps) exists for Silent Payments. Once every e.g. month or so you'd perform a scan over wifi by receiving a list with one pubkey per as-of-yet unspent transaction (fully spent transactions can be pruned). Senders are expected to send an out-of-band notification of a payment (e.g. email), allowing you to detect the payment immediately, so the monthly scanning only serves as a backup strategy for failed notifications.

While this is a theoretically sound light client implementation, the complexity involved makes me hesitant to make the strong claim that Silent Payments are suitable for light clients.

It's also important to note that these non-interactive key generation protocols (all of them) only make full sense for wallets that don't hand their xpub to a server (i.e. you'd need compact block filter support). If you do hand out the xpub, the server can do the scanning for you, or even better, they can hand out fresh keys for you.

this is a lighter weight alternative to Silent Payments (no full node required) and a more private approach than BIP47

Perhaps, but you should also compare it to the Prague protocol, and this is where I think it compares less favorably. In comparison, the Prague protocol basically trades off the scanning complexity for outsourced notifications. You send a 32 byte key + a 4 byte (non-blinded) identifier to an outsourcer and they publish it on-chain on your behalf. This ensures the sender's inputs won't be correlated with the recipient. The recipient only needs to monitor keys that are associated with their identifier (no scanning).

And as I mentioned on bitcoin-dev, I think it should also be compared to this protocol by @RobinLinus. See the bitcoin-dev post for my description of its benefits.

@clarkmoody
Copy link
Contributor

It's also important to note that these non-interactive key generation protocols (all of them) only make full sense for wallets that don't hand their xpub to a server (i.e. you'd need compact block filter support). If you do hand out the xpub, the server can do the scanning for you, or even better, they can hand out fresh keys for you.

Agreed. The assumption for all of these proposals is that we are attempting to have good privacy without having to run a server. Other proposals like Lightning Address and those that use the .well-known path on servers are able to achieve great privacy once a server enters into the mix.

Perhaps, but you should also compare it to the Prague protocol, and this is where I think it compares less favorably. In comparison, the Prague protocol basically trades off the scanning complexity for outsourced notifications. You send a 32 byte key + a 4 byte (non-blinded) identifier to an outsourcer and they publish it on-chain on your behalf. This ensures the sender's inputs won't be correlated with the recipient. The recipient only needs to monitor keys that are associated with their identifier (no scanning).

Such a notification service is also described in Appendix C, but it only helps to break the wallet clustering analysis around notification. This BIP blinds the recipient key, which is a strict privacy improvement over the Prague Protocol. Given that every light wallet will need to source OP_RETURN data from somewhere (anyone want to write a new block filter for that data - BIP159? 😄 ), a light scanning requirement on a few chunks of data each block should not be too much of a burden, especially for the privacy benefit.

And as I mentioned on bitcoin-dev, I think it should also be compared to this protocol by @RobinLinus. See the bitcoin-dev post for my description of its benefits.

That is also an interesting idea, but it's not really a spec that can be scrutinized at the level of this BIP.

Overall, I don't think we need to include an exhaustive comparison to all other proposals & protocols into this BIP in order to merge. That sort of analysis should be left up to mailing list posts or blog posts that help wallets decide which protocols to implement. Indeed, we are not required to amend previous BIPs in light of newer proposals.

@RubenSomsen
Copy link

strict privacy improvement over the Prague Protocol

This seems incorrect to me. The Prague protocol doesn't leak any meaningful information because the sender pubkey is completely fresh and unlinkable. All that is revealed is that some completely anonymous person probably intends to pay a known person (or pseudonym) at some point in the future, which is fairly harmless information. The end result is the same, but the Prague protocol replaces scanning with outsourcing.

While this is not in the write-up, in Prague I even argued that it might be acceptable for the sender pubkey to be known as well (which could save more bytes) because anyone could have published the notification (i.e. Carol could link Alice and Bob without their consent), meaning there is full plausible deniability.

a new block filter for that data

I'm not sure if you mean this BIP or the Prague protocol, but afaict in neither case a block filter is required. You just need the OP_RETURN data that is relevant to you, which is every sender/recipient key pair in the case of this BIP and only the sender keys of your specific recipient key in the case of the Prague protocol.

interesting idea, but it's not really a spec that can be scrutinized at the level of this BIP

Yes, in my opinion ideally it should have been scrutinized before this BIP was written (i.e. when I made the post on bitcoin-dev), but we're past that point now.

I don't think we need to include an exhaustive comparison to all other proposals & protocols into this BIP in order to merge

I agree, it's not relevant to the question of whether this should be merged. I am merely responding to statements that seemed incomplete/inaccurate to me.

@michaelfolkson
Copy link
Contributor

You're still happy to be a co-author on this BIP though @RubenSomsen despite the above disagreements? An alternative would be you are listed in the acknowledgements rather than as a co-author if you think this is heading in a direction where you disagree with some of what is included in the BIP. Ideally we'd avoid future disputes between co-authors on a BIP if we can.

@RubenSomsen
Copy link

An alternative would be you are listed in the acknowledgements

That makes more sense to me as well. On bitcoin-dev I suggested that the authors of the Prague protocol (namely myself, @afilini, and @Kixunil) be acknowledged, since it seems to have served as an inspiration (as well as Silent Payments). In response I was made co-author.

@clarkmoody
Copy link
Contributor

ACK aa4f030

@alfred-hodler
Copy link
Contributor Author

@luke-jr this BIP is completed, we have nothing to add to it.

@luke-jr luke-jr merged commit cf42008 into bitcoin:master Sep 29, 2022
@luke-jr
Copy link
Member

luke-jr commented Sep 29, 2022

Merged.

Nevertheless, please do address the review comments. I would discourage implementation of anything with on-chain signalling.

@clarkmoody
Copy link
Contributor

Nevertheless, please do address the review comments.

Reading back through the comments, it seems like the feedback that remains to be addressed is:

  • What are the drawbacks of BOLT12, benefits of this over BOLT12, etc?
  • Objection to using OP_RETURN
  • Some points from this mailing list post

@Pantamis
Copy link

Pantamis commented Oct 5, 2022

Is there something preventing Alice to add an output in the notification transaction so that she can pay Bob at the same time she notifies him ? (by sending her first payment to P + H(S|0)G in the notification transaction)

I think it's worth specifying in the BIP whether this is possible or not.

@clarkmoody
Copy link
Contributor

Is there something preventing Alice to add an output in the notification transaction so that she can pay Bob at the same time she notifies him ? (by sending her first payment to P + H(S|0)G in the notification transaction)

I think it's worth specifying in the BIP whether this is possible or not.

There is nothing preventing this behavior, but I would recommend against it since it leaks information about the stealth address. I agree that an additional section on UTXO handling and notification would be helpful. Appendix C describes a notification service that would completely de-link the two parties (while trusting the third a bit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
9 participants