Skip to content

Conversation

@apoelstra
Copy link
Member

This ELIP describes CT descriptors, as needed to attach blinding keys to existing descriptors and therefore create confidential addresses.

It does not cover:

  • Ordinary Elements descriptors, e.g. eltr; these need a separate standard preceding this one
  • Pegins or pegouts
  • Anything related to PAK keys

Copy link
Contributor

@jgriffiths jgriffiths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes:

  1. I strongly prefer to use 'private key' rather than 'secret key' for EC key pair terminology, this matches other BIPs better, and as noted here here there may be confusion with 'secret' public keys. My suggested changes use 'secret' just because thats the minimum review delta, but I think it should be globally changed so the only reference to secrecy is the notion of confusion around secret public keys.

  2. Similarly does the naming of hash_to_sk conflict with any current or proposed descriptor/miniscript operator that produces or takes private keys? I feel like hash_to_scalar or something other than sk would be a better name.

  3. There is a negligible but non-zero chance that the hash_to_sk construction will produce an invalid key, a resolution or suggested action should this occur should be listed for completeness, as done with bip32 for example, despite it being for all intents and purposes cryptographically unreachable.

This means they must be chosen by the sender of a transaction and somehow securely communicated to the recipient.

The way this is done is that confidential addresses contain the public key of an extra "blinding key" pair created by the recipient.
When blinding, the sender chooses a fresh EC Diffie-Hellman (ECDH) secret key, encodes the corresponding public key in the "nonce" field of the transaction, derives an ECDH secret between the fresh key and the public blinding key, uses the ECDH secret to encrypt the blinding factors for the output, and encodes the encrypted result in the rangeproof.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When blinding, the sender chooses a fresh EC Diffie-Hellman (ECDH) secret key, encodes the corresponding public key in the "nonce" field of the transaction, derives an ECDH secret between the fresh key and the public blinding key, uses the ECDH secret to encrypt the blinding factors for the output, and encodes the encrypted result in the rangeproof.
When blinding, the sender chooses a fresh EC Diffie-Hellman (ECDH) secret key, encodes the corresponding public key in the "nonce" field of the transaction, derives an ECDH secret between the fresh secret key and the public blinding key, uses the ECDH secret to encrypt the blinding factors for the output, and encodes the encrypted result in the rangeproof.


The way this is done is that confidential addresses contain the public key of an extra "blinding key" pair created by the recipient.
When blinding, the sender chooses a fresh EC Diffie-Hellman (ECDH) secret key, encodes the corresponding public key in the "nonce" field of the transaction, derives an ECDH secret between the fresh key and the public blinding key, uses the ECDH secret to encrypt the blinding factors for the output, and encodes the encrypted result in the rangeproof.
The recipient, when recognizing a scriptPubKey corresponding to the ordinary part of her confidential address, uses the blinding key for that address in conjunction with the nonce field of the txout to re-derive the ECDH secret and decrypt the blinding factors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The recipient, when recognizing a scriptPubKey corresponding to the ordinary part of her confidential address, uses the blinding key for that address in conjunction with the nonce field of the txout to re-derive the ECDH secret and decrypt the blinding factors.
The recipient, when recognizing a scriptPubKey corresponding to the ordinary part of her confidential address, uses the secret blinding key for that address in conjunction with the nonce field of the txout to re-derive the ECDH secret and decrypt the blinding factors.

When blinding, the sender chooses a fresh EC Diffie-Hellman (ECDH) secret key, encodes the corresponding public key in the "nonce" field of the transaction, derives an ECDH secret between the fresh key and the public blinding key, uses the ECDH secret to encrypt the blinding factors for the output, and encodes the encrypted result in the rangeproof.
The recipient, when recognizing a scriptPubKey corresponding to the ordinary part of her confidential address, uses the blinding key for that address in conjunction with the nonce field of the txout to re-derive the ECDH secret and decrypt the blinding factors.

The purpose of this document is to define a standard way for the recipient's wallet to compute blinding key pairs. There are a number of requirements:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The purpose of this document is to define a standard way for the recipient's wallet to compute blinding key pairs. There are a number of requirements:
This document defines a standard way for the recipient's wallet to compute blinding key pairs. There are a number of requirements:


The purpose of this document is to define a standard way for the recipient's wallet to compute blinding key pairs. There are a number of requirements:

* Confidential addresses, including blinding key, should be derivable from a single descriptor without extra data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Confidential addresses, including blinding key, should be derivable from a single descriptor without extra data
* Confidential addresses, including their attached blinding key, should be derivable from a single descriptor without extra data

The purpose of this document is to define a standard way for the recipient's wallet to compute blinding key pairs. There are a number of requirements:

* Confidential addresses, including blinding key, should be derivable from a single descriptor without extra data
* Wallets should be able to choose the granularity of their blinding keys, so that the revelation of private blinding keys may unblind one, a subset of, or all, its blinded outputs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Wallets should be able to choose the granularity of their blinding keys, so that the revelation of private blinding keys may unblind one, a subset of, or all, its blinded outputs
* Wallets should be able to choose the granularity of their blinding keys, so that the revelation of secret blinding keys may unblind one, a subset, or all of its blinded outputs

* `hash_to_sk(<key1>, <key2>, ...)` which indicates that the blinding key is produced by a sorted hash of the listed key(s), which will be one or more public keys; or
* `<key>` which is an ordinary (extended) public key which will be used as a blinding key

We note that even if the descriptor includes secret keys, as it will when used for wallet backups, the keys included in the `hash_to_sk` combinator will be public keys.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
We note that even if the descriptor includes secret keys, as it will when used for wallet backups, the keys included in the `hash_to_sk` combinator will be public keys.
We note that even if the descriptor includes secret keys, as it will when used for wallet backups, the keys included in the `hash_to_sk` combinator must be public keys.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be clearer IMO, below it states '''hash_to_sk''' takes a list of public or private keys. I take it the meaning is that the hashed value is computed on the public keys, and its the public keys that are sorted before being hashed. Would be good to be explicit/consistent between here and the hash_to_sk description below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the meaning is that the hash is computed using the public keys.

It would be quite difficult for implementors to force public keys in the actual descriptor even when they have private keys elsewhere, since they (probably) have only a single key type. So saying that they must be public keys is wrong. What I mean to say is that they will be coerced into secp256k1 public keys, in the same way that they would be if they were included in a script, and then hashed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to be explicit/consistent between here and the hash_to_sk description below.

Agree. Do you have a suggestion?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps something like:

Note that if the descriptor includes secret keys, as it will when used for wallet backups, the `hash_to_sk` combinator will convert them to public keys internally before sorting and computing its hash result.

?

===Drawbacks===

Our scheme uses derived "public" keys (i.e., EC points) to compute secret blinding keys, because
* many hardware wallets are incapable of producing rangeproofs, which require significant memory and processing power, but they will not reveal secret keys to the host computer, only public keys;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* many hardware wallets are incapable of producing rangeproofs, which require significant memory and processing power, but they will not reveal secret keys to the host computer, only public keys;
* Many hardware wallets are incapable of producing rangeproofs, which require significant memory and processing power, but they will not reveal secret keys to the host computer, only public keys;


Our scheme uses derived "public" keys (i.e., EC points) to compute secret blinding keys, because
* many hardware wallets are incapable of producing rangeproofs, which require significant memory and processing power, but they will not reveal secret keys to the host computer, only public keys;
* public keys alone cannot be used for multiparty addresses, since we require that each individual participant be able to unblind confidential outputs sent to a multiparty address.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* public keys alone cannot be used for multiparty addresses, since we require that each individual participant be able to unblind confidential outputs sent to a multiparty address.
* Public keys alone cannot be used for multiparty addresses, since we require that each individual participant be able to unblind confidential outputs sent to a multiparty address.


`<key>` is one of the following three options
* '''slip77''', encoded as `slip77(<64 hex characters>)`, whose semantics are that the 64 hex characters are interpreted as the 32-byte `master_blinding_key` in SLIP77; the `scriptPubKey` for SLIP77 is computed as normal from the ordinary descriptor.
For both private and public descriptors, these 32 bytes are encoded identically.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be clearer IMO, they are 'encoded identically' because they are the same key no?. Maybe something like "both private and public descriptors encode the same master_blinding_key bytes, since the SLIP77 key is entropy rather than an EC key pair." ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, thanks! That is a useful clarification especially since for the non-SLIP77 cases, there is a meaningful distinction between private and public keys.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to `These 32 bytes will be the same for both private and public descriptors; they have no "public" equivalent.'

    * every public key, in 33-byte compressed encoding, concatenated in lexicographic order
The output of this hash, when interpreted as an integer (most significant byte first) modulo the group order, is the blinding ''secret'' key.
* '''bare key''', which simply encodes a single key in the same way as the keys in `ordinary descriptor` are encoded.
This may be a private or public key; it is used as the blinding key for the address. 
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This may be a private or public key; it is used as the blinding key for the address. 
This may be a private or public key; its public key is used as the blinding key for the address. 

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm @apoelstra wrote that part but as I understand it, it should be possible to specify a private blinding key, too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to capture here that the confidential address only includes the public blinding key. If a private key is given, the address generated will include the public key corresponding to the private key.

@apoelstra
Copy link
Member Author

Thanks @jgriffiths for your detailed review! I will accept all your suggestiotns, except to change secret to private even in the places that you used "secret".

Regarding the name hash_to_sk, what if I used hash_to_private? I don't want to use scalar because I'd like to indicate to the user that the output of this function is private key data, and that anyone who is able to compute it (i.e. anyone with access to the descriptor) can unblind outputs.

@jgriffiths
Copy link
Contributor

jgriffiths commented Oct 7, 2022

Also missing I think are a reference implementations section listing the elements-miniscript impl (until there are more impls), and a set of test vectors.

For test vectors for hash_to_private I think a mix of public and private keys out of public-key-order would be good, my preference is the input string and expected output private key encoded in WIF. Perhaps we want to specify that all hash_to_private keys must belong to the same network and have an invalid test vector which mixes them. Also an invalid vector with an input that is not a public/private key and with an empty list.

Copy link

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Suggested a minor rewrite to be consistent how it's written in descriptors.md

Here `<ordinary descriptor>` refers to any existing descriptor, e.g. `wsh(...)` or `tr(...)` and `<key>` refers to a special construction which must be one of:
* `slip77(<master secret>)` which indicates that blinding keys for these addresses are derived via SLIP77; or
* `hash_to_sk(<key1>, <key2>, ...)` which indicates that the blinding key is produced by a sorted hash of the listed key(s), which will be one or more public keys; or
* `<key>` which is an ordinary (extended) public key which will be used as a blinding key

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should rename <key> to BLINDING_KEY (note the caps). KEY expressions are well-defined in descriptors spec.. In the descriptor spec, all the expressions are using the CAPS so it would be good to be consistent.

We can then write:

`BLINDING_KEY` expressions are either:
- slip77(<`64-byte-hex`>)
- hash_to_sk(<`KEY_1`>, <`KEY_2`>, <`KEY_3`>.... <`KEY_n`>)
- `KEY`

while referring to the key as defined in the spec.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should cover all cases with keys xpubs etc. If required, we should prohibit uncompressed keys here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My current reference logic just silently compresses uncompressed keys. Probably we should forbid them instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are voting, I vote for disallowing please.

@real-or-random
Copy link

real-or-random commented Oct 7, 2022

1. I strongly prefer to use 'private key' rather than 'secret key' for EC key pair terminology, this matches other BIPs better, and as noted here here there may be confusion with 'secret' public keys. My suggested changes use 'secret' just because thats the minimum review delta, but I think it should be globally changed so the only reference to secrecy is the notion of confusion around secret public keys.

I don't agree but I don't have a strong opinion here. I think "secret" and "private" are interchangeable and both are good. It just should be consistent. libsecp256k1 uses "secret", and "secret" has the tiny advantage that you can write sk and pk.

2. Similarly does the naming of `hash_to_sk` conflict with any current or proposed descriptor/miniscript operator that produces or takes private keys? I feel like `hash_to_scalar` or something other than `sk` would be a better name.

I think it the name should make clear that the output is secret (or private). Or do you see a change that this is reused in a context where the result is not secret?

3. There is a negligible but non-zero chance that the `hash_to_sk` construction will produce an invalid key, a resolution or suggested action should this occur should be listed for completeness, as done with bip32 for example, despite it being for all intents and purposes cryptographically unreachable.

Actually, what's a valid secret blinding key? The same as a signing key, so 0 is invalid? Or is 0 actually allowed?

Assuming that 0 is the only scalar which is an invalid secret blinding key, this happens when the hash is 0 or the group order, so the probability of this happening is 2/2^-256 = 2^-255 when SHA256 is modeled as a RO. I'm not strictly against specifying what to do in these cases but discussion about this tend to be a waste of time, as this simply won't happen in practice. A good choice is aborting or incrementing by 1.

@real-or-random
Copy link

Assuming that 0 is the only scalar which is an invalid secret blinding key, this happens when the hash is 0 or the group order, so the probability of this happening is 2/2^-256 = 2^-255 when SHA256 is modeled as a RO. I'm not strictly against specifying what to do in these cases but discussion about this tend to be a waste of time, as this simply won't happen in practice. A good choice is aborting or incrementing by 1.

Actually the draft implementation does not match the spec, which says take the hash mod the group order. But we can also change the spec to simply forbid every hash which is too large. None of this really matters in the end.

@jgriffiths
Copy link
Contributor

jgriffiths commented Oct 7, 2022

I missed the hash being taken mod n, if thats the case then then no further action is needed except to state that 0 is invalid.

@apoelstra
Copy link
Member Author

  • changed "secret key" to "private key" everywhere on account of Jon caring the most
  • added a "if this hash is zero the resultineg descriptor is invalid" sentence
  • rename hash_to_sk to hash_to_private
  • addressed all other comments

At this point the remaining TODOs are

  • Update reference impl to use hash_to_private (and address comments on that PR)
  • Generate test vectors from the reference impl

@apoelstra
Copy link
Member Author

I don't know what's wrong with github. It shows new commits on my branch but they aren't appearing on this PR.

@apoelstra
Copy link
Member Author

I have a couple questions:

  • when serializing descriptors as strings, should the keys be sorted at all? should we require they be sorted when parsing?
  • should two descriptors compare equal if they differ only in the order of hash_to_private keys?

Bear in mind that the keys in the descriptor may be xpubs, potentially with wildcards, and therefore will not sort the same way as the EC keys that get hashed.

@jgriffiths
Copy link
Contributor

jgriffiths commented Oct 20, 2022

* when serializing descriptors as strings, should the keys be sorted at all? should we require they be sorted when parsing?
* should two descriptors compare equal if they differ only in the order of `hash_to_private` keys?

As you note, there is the question of sorted in what sense - the expressions or the resulting pubkeys. With core descriptors you don't need to parse it or do EC math to produce and verify a checksum. You could require that the list items are sorted lexicographically, but it doesn't add much since you still have to sort the pubkeys when processing.

Really, we missed the boat here in Bitcoin core because the checksum is produced on the input text and not a canonical representation. e.g. sortedmulti does not require that the items are sorted, and it produces different checksums for different orders. Even accepting different hardening characters h and ' produce different checksums, and that misfeature can be used to swap the sort order of arbitrary key expressions if they share a common root path with a hardened element.

This is frustrating, but that's what we have to work with. Given this, I suspect its not worth trying to make elements specific operators non-malliable, since you'd also have to fix core malliabliity to make generalised expressions non-malliable, and that would result in most descriptor checksums mismatching between core and Elements which doesn't feel like a useful state to end up in.

@real-or-random
Copy link

I don't know what's wrong with github. It shows new commits on my branch but they aren't appearing on this PR.

I think your fork doesn't even exist anymore (or it's private).

@apoelstra
Copy link
Member Author

Ok @jgriffiths, makes sense. I won't try enforcing any sort order (or making any internal operations sort-order-independent).

For what it's worth, it's not that we "missed the boat", but that we simply could not come up with a canonical representation. ' vs h, sort order, etc., is easy enough, but deciding when xpubs of different length are equivalent was not. (You can fingerprint the final derived path, but not when wildcards are present, and we technically support hardened wildcards and post-wildcard hardened steps which makes "derive at some fixed index" impossible, and so on ... and this is not even touching issues with taproot/partial descriptors that we knew we'd eventually have to deal with).

@apoelstra
Copy link
Member Author

Good catch @real-or-random. I think this repo was original private by accident, and changing the visibility of the two forks independently has caused some github bug to appear. I fixed the visibility on my fork but that doesn't seem to have helped.

@apoelstra apoelstra closed this by deleting the head repository Oct 20, 2022
@apoelstra
Copy link
Member Author

Neat, apparently Github does allow me to take actions that will permanently auto-close the PR without warning, it just prevents me from doing anything useful.

Moving to #2, sorry for the confusion.

@jgriffiths
Copy link
Contributor

For what it's worth, it's not that we "missed the boat", but that we simply could not come up with a canonical representation.

Yes, I think you can have a canonical representation only for a given collection of conditions excluding the actual keys. That allows you to say things like 'these are bip44 compatible' or 'this is a BOLT-standard spending path' but not compare two expressions where say one is for public consumption and the other is used by the signer.

apoelstra added a commit to ElementsProject/elements-miniscript that referenced this pull request Aug 23, 2023
725f450 confidential: hack in view descriptor support (Andrew Poelstra)
6359f03 confidential: switch test vectors to use xpubs rather than bare keys (Andrew Poelstra)
a332208 re-export confidential::Descirptor as ConfidentialDescirptor (Andrew Poelstra)
694628f CT descriptors: add fuzz test (Andrew Poelstra)
2b5bd55 CT descriptors: generate ELIP test vectors (Andrew Poelstra)
604c7e3 CT descriptors: add test vectors (Andrew Poelstra)
e9ac0bc CT descriptors: add `bare` module (Andrew Poelstra)
f5b2fe5 CT descriptors: add `slip77` module (Andrew Poelstra)
277b1a5 CT descriptors: block in module (Andrew Poelstra)
450d4d1 expression: stop special-casing the text `slip77` (Andrew Poelstra)
5d2755d delete descriptor::Blinded (Andrew Poelstra)
67e9422 descriptor: expose checksum function to whole crate (Andrew Poelstra)
b11c818 descriptor: fix Display impl for legacy CSFS covenants (Andrew Poelstra)

Pull request description:

  I'm aware that there are already a basic form of CT descriptors in this library, and this does not remove them, it just adds a new module which should act as a reference implementation for ElementsProject/ELIPs#1

  WIP because it needs a new rust-elements release to get tagged hashes in `bitcoin_hashes`.

ACKs for top commit:
  sanket1729:
    ACK 725f450.

Tree-SHA512: 52d5961e653f19a6378f6abfbf812cedfb295e56cc08ea6ab8c80cdd5c0ee3b8da209f7c65c87193bf42448c0d5db86ead27dbe0bcb1e245f6c3f0b9ccce0f11
apoelstra added a commit that referenced this pull request Aug 23, 2023
5726891 rename CT descriptor ELIP to ELIP 150 (Andrew Poelstra)
ee87671 add non-xkey test vectors (Andrew Poelstra)
bb771e5 specify KEY expressions more clearl (Andrew Poelstra)
530cbf4 ct descriptors: add test vectors (Andrew Poelstra)
ad50681 typos (Andrew Poelstra)
612a554 CT descriptors: drop `hash_to_priv`, update bare keys to p2c to the spk (Andrew Poelstra)
717f856 prefix existing descriptor names with `el` (Andrew Poelstra)
44805cd change "concrete" to "derived" and clarify use (Andrew Poelstra)
468223b fix nested list formatting (use mw format not md) (Andrew Poelstra)
979a2ae search-and-replace backticks with <code>..</code> (Andrew Poelstra)
5cdd1a5 forbid private keys in CT ELIP (Andrew Poelstra)
1c87825 initial draft of CT descriptor ELIP (Andrew Poelstra)

Pull request description:

  This ELIP describes CT descriptors, as needed to attach blinding keys to existing descriptors and therefore create confidential addresses.

  It does not cover:

  * Ordinary Elements descriptors, e.g. eltr; these need a separate standard preceding this one
  * Pegins or pegouts
  * Anything related to PAK keys

  Dupliicate of #1 created because of Github bugs.

ACKs for top commit:
  sanket1729:
    ACK 5726891 . One small missing thing is that test vectors don't mention the address was created for elements regtest network.

Tree-SHA512: 910792834c53e1aa993424eb403cd83d3b5d7d9c7270a5d1213803966cde4b2505875923ce71993a6e12ae5ba0513db7176e8ede395131cfe69ca0697efd48ab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants