New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZIP 32] Shielded Hierarchical Deterministic Wallets #157

Merged
merged 52 commits into from Oct 5, 2018

Conversation

Projects
None yet
4 participants
@str4d
Copy link
Contributor

str4d commented May 30, 2018

No description provided.


- *a* || *b* means the concatenation of sequences *a* then *b*.

- [*k*] *P* means scalar multiplication in the group *P*.

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

scalar multiplication of the (elliptic curve) group element P by the scalar k.

- BLAKE2b-512(*p*, *x*) refers to unkeyed BLAKE2b-512 in sequential mode, with an output digest length of 64
bytes, 16-byte personalization string *p*, and input *x*.

- PRF\ :sup:`expand`\ (*sk, t*) := BLAKE2b-512("Zcash_ExpandSeed", *sk* || *t*)

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

Nitpick: *sk*, *t* (the comma should not be in italic :-) )

- We do not derive Sapling public keys directly, as this would prevent the use of diversified addresses.
Instead, we derive Sapling full viewing keys, from which payment addresses can be generated. This maintains
the trust semantics of BIP 32: someone with access to a BIP 32 extended public key is able to view all
transactions involving that address, which a Sapling full viewing key also enables.

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

With hindsight, it would have been useful to support an "address derivation key" which allows creating (and linking) diversified addresses but without being able to decrypt notes sent to them. I don't see a way to do this with Sapling's current key derivation structure, though. We should consider this issue for the next circuit upgrade — at which time we'll have more experience with diversified addresses.

(Aside: Sapling payment addresses were intentionally made not to be "rediversifiable" –i.e. it is not possible to create one diversified address directly from another– as part of preventing Brian Warner's attack in which a note for one address is encrypted to a different address in order to attempt to link them. But that attack does not preclude support for an address derivation key, because we would not be trying to prevent linking of addresses given this key.)

Specification: Sprout key derivation
====================================

For completeness, we define a system for deriving a tree of Sprout keys components. It is unlikely that this

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

s/key/keys/


- ``coin_type``: a constant identifying the cybercoin that this subtree's keys are used with. For
compatibility with existing BIP 44 implementations, we use the same constants as defined in SLIP 44
[#slip-0044]_.

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

Add a note that all testnets share coin_type index 1.


Unlike BIP 44, neither Sprout nor Sapling have a `change` path level. This is because shielded addresses are
never publicly visible in transactions, which means that sending change back to the originating address is
indistinguishable from using a change address.

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

This only makes sense if you already know that the motivation for using separate change addresses in Bitcoin was to attempt to improve privacy.

Wallets MUST support generating the default payment address (corresponding to the default diversifier). They
MAY also support generating a stream of payment addresses for a given account, if they wish to maintain the
user experience of giving a unique address to each recipient. Note that a given account can have a maximum of
2\ :sup:`88` payment addresses.

This comment has been minimized.

@daira

daira May 30, 2018

Contributor

We need some discussion of diversifier collisions here. If wallet implementors attempt to do this in the most obvious way, they will start to hit collisions after ~244 (or fewer, with lower probability) addresses. Should we define a PRP-based derivation method?

This comment has been minimized.

@daira

daira Jun 1, 2018

Contributor

Correction: ~243.5 (because half of diversifiers are invalid). And the maximum number is 287.

@str4d str4d force-pushed the str4d:zip-0032 branch 4 times, most recently from 18b1bc6 to 96a829b Jun 7, 2018

@daira daira changed the title ZIP 32: Shielded Hierarchical Deterministic Wallets [ZIP 32] Shielded Hierarchical Deterministic Wallets Jun 10, 2018

@str4d

This comment has been minimized.

Copy link
Contributor

str4d commented Jun 20, 2018

@daira and I had a pairing to figure out how to derive pseudo-random diversifiers without hitting the 244 birthday bound on collisions, by using a PRP. We've decided to use FFX-A2 (as being considered by NIST), with 88-bit messages and the zero-length byte string as tweak. The one additional change we are making is to use 256-bit AES keys (matching the rest of the key material); as the key size is not technically a tunable parameter within the A2 parameter set, we will refer to this as FFX-AES256-A2. EDIT: NIST standardised this in SP 800-38G as FF1-AES256.

@daira daira force-pushed the str4d:zip-0032 branch 2 times, most recently from a69ef8c to 30493b3 Jul 4, 2018

@@ -199,7 +199,7 @@ The 88-bit diversifiers for a Sapling extended key are derived from its diversif
In order to reach the maximum possible diversifier range without running into the birthday bound, we use
FF1-AES256 as a Pseudo-Random Permutation as follows:

- Let *j* be the index of the desired diversifier.
- Let *j* be the index of the desired diversifier, in the range 0 .. 2\ :sup:`88`\ -1.
- *d*\ :sub:`i,j` = FF1-AES256.Encrypt(*dk*\ :sub:`i`\ , "", I2LEBSP\ :sub:`88`\ (*j*))

The default diversifier for a Sapling extended key is defined to be *d*\ :sub:`i,0`\ .

This comment has been minimized.

@str4d

str4d Jul 5, 2018

Contributor

Hmm, we need to be clearer here: it's not simply j = 0, but the first j that results in a valid diversifier.

32 bytes, 16-byte personalization string *p*, and input *x*.

- BLAKE2b-512(*p*, *x*) refers to unkeyed BLAKE2b-512 in sequential mode, with an output digest length of
64 bytes, 16-byte personalization string *p*, and input *x*.

This comment has been minimized.

@str4d

str4d Jul 5, 2018

Contributor

Hmm, does unkeyed BLAKE2b-512 in sequential mode, with an output digest length of 64 bytes (and likewise for BLAKE2b-256) convey the correct meaning? We want to indicate that the only difference between the two is the digest length (so maybe unkeyed BLAKE2b in [...], but that it is not the same thing as truncating (so maybe we need more than output digest length).

This comment has been minimized.

@daira

daira Jul 6, 2018

Contributor

This is the same wording as in the protocol spec, and I believe it's unambiguous.

Show resolved Hide resolved zip-0032.rst
parent address tag *parent_addr_tag* and child number *i*, is represented as a
byte sequence::

I2LEOSP\ :sub:`8`\ (*depth*) || *parent_addr_tag* || I2LEOSP\ :sub:`32`\ (*i*) || *ASK* || *c*

This comment has been minimized.

@str4d

str4d Jul 5, 2018

Contributor

Put ASK after c to match the Sapling encodings.

@daira daira force-pushed the str4d:zip-0032 branch 2 times, most recently from f78fa32 to 1824b3d Jul 6, 2018

str4d and others added some commits May 30, 2018

Define I2LEOSP_l(k) and use it to encode the child key indices
Note that this means they are encoded in little-endian order, which is the
opposite of BIP 32.
ZIP 32: use FF1-AES256 as the PRP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Remove hardening from example public-key HD path
Hardened derivation is undefined for an extended FVK
Cosmetic improvements.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Reference version 2018.0-beta-21 or later of the Sapling protocol spec.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Define r_J.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Cosmetics.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Update another reference to the Sapling spec version.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Cosmetics.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Say that ZIP 32 does not supplant the use of BIPs 32 & 44 for transpa…
…rent addresses.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

@daira daira force-pushed the str4d:zip-0032 branch from 7f959cd to 813a889 Jul 25, 2018

.. [#bip-0044] `BIP 44: Multi-Account Hierarchy for Deterministic Wallets <https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki>`_
.. [#slip-0044] `SLIP 44: Registered coin types for BIP-0044 <https://github.com/satoshilabs/slips/blob/master/slip-0044.md>`_
.. [#bip-0173] `BIP 173: Base32 address format for native v0-16 witness outputs <https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki>`_
.. [#sapling-spec] `Zcash Protocol Specification, Version 2018.0-beta-21 or later [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/sapling.pdf>`_

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

update these links to protocol.pdf

This comment has been minimized.

@daira

daira Aug 30, 2018

Contributor

The filename changes weren't made until 2018.0-beta-25. But it's correct to update it to protocol.pdf if the version is changed to be later than that.

- *dk*\ :sub:`i` = truncate\ :sub:`32`\ (PRF\ :sup:`expand`\ (*I*\ :sub:`L`\ , [0x16] || *dk*\ :sub:`par`\ ))
- *c*\ :sub:`i` = *I*\ :sub:`R`

Deriving a child extended full viewing key

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

Change to "Deriving a normal child extended full viewing key from a parent extended full viewing key"?

This comment has been minimized.

@daira

daira Aug 30, 2018

Contributor

Isn't this implied? The subheading above it doesn't say "Deriving a child extended spending key from a parent extended spending key".

- *dk*\ :sub:`i` = truncate\ :sub:`32`\ (PRF\ :sup:`expand`\ (*I*\ :sub:`L`\ , [0x16] || *dk*\ :sub:`par`\ ))
- *c*\ :sub:`i` = *I*\ :sub:`R`

Diversifier derivation

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

dk seems better than dki in this section

This comment has been minimized.

@daira

daira Aug 30, 2018

Contributor

Only if di,j is also changed to dj.

Show resolved Hide resolved zip-0032.rst

m_Sapling / purpose' / coin_type' / account'

Wallets MUST support generating the default payment address (corresponding to the default diversifier). They

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

perhaps add 'as defined above' after 'default diversifier'

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

In summary to discussions, would change sentence to
"Furthermore, Wallets MUST support generating the default payment address (corresponding to the default diversifier as defined above) for any account they support".

- ToScalar(*x*) := LEOS2IP\ :sub:`512`\ (*x*) (mod *r*\ :sub:`J`\ ), where *r*\ :sub:`J` \ is the order
of the Jubjub large prime subgroup.

- DiversifyHash(*d*) maps a diversifier *d* to a base point on the Jubjub elliptic curve, or to ⊥ if the

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

perhaps add 'prime order subgroup' after 'elliptic curve'


The above key path levels include an account identifier, which in all user interfaces is represented as a
"bucket of funds" under the control of a single spending authority. Therefore, wallets implementing Sapling
ZIP 32 derivation MUST support the following path::

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

add 'for any account' in {0,..,2^31-1}' ?

- Use *I*\ :sub:`L` as the master spending key *sk*\ :sub:`m`\ , and *I*\ :sub:`R` as the master chain code
*c*\ :sub:`m`\ .
- Calculate the master extended spending key *m*\ :sub:`Sapling` = (*ask*\ :sub:`m`\ , *nsk*\ :sub:`m`\ ,
*ovk*\ :sub:`m`\ , *dk*\ :sub:`m`\ , *c*\ :sub:`m`\ ) via the standard Sapling derivation

This comment has been minimized.

@arielgabizon

arielgabizon Aug 14, 2018

Contributor

via -> similarly - cause dk isn't in the Sapling derivation

----------------------

The 88-bit diversifiers for a Sapling extended key are derived from its diversifier key *dk*\ :sub:`i`\ .
In order to reach the maximum possible diversifier range without running into the birthday bound, we use

This comment has been minimized.

@arielgabizon

arielgabizon Aug 23, 2018

Contributor

Could you elaborate about what you mean 'running into the birthday problem'? I think you mean also you want to get some independence of diversifier sequences of different base keys. If I just want no collisions, I could define d_{i,j} = j.

This comment has been minimized.

@daira

daira Sep 18, 2018

Contributor

Yes, diversifiers should follow an independent random sequence for each ivk. Otherwise, the diversifier leaks information about how many previous diversifiers were obtained from that ivk.

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

That makes sense, this should be explained though.. added two new comments with suggested phrasings.

@ebfull

ebfull approved these changes Sep 10, 2018

@ebfull

This comment has been minimized.

Copy link
Contributor

ebfull commented Sep 10, 2018

My ACK is to indicate that I approve of the design -- though I would personally have advocated against specifying a Sprout key derivation model. Also, I think it may be worth mentioning the security properties like BIP32 does. (We should indicate that using the same seed byte sequence for master key derivation in BIP32 and ZIP32 is fine, even though it may be obvious to some.)

I'm confused about how this document progresses through the ZIP process, and whether we're supposed to label this as a draft or not. Do we not merge until we have test vectors and the spec is complete?

str4d added some commits Sep 18, 2018

@@ -329,7 +329,7 @@ relevant transactions.

The above key path levels include an account identifier, which in all user interfaces is represented as a
"bucket of funds" under the control of a single spending authority. Therefore, wallets implementing Sapling
ZIP 32 derivation MUST support the following path::
ZIP 32 derivation MUST support the following path for any account in range {0..2\ :sup:`31`\ -1}::

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

account -> account'

This comment has been minimized.

@str4d

str4d Sep 18, 2018

Contributor

Account is already defined earlier up as hardened, so the meaning here is clear enough.

This comment has been minimized.

@str4d

str4d Sep 18, 2018

Contributor

(and account is in range 0..2^31-1, while account' is technically in range 2^31..2^32-1 which doesn't read as easily)

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

right.

----------------------

The 88-bit diversifiers for a Sapling extended key are derived from its diversifier key *dk*.
In order to reach the maximum possible diversifier range without running into the birthday bound, we use

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

"without running into the birthday bound," --> "without running into repetitions due to the birthday bound,".

Diversifier derivation
----------------------

The 88-bit diversifiers for a Sapling extended key are derived from its diversifier key *dk*.

This comment has been minimized.

@arielgabizon

arielgabizon Sep 18, 2018

Contributor

Add after sentence "To prevent the diversifier leaking how many diversified addresses have already been generated for an account; we make the sequence of diversifiers pseudorandom and uncorrelated to that of any other account."

arielgabizon and others added some commits Sep 20, 2018

@str4d str4d force-pushed the str4d:zip-0032 branch from d1db943 to 975a2aa Sep 20, 2018

@daira daira merged commit f6f47a0 into zcash:master Oct 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment