Added public key compression specs (#137)

## What has changed? I've added detailed specs for the implementation of public key compression and decompression. The specifications detail the use of the following `multiformat` features: - `multibase` - `multicodec` - `unsigned-varint` `multiformat` is used to ensure that the implementation has as much flexibility and robustness as feasible. ## Why make the change? The usage of key de/compression is outside the typical usage of public keys and requires a degree of background knowledge to correctly implement. The purpose of this specification change is to provide this needed background knowledge. Please also see - status-im/status-go#1937 - status-im/status-go#1990 --- * Added Public key compression specs * Added recommendation for encoding type of compressed keys * Added unrecognised words to wordlist * Add multibase to the wordlist * Added a basic example of the multiformat EC key compression concept * Added parsable to wordlist * Hex is the only Lingua Franca we need * Language to make pk de/compression SHOULD implement * Added terms to glossary explaining key de/compression * Change terminology from compress to serialise * Added rationale for public key compression * Added deserialization to the wordlist * Concise sentence * Added url to the wordlist
status-im · Jun 24, 2020 · e98a9b7 · e98a9b7
1 parent 5aa330c
commit e98a9b7
Show file tree

Hide file tree

Showing 2 changed files with 226 additions and 6 deletions.
diff --git a/docs/stable/2-account.md b/docs/stable/2-account.md
@@ -6,7 +6,7 @@ title: 2/ACCOUNT
 
 # 2/ACCOUNT
 
-> Version: 0.3
+> Version: 0.4
 > 
 > Status: Stable
 >
@@ -39,6 +39,14 @@ This specification explains what Status account is, and how a node establishes t
         - [Identicon](#identicon)
         - [3 word pseudonym / Whisper/Waku key fingerprint](#3-word-pseudonym--whisperwaku-key-fingerprint)
         - [ENS name](#ens-name)
+- [Public Key Serialization](#public-key-serialization)
+  - [Basic Serialization Example](#basic-serialization-example)
+  - [Public Key "Compression" Rationale](#public-key-compression-rationale)
+  - [Key Encoding](#key-encoding)
+  - [Public Key Types](#public-key-types)
+  - [De/Serialization Process Flow](#deserialization-process-flow)
+    - [Serialization Example](#serialization-example)
+    - [Deserialization Example](#deserialization-example)
 - [Security Considerations](#security-considerations)
 - [Changelog](#changelog)
   - [Version 0.3](#version-03)
@@ -106,11 +114,11 @@ The node verifies or derives everything else associated with the contact from th
 
 ### Terms Glossary
 
-| term             | description |
-| ---------------- | ----------- |
-| privkey          | ECDSA secp256k1 private key |
-| pubkey           | ECDSA secp256k1 public key |
-| Whisper/Waku key | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
+| term                      | description |
+| ------------------------- | ----------- |
+| privkey                   | ECDSA secp256k1 private key |
+| pubkey                    | ECDSA secp256k1 public key |
+| Whisper/Waku key          | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
 
 
 ### Contact Discovery
@@ -213,12 +221,212 @@ All messages sent are encrypted with the public key of the destination and signe
 
 -->
 
+## Public Key Serialization
+
+Idiomatically known as "public key compression" and "public key decompression".
+
+The node SHOULD provide functionality for the serialization and deserialization of public / chat keys.
+
+For maximum flexibility, when implementing this functionality, the node MUST support public keys encoded in a range of encoding formats, detailed below.
+
+### Basic Serialization Example
+
+In the example of a typical hexadecimal encoded elliptical curve (EC) public key (such as a secp256k1 pk),
+
+```text
+0x04261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
+```
+
+minor modification for compatibility and flexibility makes the key self-identifiable and easily parsable, 
+
+```text
+fe70104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
+```
+
+EC serialization and compact encoding produces a much smaller string representation of the original key.
+
+```text
+zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB
+```
+
+### Public Key "Compression" Rationale
+
+Serialized and compactly encoded ("compressed") public keys have a number of UI / UX advantages over non-serialized less densely encoded public keys.
+
+Compressed public keys are smaller, and users may perceive them as less intimidating and less unnecessarily large. Compare the "compressed" and "uncompressed" version of the same public key from above example:
+
+- `0xe70104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6`
+- `zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB`
+
+The user can transmit and share the same data, but at one third of the original size. 136 characters uncompressed vs 49 characters compressed, giving a significant character length reduction of 64%. 
+
+The user client app MAY use the compressed public keys throughout the user interface. For example in the `status-react` implementation of the user interface the following places could take advantage of a significantly smaller public key:  
+
+- `Onboarding` > `Choose a chat name`
+- `Profile` > `Header`
+- `Profile` > `Share icon` > `QR code popover`
+- `Invite friends` url from `Invite friends` button and `+ -button` > `Invite friends`
+- Other user `Profile details` 
+- `Profile details` > `Share icon` > `QR code popover`
+
+In the case of QR codes a compressed public key can reduce the complexity of the derived codes:
+
+| Uncompressed | Compressed |
+| --- | --- |
+|<img src="https://user-images.githubusercontent.com/5702426/80531063-e98fcc80-8991-11ea-8c02-c354b5828d35.png" width="400" />|<img src="https://user-images.githubusercontent.com/5702426/80501933-f4356c00-8967-11ea-87d8-eae18becece9.png" width="400"/>|
+
+
+### Key Encoding
+
+When implementing the pk de/serialization functionality, the node MUST use the [multiformats/multibase](https://github.com/multiformats/multibase) encoding protocol to interpret incoming key data and to return key data in a desired encoding.
+
+The node SHOULD support the following `multibase` encoding formats.
+
+```csv
+encoding,          code, description,                                              status
+identity,          0x00, 8-bit binary (encoder and decoder keeps data unmodified), default
+base2,             0,    binary (01010101),                                        candidate
+base8,             7,    octal,                                                    draft
+base10,            9,    decimal,                                                  draft
+base16,            f,    hexadecimal,                                              default
+base16upper,       F,    hexadecimal,                                              default
+base32hex,         v,    rfc4648 case-insensitive - no padding - highest char,     candidate
+base32hexupper,    V,    rfc4648 case-insensitive - no padding - highest char,     candidate
+base32hexpad,      t,    rfc4648 case-insensitive - with padding,                  candidate
+base32hexpadupper, T,    rfc4648 case-insensitive - with padding,                  candidate
+base32,            b,    rfc4648 case-insensitive - no padding,                    default
+base32upper,       B,    rfc4648 case-insensitive - no padding,                    default
+base32pad,         c,    rfc4648 case-insensitive - with padding,                  candidate
+base32padupper,    C,    rfc4648 case-insensitive - with padding,                  candidate
+base32z,           h,    z-base-32 (used by Tahoe-LAFS),                           draft
+base36,            k,    base36 [0-9a-z] case-insensitive - no padding,            draft
+base36upper,       K,    base36 [0-9a-z] case-insensitive - no padding,            draft
+base58btc,         z,    base58 bitcoin,                                           default
+base58flickr,      Z,    base58 flicker,                                           candidate
+base64,            m,    rfc4648 no padding,                                       default
+base64pad,         M,    rfc4648 with padding - MIME encoding,                     candidate
+base64url,         u,    rfc4648 no padding,                                       default
+base64urlpad,      U,    rfc4648 with padding,                                     default
+```
+
+**Note** this specification RECOMMENDs that implementations extend the standard `multibase` protocol to parse strings prepended with `0x` as `f` hexadecimal encoded bytes.
+
+Implementing this recommendation will allow the node to correctly interpret traditionally identified hexadecimal strings (e.g. `0x1337c0de`).
+
+*Example:*
+
+`0xe70102261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc`
+
+SHOULD be interpreted as 
+
+`fe70102261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc`
+
+This specification RECOMMENDs that the consuming service of the node uses a compact encoding type, such as base64 or base58 to allow for as short representations of the key as possible.
+
+### Public Key Types
+
+When implementing the pk de/serialization functionality, The node MUST support the [multiformats/multicodec](https://github.com/multiformats/multicodec) key type identifiers for the following public key type.
+
+| Name               | Tag | Code   | Description                          |
+| ------------------ | --- | ------ | ------------------------------------ |
+| `secp256k1-pub`    | key | `0xe7` | Secp256k1 public key                 |
+
+For a public key to be identifiable to the node the public key data MUST be prepended with the relevant [multiformats/unsigned-varint](https://github.com/multiformats/unsigned-varint) formatted code.
+
+*Example:*
+
+Below is a representation of an deserialized secp256k1 public key.
+
+```text
+04
+26 | 1c | 55 | 67 | 5e | 55 | ff | 25
+ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
+35 | f6 | 07 | 12 | d2 | 51 | cb | aa
+ab | 97 | bd | 50 | 05 | 4c | 6e | bc
+3c | d4 | e2 | 22 | 00 | c6 | 8d | af
+74 | 93 | e1 | f8 | da | 6a | 19 | 0a
+68 | a6 | 71 | e2 | d3 | 97 | 78 | 09
+61 | 24 | 24 | c7 | c3 | 88 | 8b | c6
+```
+
+The `multicodec` code for a secp256k1 public key is `0xe7`.
+
+After parsing the code `0xe7` as a `multiformats/uvarint`, the byte value is `0xe7 0x01`, prepending this to the public key results in the below representation.
+
+```text
+e7 | 01 | 04
+26 | 1c | 55 | 67 | 5e | 55 | ff | 25
+ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
+35 | f6 | 07 | 12 | d2 | 51 | cb | aa
+ab | 97 | bd | 50 | 05 | 4c | 6e | bc
+3c | d4 | e2 | 22 | 00 | c6 | 8d | af
+74 | 93 | e1 | f8 | da | 6a | 19 | 0a
+68 | a6 | 71 | e2 | d3 | 97 | 78 | 09
+61 | 24 | 24 | c7 | c3 | 88 | 8b | c6
+```
+
+### De/Serialization Process Flow
+
+When implementing the pk de/serialization functionality, the node MUST be passed a `multicodec` identified public key, of the above supported types, encoded with a valid `multibase` identifier.
+
+This specification RECOMMENDs that the node also accept an encoding type parameter to encode the output data. This provides for the case where the user requires the de/serialization key to be in a different encoding to the encoding of the given key. 
+
+#### Serialization Example
+
+A hexadecimal encoded secp256k1 public chat key typically is represented as below:
+
+```text
+0x04261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
+``` 
+
+To be properly interpreted by the node for serialization the public key MUST be prepended with the `multicodec` `uvarint` code `0xea 0x01` and encoded with a valid `multibase` encoding, therefore giving the following:
+
+```text
+fea0104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
+``` 
+
+If adhering to the specification recommendation to provide the user with an output encoding parameter, the above string would be passed to the node with the following `multibase` encoding identifier.
+
+In this example the output encoding is defined as `base58 bitcoin`.
+
+```text
+z
+``` 
+
+The return value in this case would be
+
+```text
+zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB
+```
+
+Which after `multibase` decoding can be represented in bytes as below:
+
+```text
+e7 | 01 | 02
+26 | 1c | 55 | 67 | 5e | 55 | ff | 25
+ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
+35 | f6 | 07 | 12 | d2 | 51 | cb | aa
+ab | 97 | bd | 50 | 05 | 4c | 6e | bc
+```
+
+#### Deserialization Example
+
+For the user, the deserialization process is exactly the same as serialization with the exception that the user MUST provide a serialized public key for deserialization. Else the deserialization algorithm will fail.
+
+For further guidance on the implementation of public key de/serialization consult the [`status-go` implementation and tests](https://github.com/status-im/status-go/blob/c9772325f2dca76b3504191c53313663ca2efbe5/api/utils_test.go).  
+
 ## Security Considerations
 
 -
 
 ## Changelog
 
+### Version 0.4
+
+Released // TODO
+
+- Added details of public key serialization and deserialization
+
 ### Version 0.3
 
 Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)

diff --git a/wordlist.txt b/wordlist.txt
@@ -11,6 +11,7 @@ blockable
 BlockByHash
 BlockByNumber
 blockchain
+BLS
 BundleContainer
 chainId
 Changelog
@@ -34,9 +35,13 @@ DApp
 dapp
 dapps
 DDoS
+de
+De
 decrypt
 decrypted
 decrypting
+Deserialization
+deserialization
 deserialized
 deterministically
 devp
@@ -130,7 +135,10 @@ merkle
 MessageType
 mixnets
 multiaccount
+multibase
 multicasting
+multicodec
+multiformats
 multihash
 MVDS
 myPrivateKey
@@ -141,6 +149,7 @@ oneof
 Oskar
 PairInstallation
 params
+parsable
 partitionsNum
 partitionTopic
 peerID
@@ -160,6 +169,7 @@ pre
 prekey
 prekeys
 prepend
+prepended
 prepending
 privkey
 protobuf
@@ -215,8 +225,10 @@ unix
 Upgradability
 URI
 URIs
+url
 uuid
 UX
+varint
 Volodymyr
 Volosovskyi
 Vp