Added public key compression specs #137

Samyoul · 2020-06-17T16:40:25Z

What has changed?

I've added detailed specs for the implementation of public key compression and decompression. The specifications detail the use of the following multiformat features:

multibase
multicodec
unsigned-varint

multiformat is used to ensure that the implementation has as much flexibility and robustness as feasible.

Why make the change?

The usage of key de/compression is outside the typical usage of public keys and requires a degree of background knowledge to correctly implement. The purpose of this specification change is to provide this needed background knowledge.

Please also see

docs/stable/2-account.md

oskarth · 2020-06-18T03:20:05Z

Thanks for the spec PR! Is there any chance we could people from Core (or more broadly, Status client implementers like desktop) more involved in these spec reviews as well? Currently I only see @cammellos as part of Core on reviews, where the rest are part of Nimbus/Vac.

cc @andremedeiros @hesterbruikman @iurimatias

arnetheduck · 2020-06-18T07:08:14Z

There's a few things I don't understand here:

where do these keys expected to appear? ie are they a pure UX feature or do the keys appear on the wire?
what are the BLS keys used for
why both G1/G2 field variants?

arnetheduck · 2020-06-18T07:34:45Z

also, where did the BLS serializations come from? the process of standardizing them has not settled yet cc @mratsim

oskarth

Some questions

docs/stable/2-account.md

oskarth · 2020-06-18T10:12:13Z

docs/stable/2-account.md

+
+Released // TODO
+
+- Added details of public key compression and decompression


The way the spec is currently writing this is currently a breaking change

I'm going rewrite the language to make public key de/compression implementation a SHOULD and make all MUSTs conditional on if de/compression functionality is being implemented.

Samyoul · 2020-06-18T11:40:53Z

@arnetheduck

where do these keys expected to appear? ie are they a pure UX feature or do the keys appear on the wire?

Yes, this functionality is (currently) purely for UX purposes, see my comment here status-im/status-go#1937 (comment) and the issue on status-im/status-mobile#10325

what are the BLS keys used for

why both G1/G2 field variants?

In my implementation I added BLS key handling for the sake of providing functionality that was simple to implement and potentially useful in the future. But on reflection I can't justify the mandatory use of BLS keys in the spec. I'll make them a RECOMMENDATION in the case that there are UX reasons for using BLS keys.

also, where did the BLS serializations come from? the process of standardizing them has not settled yet

The BLS serialisation came from the zkcrypto spec https://github.com/zkcrypto/pairing/tree/master/src/bls12_381#serialization

arnetheduck · 2020-06-18T12:45:05Z

In my implementation I added BLS key handling for the sake of providing functionality that was simple to implement and potentially useful in the future

YAGNI / it's going to be obsolete by the time of that future

arnetheduck · 2020-06-18T12:49:46Z

docs/stable/2-account.md

@@ -213,12 +220,193 @@ All messages sent are encrypted with the public key of the destination and signe

 -->

+## Public Key Compression


this is serialization - it's not compression, as in zip files. when serializing, you have the option to do so in a compressed format which replaces one of the two coordinates with a flag - you don't decompress it either, you deserialize or parse it

Most libraries call this functionality "compress" and "decompress", or "from_compressed", "to_compressed", "from_decompressed" and "to_decompressed". The common parlance for this functionality is compress and decompress.

To be technically accurate I can add to the Terms Glossary that the terms "public key compression" and "public key decompression" are colloquial terms for the technical terms "public key serialisation" and "public key deserialisation".

The target reader and beneficiary of this document is much more likely to search for and understand the terms compress and decompress, so my opinion is that we keep the idiomatic phraseology in the main body of the document for this reason.

On reflection I've decided to implement this differently, I've taken your suggestion on board and now use the "serialize" terminology, with only a minor sentence detailing the idiomatic version of the technically correct phrases.

arnetheduck · 2020-06-18T13:00:42Z

docs/stable/2-account.md

+Below is a representation of an uncompressed secp256k1 public key.
+
+```go
+[]byte{


hm, are we now using golang as lingua franca in specs? the extra []byte{ here, do we define what it is somewhere?

I've adjusted this. Should we presume the reader understands that the bytes are encoded in hex and are ordered left to right, top to bottom?

docs/stable/2-account.md

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

oskarth

generally lgtm but some more clear rationale / where it is used would be useful

oskarth · 2020-06-22T10:33:33Z

docs/stable/2-account.md

@@ -39,6 +39,13 @@ This specification explains what Status account is, and how a node establishes t
        - [Identicon](#identicon)
        - [3 word pseudonym / Whisper/Waku key fingerprint](#3-word-pseudonym--whisperwaku-key-fingerprint)
        - [ENS name](#ens-name)
+- [Public Key Compression](#public-key-compression)


I'm missing some rationale for this feature. I know the gist of it but it'd be good to have it documented. Where exactly is it used etc?

I've added a new subsection detailing where exactly this functionality would be used and why this functionality would be used. d4cc96c

Samyoul · 2020-06-22T22:16:29Z

@cammellos and @gravityblast would you mind giving me your thoughts on this spec submission?

@decanus would you mind re-reviewing the changes made to this submission?

@oskarth do you feel that the rationale given is sufficient and gives sufficiently specific information about the places that would use key "compression"?

Thank you

oskarth · 2020-06-23T04:10:15Z

@Samyoul seems good to me, thanks!

cammellos

I am still not completely sold on the why, but I won't rehash the conversation :)
But otherwise looks good to me

Samyoul · 2020-06-24T10:05:47Z

@decanus would you mind unblocking the merge of this PR? Thank you. 🙂

Samyoul added 2 commits June 17, 2020 17:27

Added Public key compression specs

d41a479

Added recommendation for encoding type of compressed keys

bff77a2

Samyoul requested review from cammellos, arnetheduck, oskarth, decanus and kdeme June 17, 2020 16:40

Samyoul added 4 commits June 17, 2020 21:25

Added unrecognised words to wordlist

0b62cc5

Add multibase to the wordlist

025413d

Added a basic example of the multiformat EC key compression concept

40891c9

Added parsable to wordlist

d776bff

This was referenced Jun 17, 2020

Feature/key compression status-im/status-go#1990

Merged

Compress public key for chat status-im/status-go#1937

Closed

oskarth reviewed Jun 18, 2020

View reviewed changes

docs/stable/2-account.md Outdated Show resolved Hide resolved

oskarth suggested changes Jun 18, 2020

View reviewed changes

oskarth requested a review from gravityblast June 18, 2020 10:12

arnetheduck reviewed Jun 18, 2020

View reviewed changes

Samyoul added 3 commits June 18, 2020 15:52

Hex is the only Lingua Franca we need

f63a31a

Language to make pk de/compression SHOULD implement

2eb6121

Added terms to glossary explaining key de/compression

4a76789

decanus suggested changes Jun 19, 2020

View reviewed changes

docs/stable/2-account.md Outdated Show resolved Hide resolved

Update docs/stable/2-account.md

149b451

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

Samyoul requested review from decanus and oskarth June 19, 2020 10:49

oskarth approved these changes Jun 22, 2020

View reviewed changes

Samyoul added 6 commits June 22, 2020 13:42

Change terminology from compress to serialise

c76ab8e

resolve merge conflict

6e8dfae

Added rationale for public key compression

d4cc96c

Added deserialization to the wordlist

0a5d084

Concise sentence

f2d8784

Added url to the wordlist

7fc4edb

Samyoul requested a review from arnetheduck June 22, 2020 22:12

cammellos approved these changes Jun 23, 2020

View reviewed changes

Samyoul self-assigned this Jun 23, 2020

Merge branch 'master' into update/key-compression

ab8d1e4

decanus approved these changes Jun 24, 2020

View reviewed changes

Samyoul merged commit e98a9b7 into master Jun 24, 2020

Samyoul deleted the update/key-compression branch June 24, 2020 10:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added public key compression specs #137

Added public key compression specs #137

Samyoul commented Jun 17, 2020

oskarth commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

oskarth left a comment

oskarth Jun 18, 2020

Samyoul Jun 18, 2020

Samyoul Jun 18, 2020

Samyoul commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

arnetheduck Jun 18, 2020 •

edited

Loading

Samyoul Jun 18, 2020

Samyoul Jun 22, 2020

arnetheduck Jun 18, 2020

Samyoul Jun 18, 2020

oskarth left a comment

oskarth Jun 22, 2020

Samyoul Jun 22, 2020

Samyoul commented Jun 22, 2020

oskarth commented Jun 23, 2020

cammellos left a comment

Samyoul commented Jun 24, 2020


		Released // TODO

		- Added details of public key compression and decompression

		@@ -213,12 +220,193 @@ All messages sent are encrypted with the public key of the destination and signe

		-->

		## Public Key Compression

Added public key compression specs #137

Added public key compression specs #137

Conversation

Samyoul commented Jun 17, 2020

What has changed?

Why make the change?

oskarth commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

oskarth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Samyoul commented Jun 18, 2020

arnetheduck commented Jun 18, 2020

arnetheduck Jun 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oskarth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Samyoul commented Jun 22, 2020

oskarth commented Jun 23, 2020

cammellos left a comment

Choose a reason for hiding this comment

Samyoul commented Jun 24, 2020

arnetheduck Jun 18, 2020 •

edited

Loading