Update/waku replace (#114)

Introduces Waku in to the relevant places in the specs.

* First draft changes to 1-client.md

* initial waku replace work on 2-account.md

* Rename 3-whisper-usage to waku

There may be broken links after this change

* Initial update for 3-waku-usage.md

* Adjustment to use WHISPER concurrently with WAKU

* Converting 'moving away from whisper' section to `why use waku`

* Added clarification about Open Whisper Systems X3DH usage

* Updated transport privacy, include both whisper and waku

* Updated 1-client, language supports concurrent whisper and waku

* Updated 2-account.md to be whisper/waku concurrent

* Reverted 3-whisper-usage changes, added 9-waku-usage

* Complete revert of 3-whisper-usage

* revert 3-whisper-usage.md

* Fix to 3-whisper-usage.md links

* Update docs/stable/1-client.md

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Removal of personal pronouns from 1-client.md

* Update to 5-secure-transport.md

* Implemented combined 'whispe/ waku relayer' role

* Addressed ambiguous and stale links in 6-payloads

* Updated 6-payloads

* Fix broken / stale links in 7-group-chat.md

* Fixed stale link 8-eips

* link fix and todo resolved for 1-client.md

* Added 10-waku-mailserver document

* Added gitignore

* Minor tweaks and fixes

* Added Whisper / Waku briding section to 3-whisper-usage

* Updated 10-waku-mailserver

* Updated 9-waku-usage.md

* Made changes to keep Mailserver term consistent

* Removed some redundant comments

* Removed seemingly vistigual '???'

* Including large payload PoW

See https://github.com/status-im/status-go/blob/7b6af0a7ccb8e39eeb2ab71ccca9841d75d57416/protocol/message_processor.go#L25-L35

* Changed 'mail server' to 'Mailserver'

* Added details about status-options

* Implemented final changes before review

* Tpyo fix

* Removed incorrect reference to topic-interest

* removed .gitignore, defer to global_gitignore

* Addressing feedback

* Revert draft/7 link fix

Readdressed in GH-120

* Reverted link fixes for stable/5

Addressing in GH-120

* Revert tpyo fix

Addressed in GH-119

* Reverted stable/6 link fixes

Addressed in GH-120

* Missed a minor link change

* Revert link fix in stable/1

Addressed in GH-120

* Revert tpyo fix in stable/1

Addressed in GH-119

* Revert stable/1 link fix

Addressed in GH-120

* Revert stable/2 link fix

Addressed in GH-120

* Revert link fix to stable/2

* Revert stable/3 link fixes

Addressed in GH-120

* Revert adding missing topic link

Addressed in GH-120

* Revert stable/4 link fix

Addressed in GH-120

* Revert stable/5 link fix

Addressed in GH-120

* Revert stable/8 link fix

Addressed in GH-120

* Partial revert of stable/1 node roles

* Resolved whispermail reference query

See GH-125

* Added changelog and version bump ...

Added my name where appropriate and addressed some minor issues

* Added consistent capitalisation to 'Whisper'

* Resolved jRPC call issue for waku

* Updated spelling

* Addressing minor feedback issues

* Update docs/stable/10-waku-mailserver.md

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Update docs/stable/10-waku-mailserver.md

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

Author: Samyoul

docs/stable/1-client.md

@@ -0,0 +1,129 @@
+---
+permalink: /spec/10
+parent: Stable specs
+title: 10/WAKU-MAILSERVER
+---
+
+# 10/WAKU-MAILSERVER
+
+> Version: 0.1
+>
+> Status: Stable
+>
+> Authors: Adam Babik <adam@status.im>, Oskar Thorén <oskar@status.im>, Samuel Hawksby-Robinson <samuel@status.im> (alphabetical order)
+
+- [Status Waku Mailserver Specification](#10waku-mailserver)
+  - [Abstract](#abstract)
+  - [`Mailserver`](#mailserver)
+    - [Archiving messages](#archiving-messages)
+    - [Requesting messages](#requesting-messages)
+    - [Receiving historic messages](#receiving-historic-messages)
+  - [Security considerations](#security-considerations)
+    - [Confidentiality](#confidentiality)
+    - [Altruistic and centralized operator risk](#altruistic-and-centralized-operator-risk)
+    - [Privacy concerns](#privacy-concerns)
+    - [Denial-of-service](#denial-of-service)
+  - [Changelog](#changelog)
+    - [Version 0.1](#version-01)
+
+## Abstract
+
+Being mostly offline is an intrinsic property of mobile clients. They need to save network transfer and battery consumption to avoid spending too much money or constant charging. Waku protocol, on the other hand, is an online protocol. Messages are available in the Waku network only for short period of time calculate in seconds.
+
+Waku Mailserver is a specification that allows messages to be stored permanently and to allows the stored messages to be delivered to requesting client nodes, regardless if the messages are not available in the network due to the message TTL expiring.
+
+## `Mailserver`
+
+From the network perspective, a `Mailserver` is just like any other Waku node. The only difference is that a `Mailserver` has the capability of archiving messages and delivering them to its peers on-demand.
+
+It is important to notice that a `Mailserver` will only handle requests from its direct peers and exchanged packets between a `Mailserver` and a peer are p2p messages.
+
+### Archiving messages
+
+A node which wants to provide `Mailserver` functionality MUST store envelopes from
+incoming message packets (Waku packet-code `0x01`). The envelopes can be stored in any
+format, however they MUST be serialized and deserialized to the Waku envelope format.
+
+A `Mailserver` SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
+
+### Requesting messages
+
+In order to request historic messages, a node MUST send a packet P2P Request (`0x7e`) to a peer providing `Mailserver` functionality. This packet requires one argument which MUST be a Waku envelope.
+
+In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
+
+```
+[ Lower, Upper, Bloom, Limit, Cursor ]
+```
+
+`Lower`: 4-byte wide unsigned integer (UNIX time in seconds; oldest requested envelope's creation time)  
+`Upper`: 4-byte wide unsigned integer (UNIX time in seconds; newest requested envelope's creation time)  
+`Bloom`: 64-byte wide array of Waku topics encoded in a bloom filter to filter envelopes  
+`Limit`: 4-byte wide unsigned integer limiting the number of returned envelopes  
+`Cursor`: an array of a cursor returned from the previous request (optional)
+
+The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit` so that the requester can send another request using the obtained `Cursor` value. What exactly is in the `Cursor` is up to the implementation. The requester SHOULD NOT use a `Cursor` obtained from one `Mailserver` in a request to another `Mailserver` because the format or the result MAY be different.
+
+The envelope MUST be encrypted with a symmetric key agreed between the requester and the `Mailserver`.
+
+### Receiving historic messages
+
+Historic messages MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes.
+
+In order to receive historic messages from a `Mailserver`, a node MUST trust the selected `Mailserver`, that is allow to receive packets with the P2P Message code. By default, such packets are discarded.
+
+Received envelopes MUST be passed through the Waku envelope pipelines so that they are picked up by registered filters and passed to subscribers.
+
+For a requester, to know that all messages have been sent by a `Mailserver`, it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by the following parameters:
+
+```
+[ RequestID, LastEnvelopeHash, Cursor ]
+```
+
+ - `RequestID`: 32-byte wide array with a Keccak-256 hash of the envelope containing the original request  
+ - `LastEnvelopeHash`: 32-byte wide array with a Keccak-256 hash of the last sent envelope for the request  
+ - `Cursor`: an array of a cursor returned from the previous request (optional)
+
+If `Cursor` is not empty, it means that not all messages were sent due to the set `Limit` in the request. One or more consecutive requests MAY be sent with `Cursor` field filled in order to receive the rest of the messages.
+
+## Security considerations
+
+### Confidentiality
+
+All Waku envelopes are encrypted. A `Mailserver` node can not inspect their contents.
+
+### Altruistic and centralized operator risk
+
+In order to be useful, a `Mailserver` SHOULD be online most of time. That means
+users either have to be a bit tech-savvy to run their own node, or rely on someone
+else to run it for them.
+
+Currently one of Status's legal entities provides `Mailservers` in an altruistic manner, but this is
+suboptimal from a decentralization, continuance and risk point of view. Coming
+up with a better system for this is ongoing research.
+
+A Status client SHOULD allow the `Mailserver` selection to be customizable.
+
+### Privacy concerns
+
+In order to use a `Mailserver`, a given node needs to connect to it directly,
+i.e. add the `Mailserver` as its peer and mark it as trusted. This means that the
+`Mailserver` is able to send direct p2p messages to the node instead of
+broadcasting them. Effectively, it will have access to the bloom filter of
+topics that the user is interested in, when it is online as well as many
+metadata like IP address.
+
+### Denial-of-service
+
+Since a `Mailserver` is delivering expired envelopes and has a direct TCP connection with the recipient, the recipient is vulnerable to DoS attacks from a malicious `Mailserver` node.
+
+## Changelog
+
+### Version 0.1
+
+Released `TODO`
+
+- Created document
+- Forked from [4-whisper-mailserver](4-whisper-mailserver.md)
+- Change to keep `Mailserver` term consistent
+- Replaced Whisper references with Waku

docs/stable/10-waku-mailserver.md

@@ -6,11 +6,11 @@ title: 2/ACCOUNT
 
 # 2/ACCOUNT
 
-> Version: 0.2
+> Version: 0.3
 > 
 > Status: Stable
 >
-> Authors: Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
+> Authors: Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im>, Samuel Hawksby-Robinson <samuel@status.im> (alphabetical order)
 
 ## Abstract
 
@@ -38,16 +38,18 @@ In this specification we explain what Status account is, and how trust is establ
         - [Bundles](#bundles)
     - [Contact Verification](#contact-verification)
         - [Identicon](#identicon)
-        - [3 word pseudonym / whisper key fingerprint](#3-word-pseudonym--whisper-key-fingerprint)
+        - [3 word pseudonym / Whisper/Waku key fingerprint](#3-word-pseudonym--whisperwaku-key-fingerprint)
         - [ENS name](#ens-name)
 - [Security Considerations](#security-considerations)
+- [Changelog](#changelog)
+  - [Version 0.3](#version-03)
 
 <!-- markdown-toc end -->
 
 ## Introduction
 
 The core concept of an account in Status is a set of cryptographic keypairs. Namely, the combination of the following:
-1. a whisper chat identity keypair
+1. a Whisper/Waku chat identity keypair
 1. a set of cryptocurrency wallet keypairs
 
 Everything else associated with the contact is either verified or derived from the above items, including:
@@ -60,18 +62,18 @@ Everything else associated with the contact is either verified or derived from t
 ### Public/Private Keypairs 
 - An ECDSA (secp256k1 curve) public/private keypair MUST be generated via a [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki) derived path from a [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) mnemonic seed phrase.
 - The default paths are defined as such:
-    - Whisper Chat Key (`IK`): `m/43'/60'/1581'/0'/0`  (post Multiaccount integration)
+    - Whisper/Waku Chat Key (`IK`): `m/43'/60'/1581'/0'/0`  (post Multiaccount integration)
         - following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
     <!-- WE CURRENTLY DO NOT IMPLEMENT ENCRYPTION KEY, FOR FUTURE - C.P. -->
     <!-- - DB encryption Key (`DBK`): `m/43'/60'/1581'/1'/0` (post Multiaccount integration)
         - following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md) -->
     - Status Wallet paths: `m/44'/60'/0'/0/i` starting at `i=0`
         - following [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
-        - NOTE: this (`i=0`) is also the current (and only) path for Whisper key before Multiaccount integration
+        - NOTE: this (`i=0`) is also the current (and only) path for Whisper/Waku key before Multiaccount integration
 
 ### X3DH Prekey bundle creation
-- Status follows the X3DH prekey bundle scheme that Open Whisper Systems outlines [in their documentation](https://signal.org/docs/specifications/x3dh/#the-x3dh-protocol) with the following exceptions:
-    - Because there are no central servers, we do not publish one-time keys `OPK` or perform DH including them. 
+- Status follows the X3DH prekey bundle scheme that [Open Whisper Systems](https://en.wikipedia.org/wiki/Signal_Messenger#2013%E2%80%932018:_Open_Whisper_Systems) (not to be confused with the Whisper sub-protocol) outlines [in their documentation](https://signal.org/docs/specifications/x3dh/#the-x3dh-protocol) with the following exceptions:
+    - Status does not publish one-time keys `OPK` or perform DH including them, because there are no central servers in the Status implementation. 
 - A client MUST create X3DH prekey bundles, each defined by the following items:
     - Identity Key: `IK`
     - Signed prekey: `SPK`
@@ -84,14 +86,14 @@ Everything else associated with the contact is either verified or derived from t
 
 ### X3DH Prekey bundles
 - A client SHOULD regenerate a new X3DH prekey bundle every 24 hours.  This MAY be done in a lazy way, such that a client that does not come online past this time period does not regenerate or broadcast bundles.
-- The current bundle SHOULD be broadcast on a whisper topic specific to his Identity Key, `{IK}-contact-code`, intermittently.  This MAY be done every 6 hours.
+- The current bundle SHOULD be broadcast on a Whisper/Waku topic specific to his Identity Key, `{IK}-contact-code`, intermittently.  This MAY be done every 6 hours.
 - A bundle SHOULD accompany every message sent.
 - TODO: retrieval of long-time offline users bundle via `{IK}-contact-code` 
 
 ## Optional Account additions
 
 ### ENS Username
-- A user MAY register a public username on the Ethereum Name System (ENS).  This username is a user-chosen subdomain of the `stateofus.eth` ENS registration that maps to their whisper identity key (`IK`). 
+- A user MAY register a public username on the Ethereum Name System (ENS).  This username is a user-chosen subdomain of the `stateofus.eth` ENS registration that maps to their Whisper/Waku identity key (`IK`). 
 
 <!-- ### User Profile Picture
 - An account MAY edit the `IK` generated identicon with a chosen picture.  This picture will become part of the publicly broadcast profile of the account. -->
@@ -105,11 +107,11 @@ Everything else associated with the contact is either verified or derived from t
 
 ### Terms Glossary
 
-| term | description |
-| ---- | ----------- |
-| privkey | ECDSA secp256k1 private key |
-| pubkey | ECDSA secp256k1 public key |
-| whisper key | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
+| term             | description |
+| ---------------- | ----------- |
+| privkey          | ECDSA secp256k1 private key |
+| pubkey           | ECDSA secp256k1 public key |
+| Whisper/Waku key | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
 
 
 ### Contact Discovery
@@ -133,8 +135,8 @@ This can be done in the following ways:
 1. public key via public channel listening
     - `status-react/src/status_im/contact_code/core.cljs`
 1. contact codes
-2. decentralized storage (not implemented)
-3. whisper
+1. decentralized storage (not implemented)
+1. Whisper/Waku
 
 ### Initial Key Exchange
 
@@ -148,24 +150,24 @@ This can be done in the following ways:
   Signature               // Prekey signature
   Timestamp               // When the bundle was lasted created locally
   ```
-  - include BundleContainer???
+  - include BundleContainer
 - a new bundle SHOULD be created at least every 12 hours
 - a bundle is only generated when it is used
-- a bundle SHOULD be distributed on the contact code channel. This is the whisper topic `{IK}-contact-code`, where `IK` is the hex encoded public key of the user, prefixed with `0x`. The channel is encrypted in the same way public chats are encrypted.
+- a bundle SHOULD be distributed on the contact code channel. This is the Whisper and Waku topic `{IK}-contact-code`, where `IK` is the hex encoded public key of the user, prefixed with `0x`. The channel is encrypted in the same way public chats are encrypted.
 
 ### Contact Verification
 
 Once you have the information of a contact, the following can be used to verify that the key material is as it should be.
 
 #### Identicon
-A low-poly identicon is deterministically generated from the whisper chat public key.  This can then be compared out of band to ensure the receiver's public key is the one you have locally.
+A low-poly identicon is deterministically generated from the Whisper/Waku chat public key.  This can then be compared out of band to ensure the receiver's public key is the one you have locally.
 
-#### 3 word pseudonym / whisper key fingerprint
-Status generates a deterministic 3-word random pseudonym from the whisper chat public key.  This pseudonym acts as a human readable fingerprint to the whisper chat public key.  This name also shows when viewing a contact's public profile and in the chat UI.
+#### 3 word pseudonym / Whisper/Waku key fingerprint
+Status generates a deterministic 3-word random pseudonym from the Whisper/Waku chat public key.  This pseudonym acts as a human readable fingerprint to the Whisper/Waku chat public key.  This name also shows when viewing a contact's public profile and in the chat UI.
 - implementation: [gfycat](https://github.com/status-im/status-react/tree/develop/src/status_im/utils/gfycat)
 
 #### ENS name
-Status offers the ability to register a mapping of a human readable subdomain of `stateofus.eth` to their whisper chat public key.  This registration is purchased (currently by staking 10 SNT) and stored on the Ethereum mainnet blockchain for public lookup.
+Status offers the ability to register a mapping of a human readable subdomain of `stateofus.eth` to their Whisper/Waku chat public key.  This registration is purchased (currently by staking 10 SNT) and stored on the Ethereum mainnet blockchain for public lookup.
 
 <!-- TODO: Elaborate on security implications -->
 
@@ -183,7 +185,7 @@ possible connections
     - public chat
 - client - mailserver (statusd + ???)
     - a mailserver identifies itself by an [enode address](https://github.com/ethereum/wiki/wiki/enode-url-format) 
-- client - whisper node (statusd)
+- client - Whisper/Waku node (statusd)
     - a node identifies itself by an enode address
 - client - bootnode (go-ethereum)
     - a bootnode identifies itself by
@@ -217,3 +219,13 @@ All messages sent are encrypted with the public key of the destination and signe
 ## Security Considerations
 
 -
+
+## Changelog
+
+### Version 0.3
+
+Released `TODO`
+
+- Added language to include Waku in all relevant places
+- Change to keep `Mailserver` term consistent 
+- Added clarification to Open Whisper Systems