[Docs] Improve protocol documentation

docs/protocol/messages/aggregate.rst

@@ -1,2 +1,27 @@
 Aggregates
-==========
+==========
+
+AGGREGATE messages are a global key/value store mechanism.
+
+Content format
+--------------
+
+The `content` field of a FORGET message must contain the following fields:
+
+* `address` [str]: The address to which the aggregate belongs. Reserved for future developments.
+* `time` [float]: The epoch timestamp of the message.
+* `key` [str]: The user-defined ID of the aggregate.
+* `content` [Dict]: The key/value pairs making up the aggregate, as a dictionary.
+
+Update aggregates
+-----------------
+
+Users can update aggregates by sending additional AGGREGATE messages with the same content key.
+Updates are ordered by their content time field to match the order in which the user sent
+the messages originally.
+
+Retrieve aggregates
+-------------------
+
+Users can retrieve aggregates by using the `/api/v0/aggregates/{address}.json` endpoint.
+Specify the `keys` URL parameter to restrict the response to one or more aggregates.

docs/protocol/messages/forget.rst

@@ -0,0 +1,29 @@
+Forgets
+=======
+
+FORGET messages are meant to make the Aleph network forget/drop one or more messages
+sent previously.
+Users can forget any type of message, except for FORGET messages themselves.
+
+When a FORGET message is received by a node, it will immediately:
+* remove the ‘content’ and ‘item_content’ sections of the targeted messages
+* add a field ‘removed_by’ that references to the processed FORGET message
+
+In addition, any content related to the forgotten message currently stored in the DB
+will be deleted, if no other message points to the same content. For example, a file
+stored in local storage will be deleted, or a file pinned in IPFS will be unpinned.
+
+Content format
+--------------
+
+The `content` field of a FORGET message must contain the following fields:
+
+* `address` [str]: The address to which the aggregate belongs. Reserved for future developments.
+* `time` [float]: The epoch timestamp of the message.
+* `hashes` [List[str]]: The list of message hashes to forget
+* `reason` [Optional[str]]: An optional explanation of why the user wants to forget these hashes.
+
+Limitations
+-----------
+
+* At the moment, a user can only forget messages he sent himself.

docs/protocol/messages/index.rst

@@ -1,9 +1,10 @@
 ********
 Messages
 ********
-All data transferred over the aleph.im network are aleph messages and represent the core of the Aleph networking model.
 
-Message can be:
+All data transferred over the aleph.im network are Aleph messages and represent the core of the Aleph networking model.
+
+Messages can be:
 
 - sent and received on the REST or other API interfaces
 - exchanged on the peer-to-peer network
@@ -87,7 +88,32 @@ Message types
 =============
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    aggregate
+   forget
    post
+   program
+   store
+
+Item hash, type and content
+===========================
+
+Messages are uniquely identified by the `item_hash` field.
+This value is obtained by computing the hash of the `content` field.
+Currently, the hash can be obtained in one of two ways.
+If the content of the message is stored on IPFS, the `item_hash` of the message will be the CIDv0
+of this content.
+Otherwise, if the message is stored on Aleph native storage or is included in the message, the item hash
+will be the SHA256 hash of the message in hexadecimal encoding.
+In the first case, the item type will be set to "ipfs". In the second case, the item type will either
+be "inline" if the content is included in the message (serialized as a string in the `item_content` field)
+or "storage".
+Inline storage will be used for content up to 200kB.
+Beyond this size, users must update the content prior to uploading the message on IPFS or Aleph storage.
+
+Signature
+=========
+
+Aleph messages are cryptographically signed with the private key of the user.
+The signature covers the `sender`, `chain`, `type` and `item_hash` fields of the message.

docs/protocol/messages/post.rst

@@ -1,2 +1,34 @@
 Posts
-=====
+=====
+
+Posts are unique data entries, that can be amended later on, like blog posts, comments, events...
+Internally, POST messages are similar to STORE messages but differ in that they support amending
+and only support JSON content.
+
+
+Content format
+--------------
+
+The `content` field of a POST message must contain the following fields:
+
+* `address` [str]: The address to which the aggregate belongs. Reserved for future developments.
+* `time` [float]: The epoch timestamp of the message.
+* `content` [Dict]: The JSON content of the post.
+* `ref` [Optional[str]]: Used for amending. If specified, must be set to the item hash of the original
+  message that created the post to modify.
+* `type` [str]: User-defined content type.
+
+Amend posts
+-----------
+
+Users can amend posts by sending additional POST messages referencing the original message.
+To do so, the user must send a new POST message with the content `ref` field set to
+the item hash of the original POST message.
+Note that even if the user amends the message multiple times, the `ref` field must always
+reference the original message, not the amendments.
+Amendments are applied in the order of the content `time` field.
+
+Retrieve posts
+--------------
+
+Users can retrieve posts by using the `/api/v0/posts.json` endpoint.

docs/protocol/messages/program.rst

@@ -0,0 +1,49 @@
+Programs
+========
+
+PROGRAM messages create a new application that can then be run on Aleph VMs.
+
+Content format
+--------------
+
+The `content` field of a PROGRAM message must contain the following fields:
+
+.. code-block:: json
+
+    "code": {
+      "encoding": "plain | zip | tar.gzip",
+      "entrypoint": "application",
+      "ref": "str",
+      "ref": "str",
+      "use_latest": true,
+    },
+    "on": {
+      "http": true,
+      "cron": "5 4 * * *",
+      "aleph": [
+        {"type": "POST", "channel": "FOUNDATION", "content": {"type": "calculation"}}
+      ]
+    },
+    "environment":{
+      "reproducible": true,
+      "internet": false,
+      "aleph_api": false
+    },
+    "resources": {
+      "vcpus": 1,
+      "memory": 128,
+      "seconds": 1
+    },
+    "runtime": {
+      "address": "0x4cB66fDf10971De5c7598072024FFd33482907a5",
+      "comment": "Aleph Alpine Linux with Python 3.8"
+    },
+    "data": {
+      "encoding": "tar.gzip",
+      "mount": "/mnt",
+      "address": "0xED9d5B040386F394B9ABd34fD59152756b126710"
+    },
+    "export": {
+      "encoding": "tar.gzip",
+      "mount": "/mnt"
+    }

docs/protocol/messages/store.rst

@@ -0,0 +1,21 @@
+Stores
+======
+
+STORE messages tell the Aleph network to store data on behalf of the user.
+The data can either be pinned to IPFS or stored in the native Aleph storage system depending
+on the content item type.
+
+Content format
+--------------
+
+The `content` field of a STORE message must contain the following fields:
+
+* `address` [str]: The address to which the aggregate belongs. Reserved for future developments.
+* `time` [float]: The epoch timestamp of the message.
+* `item_type` [str]: `storage` or `ipfs`. Determines the network to use to fetch and store the file.
+* `item_hash` [str]: Hash of the file to store. Must be a CIDv0 for IPFS, or a SHA256 hash for native storage.
+
+Retrieve stored content
+-----------------------
+
+Users can retrieve uploaded files by using the `/api/v0/storage/raw/{hash}` endpoint.