From 33570042d8f2241ecc9a9104f5eb38fbf4bc3c95 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 28 Jul 2020 11:24:45 +0300 Subject: [PATCH 01/52] add message rfc --- text/0017-message/0017-message.md | 180 ++++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 text/0017-message/0017-message.md diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md new file mode 100644 index 000000000..b2d86cf5f --- /dev/null +++ b/text/0017-message/0017-message.md @@ -0,0 +1,180 @@ ++ Feature name: `Tangle Messages` ++ Start date: 2020-07-28 ++ RFC PR: [iotaledger/protocol-rfcs#0017](https://github.com/iotaledger/protocol-rfcs/pull/0017) + +# Summary + +A message is the object nodes gossip around in the network. It always references two other messages that are known as `parents`. It is stored as a vertex on the tangle data structure that the nodes maintain. + +The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable to build new functionality on top of the tangle. + +Some payloads may have other nested payloads embedded inside. +So upon parsing it is done layer by layer. + + + +# Motivation + +To better understand this layered design, consider the internet protocol for example. There is an Ethernet frame, that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility, once this responsibility is completed, we move on to the next layer. + +The same goes with how we parse messages. The outer layer of the message enables us to map the message to a vertex in the Tangle and perform some basic validation. The next layer may be a transaction that mutates the ledger state. The next layer may provide some extra functionality on the transactions to be used by applications. + +By making it possible to add and exchange payloads, we are creating an architecture that can be easily extended to accommodate future needs. + + +# Detailed design + +### Data Types + +The following are data types that we will use when we specify fields in the message and payloads. + +| Name | Description | +| ------ | ------------- | +| varint | An unsigned numerical value using a dynamic amount of bytes encoded in Little Endian. Evaluate by inspecting the MSB of the first byte. If it is `0` stop. If it is `1` go to the next byte. Continue applying this rule until you reach a byte with MSB `0`. Transform each byte to a 7-bit bit sequence by stripping the MSBs from each byte. Reverse the order of the 7 bit groups (Little Endian). Concatenate all the bits and evaluate as an unsigned integer. [It is similar to how it is done in Protocol Buffers](https://developers.google.com/protocol-buffers/docs/encoding). We must add [validation rules](#Message-Validation) that ensure that the varint is not malleable, and it is limited in size. | +| uint64 | An unsigned 64 bit integer encoded in Little Endian. +| ByteArray | A dynamic size byte array. It is a prefixed by a `varint` that indicates the number of bytes in the array. The following bytes are simply the content of the array. +| ByteArray[N] | A static size array of size N. Since the size is expected, no need to prefix with a varint | + + +### Message ID +The message ID will be the `Blake2b-256` hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. + + +### Message Structure + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
VersionvarintThe message version. The schema specified in this RFC is for version 1 only.
Parent #1 (trunk)ByteArray[32]The Message ID of the first Message we reference.
Parent #2 (branch)ByteArray[32]The Message ID of the second Message we reference.
Payload Lengthvarint The length of the Payload. Since its type may be unknown to the node it must be declared in advanced.
+ Payload + +
+ Generic Payload +
+ An outline of a general payload +
+ + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Payload Typevarint + The type of the payload. It will instruct the node how to parse the fields that follow. Types in the range of 0-127 are "core types" that all nodes are expected to know. +
Data FieldsANYA sequence of fields, where the structure depends on payload type.
Nested Payload (Optional) Payload Some payloads support nesting other payloads inside of them.
+
+
Nonceuint64The nonce which lets this message fulfill the Proof-of-Work requirement.
+ +### Message PoW Hash + +This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with CURL-P81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection. + +Ideally the Message ID would have been used. Then we could have fully deprecated all trinary remnants. However we must pay attention to specialized hardware that can render the PoW protection useless. The advantage of CURL that we have a scarce presence of FPGAs that can easily break it. + +### Message Validation + +A message is considered valid, if the following syntactic rules are met: + +1. The message length must not exceed X [tbd] bytes. +2. Varint fields must adhere to the following rules: + * *Malleability protection:* No `0x80` or `0x00` trailing bytes. Take for example the following 3 byte varint: `0x8F8000`. + In order to parse it we look at its bits: + `1000 1111 1000 0000 0000 0000`. + First we strip the msbs: + `000 1111 000 0000 000 0000` + Then we reverse the groups of 7 bits (Little Endian): + `000 0000 000 0000 000 1111` + Then we concatenate all the bits to finally get: + `1111` + So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes. + * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid. +2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. +4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need. +5. If the payload is valid according to the **syntactic** rules the `payload type` defines. +6. If the Message PoW Hash will contain at least the number of trailing 0 trits the node defines as required. + +### Payloads + +Each message will contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. + +| Payload Name | +| --------------------------------------- | +| [Unsigned Data](#Unsigned-Data-Payload) | +| Signed Data | +| Transaction | +| Indexation | + +### Unsigned Data Payload + +To make the Payload concept clear we will define the `unsigned data payload`. As the name suggests it simply allows to add arbitrary data to a message. A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. For example, one can record a hash of a text document on the tangle with such a payload. When a third party requests a proof that the document was published at a certain time and was not changed, one can point them to them to the message containing the `unsigned data payload` with the hash. + +The structure of the payload is simple: + + + +| Name | Type | Description | +| -------- | -------- | ----------- | +| Payload Type | varint | Must be set to **2** | +| Data | ByteArray| The data we are attaching | + +There are no validation rules for the payload. Message validation rules suffice here. + + + +# Rationale and alternatives + +Instead of creating a layered approach, we could have simply created a flat`transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. + +This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. + +# Unresolved questions + +- What should be the length of the message? +- Better protection with PoW? how does the length of the message and type of payloads should affect difficulty? +- How should we pad the message to fit it into CURL for PoW? +- To what non-trinary hash function should we switch to? \ No newline at end of file From 805758ff1ceea6d37ea2067a8969f017609baae1 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 28 Jul 2020 15:16:04 +0300 Subject: [PATCH 02/52] add payload types --- text/0017-message/0017-message.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index b2d86cf5f..58b259740 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -142,12 +142,12 @@ A message is considered valid, if the following syntactic rules are met: Each message will contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. -| Payload Name | -| --------------------------------------- | -| [Unsigned Data](#Unsigned-Data-Payload) | -| Signed Data | -| Transaction | -| Indexation | +| Payload Name | Type Value | +| --------------------------------------- | ---------- | +| Signed Transaction | 0 | +| [Unsigned Data](#Unsigned-Data-Payload) | 2 | +| Signed Data | 3 | | +| Indexation | 4 | ### Unsigned Data Payload From fa24ebb9416d0f51b7475cb41f4902f7ad3c944e Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 29 Jul 2020 00:00:49 +0300 Subject: [PATCH 03/52] corrections --- text/0017-message/0017-message.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 58b259740..8aa5f5a09 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -37,7 +37,7 @@ The following are data types that we will use when we specify fields in the mess ### Message ID -The message ID will be the `Blake2b-256` hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. +The message ID will be the `BLAKE2b-256` hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. ### Message Structure @@ -54,12 +54,12 @@ The message ID will be the `Blake2b-256` hash of the byte contents of the messag The message version. The schema specified in this RFC is for version 1 only. - Parent #1 (trunk) + Parent1 (trunk) ByteArray[32] The Message ID of the first Message we reference. - Parent #2 (branch) + Parent2 (branch) ByteArray[32] The Message ID of the second Message we reference. @@ -112,9 +112,9 @@ The message ID will be the `Blake2b-256` hash of the byte contents of the messag ### Message PoW Hash -This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with CURL-P81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection. +This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with Curl-P-81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection. -Ideally the Message ID would have been used. Then we could have fully deprecated all trinary remnants. However we must pay attention to specialized hardware that can render the PoW protection useless. The advantage of CURL that we have a scarce presence of FPGAs that can easily break it. +Ideally, the Message ID would also be used for the PoW, but then the Hash function used for the Message ID must also be used for the PoW. Separating them gives us much more flexibility with the PoW method and we can even apply the current Curl based PoW mechanism. ### Message Validation @@ -142,12 +142,12 @@ A message is considered valid, if the following syntactic rules are met: Each message will contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. -| Payload Name | Type Value | -| --------------------------------------- | ---------- | -| Signed Transaction | 0 | -| [Unsigned Data](#Unsigned-Data-Payload) | 2 | -| Signed Data | 3 | | -| Indexation | 4 | +| Payload Name | Type Value | +| --------------------------------------- | ----------- | +| Signed Transaction | 0 | +| Signed Data | 1 | +| [Unsigned Data](#Unsigned-Data-Payload) | 2 | +| Indexation | 3 | ### Unsigned Data Payload @@ -174,7 +174,8 @@ This approach seems less extensible. It might have made sense if we wanted to bu # Unresolved questions -- What should be the length of the message? +- What should be the maximum length of the message? - Better protection with PoW? how does the length of the message and type of payloads should affect difficulty? - How should we pad the message to fit it into CURL for PoW? -- To what non-trinary hash function should we switch to? \ No newline at end of file +- How should we pad the input for CURL? One proposal is to go with padding method 2 that is described here https://en.wikipedia.org/wiki/ISO/IEC_9797-1#Padding +- Do we want to move away from using CURL for PoW in the future? \ No newline at end of file From c778814d7935a2888c72a752362bf4d5a6e1f88b Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 29 Jul 2020 00:11:09 +0300 Subject: [PATCH 04/52] add space --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 8aa5f5a09..eb02d8c8b 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -168,7 +168,7 @@ There are no validation rules for the payload. Message validation rules suffice # Rationale and alternatives -Instead of creating a layered approach, we could have simply created a flat`transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. +Instead of creating a layered approach, we could have simply created a flat `transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. From 915cd2bbccd5c0854fe93e8843aa8542cd0ca071 Mon Sep 17 00:00:00 2001 From: galrogo Date: Mon, 3 Aug 2020 11:45:32 +0300 Subject: [PATCH 05/52] Add milestone payload --- text/0017-message/0017-message.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index eb02d8c8b..e40aaf561 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -145,9 +145,10 @@ Each message will contain a payload. The specification of the payloads is out of | Payload Name | Type Value | | --------------------------------------- | ----------- | | Signed Transaction | 0 | -| Signed Data | 1 | +| Milestone | 1 | | [Unsigned Data](#Unsigned-Data-Payload) | 2 | -| Indexation | 3 | +| Signed Data | 3 | +| Indexation | 4 | ### Unsigned Data Payload From b973ccf79da0c9c177d478b0f16e4c0effeb7daa Mon Sep 17 00:00:00 2001 From: galrogo Date: Thu, 6 Aug 2020 12:33:09 +0300 Subject: [PATCH 06/52] Remove dependency on payload validation --- text/0017-message/0017-message.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index e40aaf561..523cb0a67 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -133,10 +133,9 @@ A message is considered valid, if the following syntactic rules are met: `1111` So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes. * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid. -2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. +3. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. 4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need. -5. If the payload is valid according to the **syntactic** rules the `payload type` defines. -6. If the Message PoW Hash will contain at least the number of trailing 0 trits the node defines as required. +5. If the Message PoW Hash will contain at least the number of trailing 0 trits the node defines as required. ### Payloads From 5803db647e74a5974ef48d146bf17a4491231eb3 Mon Sep 17 00:00:00 2001 From: Gal Rogozinski Date: Tue, 25 Aug 2020 11:24:33 +0300 Subject: [PATCH 07/52] change to lower case Co-authored-by: Thibault Martinez --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 523cb0a67..97906d0db 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -1,4 +1,4 @@ -+ Feature name: `Tangle Messages` ++ Feature name: `tangle-message` + Start date: 2020-07-28 + RFC PR: [iotaledger/protocol-rfcs#0017](https://github.com/iotaledger/protocol-rfcs/pull/0017) @@ -178,4 +178,4 @@ This approach seems less extensible. It might have made sense if we wanted to bu - Better protection with PoW? how does the length of the message and type of payloads should affect difficulty? - How should we pad the message to fit it into CURL for PoW? - How should we pad the input for CURL? One proposal is to go with padding method 2 that is described here https://en.wikipedia.org/wiki/ISO/IEC_9797-1#Padding -- Do we want to move away from using CURL for PoW in the future? \ No newline at end of file +- Do we want to move away from using CURL for PoW in the future? From 4ce1327dfbfaa65c4e9bebccb4d0ce257a71e16f Mon Sep 17 00:00:00 2001 From: Gal Rogozinski Date: Tue, 25 Aug 2020 11:25:02 +0300 Subject: [PATCH 08/52] change we to it Co-authored-by: Thibault Martinez --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 97906d0db..3cf4d66f5 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -56,7 +56,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Parent1 (trunk) ByteArray[32] - The Message ID of the first Message we reference. + The Message ID of the first Message it references. Parent2 (branch) From 04b35a7cb19c875cffe3e317829b5aa9bdaab980 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 11:45:10 +0300 Subject: [PATCH 09/52] remove message pow section --- text/0017-message/0017-message.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 3cf4d66f5..d26fe524c 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -61,7 +61,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Parent2 (branch) ByteArray[32] - The Message ID of the second Message we reference. + The Message ID of the second Message it references. Payload Length @@ -110,12 +110,6 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag -### Message PoW Hash - -This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with Curl-P-81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection. - -Ideally, the Message ID would also be used for the PoW, but then the Hash function used for the Message ID must also be used for the PoW. Separating them gives us much more flexibility with the PoW method and we can even apply the current Curl based PoW mechanism. - ### Message Validation A message is considered valid, if the following syntactic rules are met: @@ -135,7 +129,7 @@ A message is considered valid, if the following syntactic rules are met: * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid. 3. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. 4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need. -5. If the Message PoW Hash will contain at least the number of trailing 0 trits the node defines as required. +5. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. ### Payloads From 67c80044569bbdbf9f530063abf8bb722a64cfca Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 11:59:35 +0300 Subject: [PATCH 10/52] Remove empty lines --- text/0017-message/0017-message.md | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index d26fe524c..f266e8231 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -6,13 +6,9 @@ A message is the object nodes gossip around in the network. It always references two other messages that are known as `parents`. It is stored as a vertex on the tangle data structure that the nodes maintain. -The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable to build new functionality on top of the tangle. - -Some payloads may have other nested payloads embedded inside. +The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable to build new functionality on top of the tangle. Some payloads may have other nested payloads embedded inside. So upon parsing it is done layer by layer. - - # Motivation To better understand this layered design, consider the internet protocol for example. There is an Ethernet frame, that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility, once this responsibility is completed, we move on to the next layer. @@ -21,7 +17,6 @@ The same goes with how we parse messages. The outer layer of the message enables By making it possible to add and exchange payloads, we are creating an architecture that can be easily extended to accommodate future needs. - # Detailed design ### Data Types @@ -149,8 +144,6 @@ To make the Payload concept clear we will define the `unsigned data payload`. As The structure of the payload is simple: - - | Name | Type | Description | | -------- | -------- | ----------- | | Payload Type | varint | Must be set to **2** | @@ -158,8 +151,6 @@ The structure of the payload is simple: There are no validation rules for the payload. Message validation rules suffice here. - - # Rationale and alternatives Instead of creating a layered approach, we could have simply created a flat `transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. From 9d1f832ccbc0e928b22645d10f01c391afcaf6ce Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:00:48 +0300 Subject: [PATCH 11/52] remove PoW questions --- text/0017-message/0017-message.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index f266e8231..828dc3300 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -160,7 +160,3 @@ This approach seems less extensible. It might have made sense if we wanted to bu # Unresolved questions - What should be the maximum length of the message? -- Better protection with PoW? how does the length of the message and type of payloads should affect difficulty? -- How should we pad the message to fit it into CURL for PoW? -- How should we pad the input for CURL? One proposal is to go with padding method 2 that is described here https://en.wikipedia.org/wiki/ISO/IEC_9797-1#Padding -- Do we want to move away from using CURL for PoW in the future? From 8ee6316bb11cce759acde2d636183d286bdaf9ce Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:09:34 +0300 Subject: [PATCH 12/52] Allow 0 length payloads --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 828dc3300..615b9b7ba 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -61,7 +61,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Payload Length varint - The length of the Payload. Since its type may be unknown to the node it must be declared in advanced. + The length of the Payload. Since its type may be unknown to the node it must be declared in advanced. 0 length means no payload will be attached. @@ -128,7 +128,7 @@ A message is considered valid, if the following syntactic rules are met: ### Payloads -Each message will contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. +A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. | Payload Name | Type Value | | --------------------------------------- | ----------- | From 2caf73138667ddd3ef04acf68755dbf359c9f34a Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:23:39 +0300 Subject: [PATCH 13/52] fix sentence --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 615b9b7ba..b0bf827c8 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -140,7 +140,7 @@ A message may contain a payload. The specification of the payloads is out of sco ### Unsigned Data Payload -To make the Payload concept clear we will define the `unsigned data payload`. As the name suggests it simply allows to add arbitrary data to a message. A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. For example, one can record a hash of a text document on the tangle with such a payload. When a third party requests a proof that the document was published at a certain time and was not changed, one can point them to them to the message containing the `unsigned data payload` with the hash. +To make the Payload concept clear we will define the `unsigned data payload`. As the name suggests it simply allows to add arbitrary data to a message. A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. For example, one can record a hash of a text document on the tangle with such a payload. When a third party requests a proof that the document was published at a certain time and was not changed, one can point them to the message containing the `unsigned data payload` with the hash. The structure of the payload is simple: From a42644668bdba569e05a1a6d107c4288fc363c3c Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:36:24 +0300 Subject: [PATCH 14/52] Remove bound --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index b0bf827c8..3881a5900 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -123,7 +123,7 @@ A message is considered valid, if the following syntactic rules are met: So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes. * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid. 3. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. -4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need. +4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 5. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. ### Payloads From 5eabf5ebcdfb7745eb633b002b7f76c2e74f6955 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:37:27 +0300 Subject: [PATCH 15/52] unsigned data payload --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 3881a5900..b14cb3034 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -128,7 +128,7 @@ A message is considered valid, if the following syntactic rules are met: ### Payloads -A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `Unsigned Payload` will be specified here as an example. +A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `unsigned data payload` will be specified here as an example. | Payload Name | Type Value | | --------------------------------------- | ----------- | From 7b003e2fec4061a1765180e54b4a8b2b71dc4c0e Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 25 Aug 2020 12:51:50 +0300 Subject: [PATCH 16/52] Make capitalization consistent --- text/0017-message/0017-message.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index b14cb3034..547d5fbb2 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -19,7 +19,7 @@ By making it possible to add and exchange payloads, we are creating an architect # Detailed design -### Data Types +### Data types The following are data types that we will use when we specify fields in the message and payloads. @@ -35,7 +35,7 @@ The following are data types that we will use when we specify fields in the mess The message ID will be the `BLAKE2b-256` hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. -### Message Structure +### Message structure @@ -105,7 +105,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag
-### Message Validation +### Message validation A message is considered valid, if the following syntactic rules are met: @@ -138,7 +138,7 @@ A message may contain a payload. The specification of the payloads is out of sco | Signed Data | 3 | | Indexation | 4 | -### Unsigned Data Payload +### Unsigned data payload To make the Payload concept clear we will define the `unsigned data payload`. As the name suggests it simply allows to add arbitrary data to a message. A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. For example, one can record a hash of a text document on the tangle with such a payload. When a third party requests a proof that the document was published at a certain time and was not changed, one can point them to the message containing the `unsigned data payload` with the hash. From 385abff9c4843dcbcacdad64e49aea61cfeb40f5 Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 26 Aug 2020 15:47:39 +0300 Subject: [PATCH 17/52] Add serialization example --- text/0017-message/0017-message.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 547d5fbb2..e9a54eb3d 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -144,13 +144,21 @@ To make the Payload concept clear we will define the `unsigned data payload`. As The structure of the payload is simple: -| Name | Type | Description | -| -------- | -------- | ----------- | -| Payload Type | varint | Must be set to **2** | -| Data | ByteArray| The data we are attaching | +| Name | Type | Description | +| -------- | ----------- | ----------- | +| Payload Type | varint | Must be set to **2** | +| Data | ByteArray | The data we are attaching | There are no validation rules for the payload. Message validation rules suffice here. +### Serialization Example + +Below is a simple serialized message with the unsigned data payload that carries "Hello World" ASCII string. +Bytes are expressesd with hexadecimal numbers. + +[Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] +*`0C`*[Payload Type]**`02`**[Payload Data]`48656c6c6f20576f726c64`[Nonce]`24AFA821371892D1` + # Rationale and alternatives Instead of creating a layered approach, we could have simply created a flat `transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. From 35b86538cb50d3d8913a2f30a8c057ae1c0d1e80 Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 26 Aug 2020 15:59:17 +0300 Subject: [PATCH 18/52] add temporary links --- text/0017-message/0017-message.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index e9a54eb3d..796cca9b3 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -132,11 +132,11 @@ A message may contain a payload. The specification of the payloads is out of sco | Payload Name | Type Value | | --------------------------------------- | ----------- | -| Signed Transaction | 0 | -| Milestone | 1 | +| [Signed Transaction](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | +| [Milestone Draft](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | | [Unsigned Data](#Unsigned-Data-Payload) | 2 | -| Signed Data | 3 | -| Indexation | 4 | +| Signed Data (TBD) | 3 | +| Indexation (TBD) | 4 | ### Unsigned data payload From c7959805e3e155c76ab5a4a9b42baa2d15bf7800 Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 26 Aug 2020 16:20:25 +0300 Subject: [PATCH 19/52] fix serailization with byte array length --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 796cca9b3..b29c00755 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -157,7 +157,7 @@ Below is a simple serialized message with the unsigned data payload that carries Bytes are expressesd with hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`0C`*[Payload Type]**`02`**[Payload Data]`48656c6c6f20576f726c64`[Nonce]`24AFA821371892D1` +*`0C`*[Payload Type]**`02`**[PayloadData]*`0B`*`48656c6c6f20576f726c64`[Nonce]`24AFA821371892D1` # Rationale and alternatives From 54b0a0a695fc066143d65bd3a22104fece01cb9f Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 26 Aug 2020 16:52:10 +0300 Subject: [PATCH 20/52] remove nested payload --- text/0017-message/0017-message.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index b29c00755..e49640ccf 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -91,11 +91,6 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag ANY A sequence of fields, where the structure depends on payload type. - - Nested Payload (Optional) - Payload - Some payloads support nesting other payloads inside of them. - From 1150c720de6dd0a9c9a48ad38db0483f8e27db63 Mon Sep 17 00:00:00 2001 From: galrogo Date: Wed, 26 Aug 2020 18:13:06 +0300 Subject: [PATCH 21/52] Make serialization Pow Hash to have 9 trailing zeros --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index e49640ccf..9571b9751 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -152,7 +152,7 @@ Below is a simple serialized message with the unsigned data payload that carries Bytes are expressesd with hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`0C`*[Payload Type]**`02`**[PayloadData]*`0B`*`48656c6c6f20576f726c64`[Nonce]`24AFA821371892D1` +*`0C`*[Payload Type]**`02`**[PayloadData]*`0B`*`48656C6C6F20576F726C64`[Nonce]`513F000000000080` # Rationale and alternatives From 2a430970416dbd6ab241b6ccb11eff8d6c5b1366 Mon Sep 17 00:00:00 2001 From: galrogo Date: Sun, 30 Aug 2020 15:12:31 +0300 Subject: [PATCH 22/52] Add indexation payload --- text/0017-message/0017-message.md | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 9571b9751..80421baae 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -123,27 +123,37 @@ A message is considered valid, if the following syntactic rules are met: ### Payloads -A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `unsigned data payload` will be specified here as an example. +A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example. | Payload Name | Type Value | | --------------------------------------- | ----------- | | [Signed Transaction](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | | [Milestone Draft](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | -| [Unsigned Data](#Unsigned-Data-Payload) | 2 | -| Signed Data (TBD) | 3 | -| Indexation (TBD) | 4 | +| [Indexation Payload](#Unsigned-Data-Payload) | 2 | -### Unsigned data payload +### Indexation payload + +To make the Payload concept clear we will define the `indexation payload`. As the name suggests it allows to add an index to the encapsulating message, as well as some arbitrary data. Nodes will expose an API, that will enable to query messages by the index. +Adding those capabilities may open nodes to DOS attack vectors: +1. Proliferation of index keys that may blow up the node's DB +2. Proliferation of messages associated with the same index + +Node implementations may provide weak guarantees regarding the completion of indexes to address the above scenarios. + +Besides the index, the payload will also have a data field. + A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. If the payload will be incorporated under + the `signed transaction payload`, the content will be signed as well. -To make the Payload concept clear we will define the `unsigned data payload`. As the name suggests it simply allows to add arbitrary data to a message. A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. For example, one can record a hash of a text document on the tangle with such a payload. When a third party requests a proof that the document was published at a certain time and was not changed, one can point them to the message containing the `unsigned data payload` with the hash. The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | | Payload Type | varint | Must be set to **2** | -| Data | ByteArray | The data we are attaching | +| Index | ByteArray | The index key of the message | +| Data | ByteArray | Data we are attaching | +Note that `index` and `data` may both have a length of 0. There are no validation rules for the payload. Message validation rules suffice here. ### Serialization Example From cc15a5170879eafeef0b184d71a30f26f5ac4242 Mon Sep 17 00:00:00 2001 From: galrogo Date: Sun, 30 Aug 2020 17:46:48 +0300 Subject: [PATCH 23/52] Add indexation payload --- text/0017-message/0017-message.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 80421baae..abe09cac6 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -158,11 +158,12 @@ There are no validation rules for the payload. Message validation rules suffice ### Serialization Example -Below is a simple serialized message with the unsigned data payload that carries "Hello World" ASCII string. -Bytes are expressesd with hexadecimal numbers. +Below is a serialized valid message with the indexation payload. The index is the "SPAM" ASCII string and the message is the "Hello Iota" +ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) should have + 14 trailing zeroes. Bytes are expressed as hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`0C`*[Payload Type]**`02`**[PayloadData]*`0B`*`48656C6C6F20576F726C64`[Nonce]`513F000000000080` +*`0C`*[Payload Type]**`02`**[Index] *`04`*`9A52` [Data]*`0A`*`48656c6c6f20496f7461`[Nonce]`dc293f3333333333` # Rationale and alternatives From 3d92d83c1f1cf3ceac172f29c5cdd15531ece153 Mon Sep 17 00:00:00 2001 From: Luca Moser Date: Fri, 4 Sep 2020 16:07:41 +0200 Subject: [PATCH 24/52] use fixed size integers --- text/0017-message/0017-message.md | 35 +++++++++++-------------------- 1 file changed, 12 insertions(+), 23 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index abe09cac6..410439d8c 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -25,10 +25,11 @@ The following are data types that we will use when we specify fields in the mess | Name | Description | | ------ | ------------- | -| varint | An unsigned numerical value using a dynamic amount of bytes encoded in Little Endian. Evaluate by inspecting the MSB of the first byte. If it is `0` stop. If it is `1` go to the next byte. Continue applying this rule until you reach a byte with MSB `0`. Transform each byte to a 7-bit bit sequence by stripping the MSBs from each byte. Reverse the order of the 7 bit groups (Little Endian). Concatenate all the bits and evaluate as an unsigned integer. [It is similar to how it is done in Protocol Buffers](https://developers.google.com/protocol-buffers/docs/encoding). We must add [validation rules](#Message-Validation) that ensure that the varint is not malleable, and it is limited in size. | -| uint64 | An unsigned 64 bit integer encoded in Little Endian. -| ByteArray | A dynamic size byte array. It is a prefixed by a `varint` that indicates the number of bytes in the array. The following bytes are simply the content of the array. -| ByteArray[N] | A static size array of size N. Since the size is expected, no need to prefix with a varint | +| uint8 | An unsigned 8 bit integer encoded in Little Endian. | +| uint16 | An unsigned 16 bit integer encoded in Little Endian. | +| uint32 | An unsigned 32 bit integer encoded in Little Endian. | +| uint64 | An unsigned 64 bit integer encoded in Little Endian. | +| ByteArray[N] | A static size array of size N. | ### Message ID @@ -45,7 +46,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Version - varint + uint8 The message version. The schema specified in this RFC is for version 1 only. @@ -60,7 +61,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Payload Length - varint + uint32 The length of the Payload. Since its type may be unknown to the node it must be declared in advanced. 0 length means no payload will be attached. @@ -81,7 +82,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Payload Type - varint + uint16 The type of the payload. It will instruct the node how to parse the fields that follow. Types in the range of 0-127 are "core types" that all nodes are expected to know. @@ -105,21 +106,9 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag A message is considered valid, if the following syntactic rules are met: 1. The message length must not exceed X [tbd] bytes. -2. Varint fields must adhere to the following rules: - * *Malleability protection:* No `0x80` or `0x00` trailing bytes. Take for example the following 3 byte varint: `0x8F8000`. - In order to parse it we look at its bits: - `1000 1111 1000 0000 0000 0000`. - First we strip the msbs: - `000 1111 000 0000 000 0000` - Then we reverse the groups of 7 bits (Little Endian): - `000 0000 000 0000 000 1111` - Then we concatenate all the bits to finally get: - `1111` - So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes. - * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid. -3. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. -4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. -5. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. +2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. +3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. +4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. ### Payloads @@ -149,7 +138,7 @@ The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | -| Payload Type | varint | Must be set to **2** | +| Payload Type | uint16 | Must be set to **2** | | Index | ByteArray | The index key of the message | | Data | ByteArray | Data we are attaching | From 793378d586c55eea5691f3b714b88122ee0a41d1 Mon Sep 17 00:00:00 2001 From: Luca Moser Date: Mon, 7 Sep 2020 10:53:15 +0200 Subject: [PATCH 25/52] fixes payload type to uint32 --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 410439d8c..895f06e22 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -82,7 +82,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Payload Type - uint16 + uint32 The type of the payload. It will instruct the node how to parse the fields that follow. Types in the range of 0-127 are "core types" that all nodes are expected to know. @@ -138,7 +138,7 @@ The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | -| Payload Type | uint16 | Must be set to **2** | +| Payload Type | uint32 | Must be set to **2** | | Index | ByteArray | The index key of the message | | Data | ByteArray | Data we are attaching | From dcbf444133d46c78efd8cf8ed8c3c68b3b8eab83 Mon Sep 17 00:00:00 2001 From: Luca Moser Date: Mon, 7 Sep 2020 11:56:02 +0200 Subject: [PATCH 26/52] adds ByteArray/string type to types table --- text/0017-message/0017-message.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 895f06e22..486d74ed0 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -30,6 +30,8 @@ The following are data types that we will use when we specify fields in the mess | uint32 | An unsigned 32 bit integer encoded in Little Endian. | | uint64 | An unsigned 64 bit integer encoded in Little Endian. | | ByteArray[N] | A static size array of size N. | +| ByteArray | A dynamically sized array. A uint32 denotes its length. | +| string | A dynamically sized array of an UTF-8 encoded string. A uint16 denotes its length. | ### Message ID @@ -139,7 +141,7 @@ The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | | Payload Type | uint32 | Must be set to **2** | -| Index | ByteArray | The index key of the message | +| Index | string | The index key of the message | | Data | ByteArray | Data we are attaching | Note that `index` and `data` may both have a length of 0. From 8a69a4c7a92835207c9521c6b08e01947f0fea97 Mon Sep 17 00:00:00 2001 From: Gal Rogozinski Date: Thu, 29 Oct 2020 13:38:48 +0200 Subject: [PATCH 27/52] Fix link Co-authored-by: Sam Chen --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 486d74ed0..b4c29fd2f 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -120,7 +120,7 @@ A message may contain a payload. The specification of the payloads is out of sco | --------------------------------------- | ----------- | | [Signed Transaction](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | | [Milestone Draft](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | -| [Indexation Payload](#Unsigned-Data-Payload) | 2 | +| [Indexation Payload](#indexation-payload) | 2 | ### Indexation payload From 0e5d06025103d47f572870b429d08a0be0d5633a Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 10 Nov 2020 00:39:33 +0200 Subject: [PATCH 28/52] add network id --- text/0017-message/0017-message.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index b4c29fd2f..ce95b926c 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -51,6 +51,11 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag uint8 The message version. The schema specified in this RFC is for version 1 only. + + NetworkID + uint64 + Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. + Parent1 (trunk) ByteArray[32] From 98e74379bf1fc14560948146c36bf27729dee60c Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 10 Nov 2020 21:29:48 +0200 Subject: [PATCH 29/52] Fix serialization --- text/0017-message/0017-message.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index ce95b926c..e3c99bac9 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -155,11 +155,11 @@ There are no validation rules for the payload. Message validation rules suffice ### Serialization Example Below is a serialized valid message with the indexation payload. The index is the "SPAM" ASCII string and the message is the "Hello Iota" -ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) should have - 14 trailing zeroes. Bytes are expressed as hexadecimal numbers. +ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) would have + 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`0C`*[Payload Type]**`02`**[Index] *`04`*`9A52` [Data]*`0A`*`48656c6c6f20496f7461`[Nonce]`dc293f3333333333` +*`11`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`8379666666666666` # Rationale and alternatives From e323b187272c4a41bba3a05b49512acaf8dd0ea6 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 10 Nov 2020 21:53:17 +0200 Subject: [PATCH 30/52] Change indexation validation rule --- text/0017-message/0017-message.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index e3c99bac9..e6d10af98 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -149,8 +149,7 @@ The structure of the payload is simple: | Index | string | The index key of the message | | Data | ByteArray | Data we are attaching | -Note that `index` and `data` may both have a length of 0. -There are no validation rules for the payload. Message validation rules suffice here. +Note that `index` must be at least 1 byte long for the payload to be valid. The `data` may have a length of 0. ### Serialization Example From dd826e02dee774f2a10c47de57e654d8a67a87c1 Mon Sep 17 00:00:00 2001 From: galrogo Date: Mon, 16 Nov 2020 12:51:43 +0200 Subject: [PATCH 31/52] Change length to 32 bytes little endian --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index e6d10af98..5c0885e06 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -158,7 +158,7 @@ ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blo 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`11`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`8379666666666666` +*`1100000000000000000000000000000000000000000000000000000000000000`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`8379666666666666` # Rationale and alternatives From 5843c38a5c2e11144ff00b06139d2bd685165333 Mon Sep 17 00:00:00 2001 From: galrogo Date: Mon, 16 Nov 2020 16:43:36 +0200 Subject: [PATCH 32/52] Remove version and fix network identifier --- text/0017-message/0017-message.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 5c0885e06..351444951 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -46,15 +46,10 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Type Description - - Version - uint8 - The message version. The schema specified in this RFC is for version 1 only. - NetworkID uint64 - Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. + Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It will usually be calculated as the hash of the concatenation of the network type and the protocol version. Parent1 (trunk) From d3a46d9f515aedd440405a591aabe38f4fc4d4f2 Mon Sep 17 00:00:00 2001 From: galrogo Date: Mon, 16 Nov 2020 20:02:56 +0200 Subject: [PATCH 33/52] change nonce --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 351444951..5d7e22106 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -153,7 +153,7 @@ ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blo 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`1100000000000000000000000000000000000000000000000000000000000000`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`8379666666666666` +*`1100000000000000000000000000000000000000000000000000000000000000`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`5d38333333333333` # Rationale and alternatives From d650bb8db91b46f4528befc14b321636a86238fe Mon Sep 17 00:00:00 2001 From: Gal Rogozinski Date: Wed, 18 Nov 2020 11:10:44 +0200 Subject: [PATCH 34/52] Apply suggestions from technical writer Co-authored-by: charlesthompson3 <74603461+charlesthompson3@users.noreply.github.com> --- text/0017-message/0017-message.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 5d7e22106..6ef303928 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -6,16 +6,16 @@ A message is the object nodes gossip around in the network. It always references two other messages that are known as `parents`. It is stored as a vertex on the tangle data structure that the nodes maintain. -The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable to build new functionality on top of the tangle. Some payloads may have other nested payloads embedded inside. -So upon parsing it is done layer by layer. +The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. +So upon parsing, it is done layer by layer. # Motivation -To better understand this layered design, consider the internet protocol for example. There is an Ethernet frame, that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility, once this responsibility is completed, we move on to the next layer. +To better understand this layered design, consider the internet protocol, for example: there is an Ethernet frame that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility and once this responsibility is completed, we move on to the next layer. -The same goes with how we parse messages. The outer layer of the message enables us to map the message to a vertex in the Tangle and perform some basic validation. The next layer may be a transaction that mutates the ledger state. The next layer may provide some extra functionality on the transactions to be used by applications. +The same is true with how messages are parsed. The outer layer of the message enables the mapping of the message to a vertex in the Tangle and performing some basic validation. The next layer may be a transaction that mutates the ledger state, and the next layer may provide some extra functionality on the transactions to be used by applications. -By making it possible to add and exchange payloads, we are creating an architecture that can be easily extended to accommodate future needs. +By making it possible to add and exchange payloads, an architecture is being created that can be easily extended to accommodate future needs. # Detailed design @@ -108,13 +108,13 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag A message is considered valid, if the following syntactic rules are met: 1. The message length must not exceed X [tbd] bytes. -2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed. +2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. ### Payloads -A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example. +A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example: | Payload Name | Type Value | | --------------------------------------- | ----------- | @@ -124,7 +124,7 @@ A message may contain a payload. The specification of the payloads is out of sco ### Indexation payload -To make the Payload concept clear we will define the `indexation payload`. As the name suggests it allows to add an index to the encapsulating message, as well as some arbitrary data. Nodes will expose an API, that will enable to query messages by the index. +To be clear, the concept of the Payload allows the addition of an index to the encapsulating message, as well as some arbitrary data. Nodes will expose an API that will enable the querying of messages by the index. Adding those capabilities may open nodes to DOS attack vectors: 1. Proliferation of index keys that may blow up the node's DB 2. Proliferation of messages associated with the same index @@ -132,7 +132,7 @@ Adding those capabilities may open nodes to DOS attack vectors: Node implementations may provide weak guarantees regarding the completion of indexes to address the above scenarios. Besides the index, the payload will also have a data field. - A message that has been attached to the tangle and approved by a milestone has useful properties: You can verify that the content of the data did not change, and you can ascertain the approximate time it was published by checking the approving milestone. If the payload will be incorporated under + A message that has been attached to the Tangle and approved by a milestone has several useful properties: verifying that the content of the data did not change and determining the approximate time it was published by checking the approving milestone. If the payload will be incorporated under the `signed transaction payload`, the content will be signed as well. @@ -157,7 +157,7 @@ ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blo # Rationale and alternatives -Instead of creating a layered approach, we could have simply created a flat `transaction message`, that is tailored for mutating the ledger state, and try to fit all the use cases there. For example with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. +Instead of creating a layered approach, we could have simply created a flat `transaction message` that is tailored for mutating the ledger state, and try to fit all the use cases there. For example, with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction, we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. From ecb8b66b7190f1acbf9e14efcfbd637e3ce7199b Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 24 Nov 2020 13:36:15 +0200 Subject: [PATCH 35/52] change index validation criteria --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 6ef303928..c42de4867 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -144,7 +144,7 @@ The structure of the payload is simple: | Index | string | The index key of the message | | Data | ByteArray | Data we are attaching | -Note that `index` must be at least 1 byte long for the payload to be valid. The `data` may have a length of 0. +Note that `index` field should be 1 to 64 bytes long for the payload to be valid. The `data` may have a length of 0. ### Serialization Example From 92a2f730d85b038ae37a67b95edb3767d57c7c26 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 24 Nov 2020 21:53:30 +0200 Subject: [PATCH 36/52] Use blake2b for networkId --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index c42de4867..1394be1bf 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -49,7 +49,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag NetworkID uint64 - Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It will usually be calculated as the hash of the concatenation of the network type and the protocol version. + Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the `BLAKE2b-256` hash of the concatenation of the network type and the protocol version string. Parent1 (trunk) From 3b330c1c22fa0e020715e4a93fe362a526f613f6 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 24 Nov 2020 21:55:08 +0200 Subject: [PATCH 37/52] message must not be more than 32 kb --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 1394be1bf..5e5e8802a 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -107,7 +107,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag A message is considered valid, if the following syntactic rules are met: -1. The message length must not exceed X [tbd] bytes. +1. The message size must not exceed 32 KB. 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. From f4f94cd90cbc2e1a46c03a0a01e13859fa4d688e Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 12 Jan 2021 22:43:49 +0200 Subject: [PATCH 38/52] Fix max message size and index encoding --- text/0017-message/0017-message.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 5e5e8802a..0db98863e 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -107,7 +107,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag A message is considered valid, if the following syntactic rules are met: -1. The message size must not exceed 32 KB. +1. The message size must not exceed 32 KiB (32 * 1024 bytes). 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. @@ -141,10 +141,11 @@ The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | | Payload Type | uint32 | Must be set to **2** | -| Index | string | The index key of the message | +| Index | string | The index key of the message, a UTF-8 encoded string | | Data | ByteArray | Data we are attaching | -Note that `index` field should be 1 to 64 bytes long for the payload to be valid. The `data` may have a length of 0. +Note that `index` field should be 1 to 64 bytes long for the payload to be valid. If `index` can't be decoded to a UTF-8 string, then the payload is invalid. The `data` may have a length of 0. + ### Serialization Example From 4e9ecdd37ceceddaa7c51fee5066ebb1ed58b18f Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 12 Jan 2021 22:56:01 +0200 Subject: [PATCH 39/52] Add number of parents --- text/0017-message/0017-message.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 0db98863e..d9df800d1 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -52,14 +52,14 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the `BLAKE2b-256` hash of the concatenation of the network type and the protocol version string. - Parent1 (trunk) - ByteArray[32] - The Message ID of the first Message it references. + Parents' length + uint8 + The number of messages we directly approve. Currently we fix this value to 2, or else the message is invalid. - Parent2 (branch) - ByteArray[32] - The Message ID of the second Message it references. + Parents + ByteArray[32 * `parents length`] + The Message IDs that are referenced. Payload Length @@ -111,6 +111,8 @@ A message is considered valid, if the following syntactic rules are met: 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. +5. `Parents' length` must be 2. + ### Payloads From f1b5775bdcf0a5c4a1e2335b7801f418bbc55183 Mon Sep 17 00:00:00 2001 From: galrogo Date: Thu, 18 Feb 2021 15:08:57 +0200 Subject: [PATCH 40/52] Index key is now a byte array --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index d9df800d1..bf2b7d12c 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -143,10 +143,10 @@ The structure of the payload is simple: | Name | Type | Description | | -------- | ----------- | ----------- | | Payload Type | uint32 | Must be set to **2** | -| Index | string | The index key of the message, a UTF-8 encoded string | +| Index | ByteArray | The index key of the message | | Data | ByteArray | Data we are attaching | -Note that `index` field should be 1 to 64 bytes long for the payload to be valid. If `index` can't be decoded to a UTF-8 string, then the payload is invalid. The `data` may have a length of 0. +Note that `index` field should be 1 to 64 bytes long for the payload to be valid. The `data` may have a length of 0. ### Serialization Example From 466d32825cf3cdd907bf6afea8d02de5c48e565d Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 2 Mar 2021 14:51:56 +0200 Subject: [PATCH 41/52] change parents rule to be between 1-8 --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index bf2b7d12c..74cbbf3f2 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -54,7 +54,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Parents' length uint8 - The number of messages we directly approve. Currently we fix this value to 2, or else the message is invalid. + The number of messages we directly approve. Can be any value between 1-8. Parents @@ -111,7 +111,7 @@ A message is considered valid, if the following syntactic rules are met: 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. -5. `Parents' length` must be 2. +5. `Parents' length` must be between 1-8. ### Payloads From d4eea485f8a30c45faa400b17737de8ada7bc214 Mon Sep 17 00:00:00 2001 From: galrogo Date: Tue, 9 Mar 2021 14:38:53 +0200 Subject: [PATCH 42/52] remove core payload --- text/0017-message/0017-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 74cbbf3f2..9085f17a4 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -86,7 +86,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Payload Type uint32 - The type of the payload. It will instruct the node how to parse the fields that follow. Types in the range of 0-127 are "core types" that all nodes are expected to know. + The type of the payload. It will instruct the node how to parse the fields that follow. @@ -109,7 +109,7 @@ A message is considered valid, if the following syntactic rules are met: 1. The message size must not exceed 32 KiB (32 * 1024 bytes). 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. -3. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is above this range. +3. If the `payload type` is known to the node. 4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. 5. `Parents' length` must be between 1-8. From 2ef7705dc31373fe0ec7f58de0b70983203dddbf Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 11:46:10 +0200 Subject: [PATCH 43/52] Update 0017-message.md --- text/0017-message/0017-message.md | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 9085f17a4..520dfbd98 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -52,13 +52,13 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the `BLAKE2b-256` hash of the concatenation of the network type and the protocol version string. - Parents' length + Parents Count uint8 The number of messages we directly approve. Can be any value between 1-8. Parents - ByteArray[32 * `parents length`] + ByteArray[32 * Parents Count] The Message IDs that are referenced. @@ -110,8 +110,8 @@ A message is considered valid, if the following syntactic rules are met: 1. The message size must not exceed 32 KiB (32 * 1024 bytes). 2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. 3. If the `payload type` is known to the node. -4. If the [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) will contain at least the number of trailing 0 trits the node defines as required. -5. `Parents' length` must be between 1-8. +4. If the message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. +5. `Parents Count` must be between 1-8. ### Payloads @@ -120,8 +120,8 @@ A message may contain a payload. The specification of the payloads is out of sco | Payload Name | Type Value | | --------------------------------------- | ----------- | -| [Signed Transaction](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | -| [Milestone Draft](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | +| [Transaction RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | +| [Milestone RFC-0019 (draft)](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | | [Indexation Payload](#indexation-payload) | 2 | ### Indexation payload @@ -152,8 +152,7 @@ Note that `index` field should be 1 to 64 bytes long for the payload to be valid ### Serialization Example Below is a serialized valid message with the indexation payload. The index is the "SPAM" ASCII string and the message is the "Hello Iota" -ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blob/message-pow/text/0024-message-pow/0024-message-pow.md) would have - 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. +ASCII string. The message PoW Hash would have 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. [Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] *`1100000000000000000000000000000000000000000000000000000000000000`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`5d38333333333333` @@ -163,7 +162,3 @@ ASCII string. The [Message PoW Hash](https://github.com/Wollac/protocol-rfcs/blo Instead of creating a layered approach, we could have simply created a flat `transaction message` that is tailored for mutating the ledger state, and try to fit all the use cases there. For example, with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction, we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. - -# Unresolved questions - -- What should be the maximum length of the message? From e5d02e29e4a66461b453ab91907c23ce7c26f6e7 Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 12:02:57 +0200 Subject: [PATCH 44/52] Update 0017-message.md --- text/0017-message/0017-message.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 520dfbd98..8c6b4dee4 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -6,7 +6,7 @@ A message is the object nodes gossip around in the network. It always references two other messages that are known as `parents`. It is stored as a vertex on the tangle data structure that the nodes maintain. -The messages will contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. +The messages contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. So upon parsing, it is done layer by layer. # Motivation @@ -108,15 +108,15 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag A message is considered valid, if the following syntactic rules are met: 1. The message size must not exceed 32 KiB (32 * 1024 bytes). -2. When parsing the message is complete, there should not be any trailing bytes left that were not parsed. -3. If the `payload type` is known to the node. -4. If the message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. -5. `Parents Count` must be between 1-8. +2. When parsing the message is complete, there must not be any trailing bytes left that were not parsed. +3. The optional `payload type` is known to the node. +4. The message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. +5. The `Parents Count` is between 1 and 8. ### Payloads -A message may contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example: +While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example: | Payload Name | Type Value | | --------------------------------------- | ----------- | From 1ba4de553392d1381f9abd401b661a3a1af8d757 Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 12:07:52 +0200 Subject: [PATCH 45/52] Update 0017-message.md --- text/0017-message/0017-message.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 8c6b4dee4..7eccb7f92 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -117,12 +117,11 @@ A message is considered valid, if the following syntactic rules are met: ### Payloads While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example: - -| Payload Name | Type Value | -| --------------------------------------- | ----------- | -| [Transaction RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | 0 | -| [Milestone RFC-0019 (draft)](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | 1 | -| [Indexation Payload](#indexation-payload) | 2 | +| Payload Name | Type Value | RFC | +| ------------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Transaction | 0 | [RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | +| Milestone | 1 | [RFC-0019 (draft)](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | +| Indexation | 2 | [RFC-0017](#indexation-payload) | ### Indexation payload From d479e9ee8f94d52be8a3bb90e92f81e4936ce9ab Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 12:12:52 +0200 Subject: [PATCH 46/52] Update 0017-message.md --- text/0017-message/0017-message.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 7eccb7f92..2ea2f054d 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -4,7 +4,7 @@ # Summary -A message is the object nodes gossip around in the network. It always references two other messages that are known as `parents`. It is stored as a vertex on the tangle data structure that the nodes maintain. +A message is the object nodes gossip around in the network. It always references two other messages that are known as _parents_. It is stored as a vertex on the tangle data structure that the nodes maintain. The messages contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. So upon parsing, it is done layer by layer. @@ -35,7 +35,7 @@ The following are data types that we will use when we specify fields in the mess ### Message ID -The message ID will be the `BLAKE2b-256` hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. +The message ID will be the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. ### Message structure @@ -49,7 +49,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag NetworkID uint64 - Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the `BLAKE2b-256` hash of the concatenation of the network type and the protocol version string. + Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the BLAKE2b-256 hash of the concatenation of the network type and the protocol version string. Parents Count @@ -92,7 +92,7 @@ The message ID will be the `BLAKE2b-256` hash of the byte contents of the messag Data Fields ANY - A sequence of fields, where the structure depends on payload type. + A sequence of fields, where the structure depends on Payload Type. @@ -109,14 +109,14 @@ A message is considered valid, if the following syntactic rules are met: 1. The message size must not exceed 32 KiB (32 * 1024 bytes). 2. When parsing the message is complete, there must not be any trailing bytes left that were not parsed. -3. The optional `payload type` is known to the node. +3. The optional `Payload Type` is known to the node. 4. The message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. 5. The `Parents Count` is between 1 and 8. ### Payloads -While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The `indexation payload` will be specified here as an example: +While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The _indexation payload_ will be specified here as an example: | Payload Name | Type Value | RFC | | ------------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Transaction | 0 | [RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | @@ -134,7 +134,7 @@ Node implementations may provide weak guarantees regarding the completion of ind Besides the index, the payload will also have a data field. A message that has been attached to the Tangle and approved by a milestone has several useful properties: verifying that the content of the data did not change and determining the approximate time it was published by checking the approving milestone. If the payload will be incorporated under - the `signed transaction payload`, the content will be signed as well. + the signed _transaction payload_, the content will be signed as well. The structure of the payload is simple: @@ -145,7 +145,7 @@ The structure of the payload is simple: | Index | ByteArray | The index key of the message | | Data | ByteArray | Data we are attaching | -Note that `index` field should be 1 to 64 bytes long for the payload to be valid. The `data` may have a length of 0. +Note that `Index` field should be 1 to 64 bytes long for the payload to be valid. The `Data` may have a length of 0. ### Serialization Example @@ -158,6 +158,6 @@ ASCII string. The message PoW Hash would have 10 trailing zeroes for the given n # Rationale and alternatives -Instead of creating a layered approach, we could have simply created a flat `transaction message` that is tailored for mutating the ledger state, and try to fit all the use cases there. For example, with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction, we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. +Instead of creating a layered approach, we could have simply created a flat transaction message that is tailored for mutating the ledger state, and try to fit all the use cases there. For example, with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction, we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. From c3ace558f6e47781ed70eca454533424d4a62a60 Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 12:26:22 +0200 Subject: [PATCH 47/52] Clarify ranges --- text/0017-message/0017-message.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 2ea2f054d..8cff6de30 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -54,17 +54,17 @@ The message ID will be the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) ha Parents Count uint8 - The number of messages we directly approve. Can be any value between 1-8. + The number of messages that are directly approved. Parents ByteArray[32 * Parents Count] - The Message IDs that are referenced. + The IDs of the messages that are directly approved. Payload Length uint32 - The length of the Payload. Since its type may be unknown to the node it must be declared in advanced. 0 length means no payload will be attached. + The length of the Payload. A length of 0 means no payload will be attached. @@ -108,11 +108,10 @@ The message ID will be the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) ha A message is considered valid, if the following syntactic rules are met: 1. The message size must not exceed 32 KiB (32 * 1024 bytes). -2. When parsing the message is complete, there must not be any trailing bytes left that were not parsed. -3. The optional `Payload Type` is known to the node. -4. The message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. -5. The `Parents Count` is between 1 and 8. - +2. The `Parents Count` is at least 1 and not larger than 8. +3. When parsing the message is complete, there must not be any trailing bytes left that were not parsed. +4. The optional `Payload Type` is known to the node. +5. The message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. ### Payloads @@ -145,7 +144,7 @@ The structure of the payload is simple: | Index | ByteArray | The index key of the message | | Data | ByteArray | Data we are attaching | -Note that `Index` field should be 1 to 64 bytes long for the payload to be valid. The `Data` may have a length of 0. +Note that `Index` field must be at least 1 byte and not longer than 64 bytes for the payload to be valid. The `Data` may have a length of 0. ### Serialization Example From 516846142ee5a46955e76038c6aeac90eeb88c67 Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 12:29:47 +0200 Subject: [PATCH 48/52] Update 0017-message.md --- text/0017-message/0017-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md index 8cff6de30..d5662a7ca 100644 --- a/text/0017-message/0017-message.md +++ b/text/0017-message/0017-message.md @@ -4,7 +4,7 @@ # Summary -A message is the object nodes gossip around in the network. It always references two other messages that are known as _parents_. It is stored as a vertex on the tangle data structure that the nodes maintain. +A message is the object nodes gossip around in the network. It always references other messages that are known as _parents_. The messages contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. So upon parsing, it is done layer by layer. From f641685d773405020b3f8802ec083ac86ded907c Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 19:28:46 +0200 Subject: [PATCH 49/52] RFC cleanup --- text/0017-message/0017-message.md | 162 -------------------- text/0017-message/0017-tangle-message.md | 186 +++++++++++++++++++++++ 2 files changed, 186 insertions(+), 162 deletions(-) delete mode 100644 text/0017-message/0017-message.md create mode 100644 text/0017-message/0017-tangle-message.md diff --git a/text/0017-message/0017-message.md b/text/0017-message/0017-message.md deleted file mode 100644 index d5662a7ca..000000000 --- a/text/0017-message/0017-message.md +++ /dev/null @@ -1,162 +0,0 @@ -+ Feature name: `tangle-message` -+ Start date: 2020-07-28 -+ RFC PR: [iotaledger/protocol-rfcs#0017](https://github.com/iotaledger/protocol-rfcs/pull/0017) - -# Summary - -A message is the object nodes gossip around in the network. It always references other messages that are known as _parents_. - -The messages contain payloads. Some of them will be core payloads that will be processed by all nodes as part of the core protocol. Some of them will be community payloads that will enable the building of new functionalities on top of the Tangle. Some payloads may have other nested payloads embedded inside. -So upon parsing, it is done layer by layer. - -# Motivation - -To better understand this layered design, consider the internet protocol, for example: there is an Ethernet frame that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility and once this responsibility is completed, we move on to the next layer. - -The same is true with how messages are parsed. The outer layer of the message enables the mapping of the message to a vertex in the Tangle and performing some basic validation. The next layer may be a transaction that mutates the ledger state, and the next layer may provide some extra functionality on the transactions to be used by applications. - -By making it possible to add and exchange payloads, an architecture is being created that can be easily extended to accommodate future needs. - -# Detailed design - -### Data types - -The following are data types that we will use when we specify fields in the message and payloads. - -| Name | Description | -| ------ | ------------- | -| uint8 | An unsigned 8 bit integer encoded in Little Endian. | -| uint16 | An unsigned 16 bit integer encoded in Little Endian. | -| uint32 | An unsigned 32 bit integer encoded in Little Endian. | -| uint64 | An unsigned 64 bit integer encoded in Little Endian. | -| ByteArray[N] | A static size array of size N. | -| ByteArray | A dynamically sized array. A uint32 denotes its length. | -| string | A dynamically sized array of an UTF-8 encoded string. A uint16 denotes its length. | - - -### Message ID -The message ID will be the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the byte contents of the message. It should be used by the nodes to index the messages and by external APIs. - - -### Message structure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NameTypeDescription
NetworkIDuint64Network identifier. This field will signify whether this message was meant for mainnet, testnet, or a private net. It also tells what protocol rules apply to the message. It is first 8 bytes of the BLAKE2b-256 hash of the concatenation of the network type and the protocol version string.
Parents Count uint8 The number of messages that are directly approved.
Parents ByteArray[32 * Parents Count]The IDs of the messages that are directly approved.
Payload Lengthuint32 The length of the Payload. A length of 0 means no payload will be attached.
- Payload - -
- Generic Payload -
- An outline of a general payload -
- - - - - - - - - - - - - - - - -
NameTypeDescription
Payload Typeuint32 - The type of the payload. It will instruct the node how to parse the fields that follow. -
Data FieldsANYA sequence of fields, where the structure depends on Payload Type.
-
-
Nonceuint64The nonce which lets this message fulfill the Proof-of-Work requirement.
- -### Message validation - -A message is considered valid, if the following syntactic rules are met: - -1. The message size must not exceed 32 KiB (32 * 1024 bytes). -2. The `Parents Count` is at least 1 and not larger than 8. -3. When parsing the message is complete, there must not be any trailing bytes left that were not parsed. -4. The optional `Payload Type` is known to the node. -5. The message PoW score (as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md)) is not less than the configured threshold. - -### Payloads - -While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The specification of the payloads is out of scope of this RFC. Below is a table of the currently specified core payloads with a link to their specifications. The _indexation payload_ will be specified here as an example: -| Payload Name | Type Value | RFC | -| ------------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Transaction | 0 | [RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | -| Milestone | 1 | [RFC-0019 (draft)](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | -| Indexation | 2 | [RFC-0017](#indexation-payload) | - -### Indexation payload - -To be clear, the concept of the Payload allows the addition of an index to the encapsulating message, as well as some arbitrary data. Nodes will expose an API that will enable the querying of messages by the index. -Adding those capabilities may open nodes to DOS attack vectors: -1. Proliferation of index keys that may blow up the node's DB -2. Proliferation of messages associated with the same index - -Node implementations may provide weak guarantees regarding the completion of indexes to address the above scenarios. - -Besides the index, the payload will also have a data field. - A message that has been attached to the Tangle and approved by a milestone has several useful properties: verifying that the content of the data did not change and determining the approximate time it was published by checking the approving milestone. If the payload will be incorporated under - the signed _transaction payload_, the content will be signed as well. - - -The structure of the payload is simple: - -| Name | Type | Description | -| -------- | ----------- | ----------- | -| Payload Type | uint32 | Must be set to **2** | -| Index | ByteArray | The index key of the message | -| Data | ByteArray | Data we are attaching | - -Note that `Index` field must be at least 1 byte and not longer than 64 bytes for the payload to be valid. The `Data` may have a length of 0. - - -### Serialization Example - -Below is a serialized valid message with the indexation payload. The index is the "SPAM" ASCII string and the message is the "Hello Iota" -ASCII string. The message PoW Hash would have 10 trailing zeroes for the given nonce in this example. Bytes are expressed as hexadecimal numbers. - -[Version] **`01`** [Parent 1] `F532A53545103276B46876C473846D98648EE418468BCE76DF4868648DD73E5D` [Parent 2] `78D546B46AEC4557872139A48F66BC567687E8413578A14323548732358914A2` [Payload Length] -*`1100000000000000000000000000000000000000000000000000000000000000`*[Payload Type]**`02`**[Index] *`04`*`5350414D` [Data]*`0A`*`48656C6C6F20496F7461`[Nonce]`5d38333333333333` - -# Rationale and alternatives - -Instead of creating a layered approach, we could have simply created a flat transaction message that is tailored for mutating the ledger state, and try to fit all the use cases there. For example, with the unsigned data use-case, we could have filled some section of the transaction with the data. Then via a flag in the transaction, we could have instructed to not pass this transaction to the service that attempts to mutate the ledger state. - -This approach seems less extensible. It might have made sense if we wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. diff --git a/text/0017-message/0017-tangle-message.md b/text/0017-message/0017-tangle-message.md new file mode 100644 index 000000000..ebae58045 --- /dev/null +++ b/text/0017-message/0017-tangle-message.md @@ -0,0 +1,186 @@ ++ Feature name: `tangle-message` ++ Start date: 2020-07-28 ++ RFC PR: [iotaledger/protocol-rfcs#0017](https://github.com/iotaledger/protocol-rfcs/pull/0017) + +# Summary + +The Tangle is the graph data structure behind IOTA. In the current IOTA protocol, the vertices of the Tangle are represented by transactions. This document proposes an abstraction of this idea where the vertices are generalized *messages*, which then contain the transactions or other structures that are processed by the IOTA protocol. Just as before, each message directly approves other messages, which are known as _parents_. + +The messages can contain payloads. These are core payloads that will be processed by all nodes as part of the IOTA protocol. Some payloads may have other nested payloads embedded inside. Hence, parsing is done layer by layer. + +# Motivation + +To better understand this layered design, consider the Internet Protocol (IP), for example: There is an Ethernet frame that contains an IP payload. This in turn contains a TCP packet that encapsulates an HTTP payload. Each layer has a certain responsibility and once this responsibility is completed, we move on to the next layer. + +The same is true with how messages are parsed. The outer layer of the message enables the mapping of the message to a vertex in the Tangle and allow us to perform some basic validation. The next layer may be a transaction that mutates the ledger state, and one layer further may provide some extra functionality on the transactions to be used by applications. + +By making it possible to add and exchange payloads, an architecture is being created that can easily be extended to accommodate future needs. + +# Detailed design + +## Data types + +The following are data types that will be used when we specify fields in the message and payloads. + +| Name | Description | +| ------------ | ----------------------------------------------------------------------------- | +| uint8 | An unsigned 8-bit integer encoded in Little Endian. | +| uint16 | An unsigned 16-bit integer encoded in Little Endian. | +| uint32 | An unsigned 32-bit integer encoded in Little Endian. | +| uint64 | An unsigned 64-bit integer encoded in Little Endian. | +| ByteArray[N] | A static size byte array of length N. | +| ByteArray | A dynamically sized byte array. A leading uint32 denotes its length in bytes. | + +## Structure + +### Message ID + +The *Message ID* is the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the entire serialized message. + +### Serialized Layout + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Network IDuint64Network identifier. This field denotes whether the message was meant for mainnet, testnet, or a private net. It also marks what protocol rules apply to the message. Usually, it will be set to the first 8 bytes of the BLAKE2b-256 hash of the concatenation of the network type and the protocol version string.
Parents Countuint8The number of messages that are directly approved.
Parents anyOf +
+ Parent +
+ References another directly approved message. +
+ + + + + + + + + + + +
NameTypeDescription
Message IDByteArray[32]The Message ID of the parent.
+
+
Payload Lengthuint32The length of the following payload in bytes. A length of 0 means no payload will be attached.
Payload oneOf +
+ Generic Payload +
+ An outline of a generic payload +
+ + + + + + + + + + + + + + + + +
NameTypeDescription
Payload Typeuint32 + The type of the payload. It will instruct the node how to parse the fields that follow. +
Data FieldsANYA sequence of fields, where the structure depends on Payload Type.
+
+
Nonceuint64The nonce which lets this message fulfill the PoW requirement.
+ + +## Message validation + +The following criteria defines whether the message passes the syntactical validation: + +- The total message size must not exceed 32 KiB (32 * 1024 bytes). +- Parents: + - `Parents Count` must be at least 1 and not larger than 8. + - `Parents` must be sorted in lexicographical order. + - Each `Message ID` must be unique. +- Payload (if present): + - `Payload Type` must match one of the values described under [Payloads](#Payloads). + - `Data fields` must be correctly parsable in the context of the `Payload Type`. + - The payload itself must pass syntactic validation. +- `Nonce` must be a valid solution of the message PoW as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md). +- There must be no trailing bytes after all message fields have been parsed. + +## Payloads + +While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The detailed specification of each payload type is out of scope of this RFC. The following table lists all currently specified payloads that can be part of a message and links to their specification. The _indexation payload_ will be specified here as an example: + +| Payload Name | Type Value | RFC | +| ------------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Transaction | 0 | [RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | +| Milestone | 1 | [RFC-0019 (draft)](https://github.com/jakubcech/protocol-rfcs/blob/jakubcech-milestonepayload/text/0019-milestone-payload/0019-milestone-payload.md) | +| Indexation | 2 | [RFC-0017](#Indexation-payload) | + +### Indexation payload + +This payload allows the addition of an index to the encapsulating message, as well as some arbitrary data. Nodes will expose an API that allows to query messages by index. + +The structure of the indexation payload is as follows: + +| Name | Type | Description | +| ------------ | ----------------------- | ------------------------------------------------------------- | +| Payload Type | uint32 | Set to value 2 to denote an Indexation Payload. | +| Index Length | uint16 | The length of the following index field in bytes. | +| Index | ByteArray[Index Length] | The index key of the message | +| Data | ByteArray | Binary data. | + +Note that `Index` field must be at least 1 byte and not longer than 64 bytes for the payload to be valid. The `Data` may have a length of 0. + +## Example + +Below is the full serialization of a valid message with an indexation payload. The index is the "IOTA" ASCII string and the data is the "hello world" ASCII string. Bytes are expressed as hexadecimal numbers. + +- Network ID (8-byte): `0000000000000000` (0) +- Parents Count (1-byte): `02` (2) +- Parents (64-byte): + - `210fc7bb818639ac48a4c6afa2f1581a8b9525e20fda68927f2b2ff836f73578` + - `db0fa54c29f7fd928d92ca43f193dee47f591549f597a811c8fa67ab031ebd9c` +- Payload Length (4-byte): `19000000` (25) +- Payload (25-byte): + - Payload Type (4-byte): `02000000` (2) + - Index Length (2-byte): `0400` (4) + - Index (4-byte): `494f5441` ("IOTA") + - Data (15-byte): + - Length (4-byte): `0b000000` + - Data (11-byte): `68656c6c6f20776f726c64` ("hello world") +- Nonce (8-byte): `ce6d000000000000` (28110) + +# Rationale and alternatives + +Instead of creating a layered approach, we could have simply created a flat transaction message that is tailored to mutate the ledger state, and try to fit all the use cases there. For example, with the indexed data use case, we could have filled some section of the transaction with that particular data. Then, this transaction would not correspond to a ledger mutation but instead only carry data. + +This approach seems less extensible. It might have made sense if we had wanted to build a protocol that is just for ledger mutating transactions, but we want to be able to extend the protocol to do more than that. From b687e55e761ac0149189f58521765467e9e670aa Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 19:35:27 +0200 Subject: [PATCH 50/52] use correct links --- text/0017-message/0017-tangle-message.md | 2 +- text/0024-message-pow/0024-message-pow.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0017-message/0017-tangle-message.md b/text/0017-message/0017-tangle-message.md index ebae58045..3c572b4b8 100644 --- a/text/0017-message/0017-tangle-message.md +++ b/text/0017-message/0017-tangle-message.md @@ -132,7 +132,7 @@ The following criteria defines whether the message passes the syntactical valida - `Payload Type` must match one of the values described under [Payloads](#Payloads). - `Data fields` must be correctly parsable in the context of the `Payload Type`. - The payload itself must pass syntactic validation. -- `Nonce` must be a valid solution of the message PoW as described in [RFC-0024](https://github.com/iotaledger/protocol-rfcs/blob/master/text/0024-message-pow/0024-message-pow.md). +- `Nonce` must be a valid solution of the message PoW as described in [RFC-0024](https://iotaledger.github.io/protocol-rfcs/0024-message-pow/0024-message-pow.html). - There must be no trailing bytes after all message fields have been parsed. ## Payloads diff --git a/text/0024-message-pow/0024-message-pow.md b/text/0024-message-pow/0024-message-pow.md index 715f8d75b..4c66b3345 100644 --- a/text/0024-message-pow/0024-message-pow.md +++ b/text/0024-message-pow/0024-message-pow.md @@ -19,7 +19,7 @@ It will be easy to adapt existing hardware and software implementations of the c The PoW score is defined as the average number of iterations required to find the number of trailing zero trits in the hash divided by the message size. The PoW validation is performed in the following way: - - Compute the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the serialized message (as described in [Draft RFC-17](https://github.com/GalRogozinski/protocol-rfcs/blob/message/text/0017-message/0017-message.md)) *excluding* the 8-byte `Nonce` field and convert the hash into its 192-trit `b1t6` encoding. (See [RFC-15](https://iotaledger.github.io/protocol-rfcs/0015-binary-to-ternary-encoding/0015-binary-to-ternary-encoding.html) for a description of the encoding.) + - Compute the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the serialized message (as described in [RFC-0017](https://iotaledger.github.io/protocol-rfcs/0017-tangle-message/0017-tangle-message.html)) *excluding* the 8-byte `Nonce` field and convert the hash into its 192-trit `b1t6` encoding. (See [RFC-0015](https://iotaledger.github.io/protocol-rfcs/0015-binary-to-ternary-encoding/0015-binary-to-ternary-encoding.html) for a description of the encoding.) - Take the 8-byte `Nonce` in little-endian representation, convert it into its 48-trit `b1t6` encoding and append it to the hash trits. - Add a padding of three zero trits to create a 243-trit string. - Compute the Curl-P-81 hash. From 2e2ef6fee94a27064ee362d7553d97fc11dfc46a Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 19:37:49 +0200 Subject: [PATCH 51/52] use correct folder --- text/{0017-message => 0017-tangle-message}/0017-tangle-message.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0017-message => 0017-tangle-message}/0017-tangle-message.md (100%) diff --git a/text/0017-message/0017-tangle-message.md b/text/0017-tangle-message/0017-tangle-message.md similarity index 100% rename from text/0017-message/0017-tangle-message.md rename to text/0017-tangle-message/0017-tangle-message.md From 92cc7ef9f3683722e6abae8032df3a0e8f5a01e8 Mon Sep 17 00:00:00 2001 From: Wolfgang Welz Date: Wed, 20 Oct 2021 19:50:30 +0200 Subject: [PATCH 52/52] Add missing data length integer --- text/0017-tangle-message/0017-tangle-message.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0017-tangle-message/0017-tangle-message.md b/text/0017-tangle-message/0017-tangle-message.md index 3c572b4b8..080353ec9 100644 --- a/text/0017-tangle-message/0017-tangle-message.md +++ b/text/0017-tangle-message/0017-tangle-message.md @@ -175,7 +175,7 @@ Below is the full serialization of a valid message with an indexation payload. T - Index Length (2-byte): `0400` (4) - Index (4-byte): `494f5441` ("IOTA") - Data (15-byte): - - Length (4-byte): `0b000000` + - Length (4-byte): `0b000000` (11) - Data (11-byte): `68656c6c6f20776f726c64` ("hello world") - Nonce (8-byte): `ce6d000000000000` (28110)