RFC: Tangle Message #17

GalRogozinski · 2020-07-28T08:37:25Z

text/0017-message/0017-message.md

cvarley100 · 2020-07-31T10:47:01Z

text/0017-message/0017-message.md

+    So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes.
+    * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid.
+2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed.
+4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need.


How do you register a custom (community) payload type?

Is 16383 sufficient for all applications?

IIRC, in our meetings we threw ideas on how it might be.
I think one idea was that community member will also create RFCs but not sure this is the best way to approach this

I think that together with community volunteers we should define at least one community payload. Then we should properly document and implement it, so other community members can easily follow.
We should probably talk about it more in a meeting.

Regarding the 16,383 (2^14 - 1) limit, I just thought it was a high enough number...
Given that we support colored coins it is hard for me to imagine that the payload feature will become so popular that this limit won't suffice...

We should really explain how we envision this community-driven payload draft process.
I guess we want to have a table somewhere where people can create a PR with a new ID and link to the document / RFC describing the payload.

Does it maybe make more sense to have a separate RFC with this table (similar to SLIP-44 or SLIP-173)? That could make the process considerably easier.

Why use a varint if you have a max bound of 2 bytes? It is way easier to define a uint16 here

See my other answer

Also Wolfgang seems to not like this bound, so maybe this bound will be removed (still need to discuss this with him)

I think the bound (for the non-core payloads) seems arbitrary and should be removed. We should allow for the full uint32 maybe even uint64 space.

I agree the message length protection suffices.
Actually, while thinking about the case for the bound, I started to question whether the message length is really needed...

text/0017-message/0017-message.md

Wollac · 2020-08-03T09:37:58Z

text/0017-message/0017-message.md

+    So the varint `0x0F` equals to `0x8F8000`, but is encoded with different bytes.
+    * *Size Protection*: The varint must not be more than 10 bytes long. A 10 byte varint can encompass all the range of `uint64`. In case of an overflow above the `uint64` range the message is invalid.
+2. When we are done parsing the message there shouldn't be any trailing bytes left that were not parsed.
+4. If the `payload type` is in the core payload range (0-127) and the node is familiar with it, or if it is in the range of 128-16383. The upper limit of 16383 is the maximal value of a 2 byte varint, and it should suffice for all the optional payload types we need.


We should really explain how we envision this community-driven payload draft process.
I guess we want to have a table somewhere where people can create a PR with a new ID and link to the document / RFC describing the payload.

Does it maybe make more sense to have a separate RFC with this table (similar to SLIP-44 or SLIP-173)? That could make the process considerably easier.

alexsporn · 2020-08-06T06:37:01Z

text/0017-message/0017-message.md

+    </tr>
+    <tr>
+        <td>Payload Length</td>
+        <td>varint</td>


What do we gain by using a varint here? Are you planning to have megabyte-sized messages? Make it a uint32 and reduce complexity instead of making even the header of the message variable in size

We wanted to not have too many types in the message.
The idea is that less types means more simple descriptions and code.

Also, for lots of cases where we describe quantities uint8 or uint16 could have sufficed.

So the idea is that choosing varints will result mostly in shorter messages

I'm not convinced it will be simpler code. It's not native primitive in many languages.
So we either implement one ourselves or use the existing one. Both would end up with more validation rules.

If uint8_t or uint16_t is sufficed, why not just use one of them? they are primitive types in most languages. The varint is good for data serialization but it is not easy to operate and memory management in a data structure, especially for microcontrollers.

due to recent conversations outside of github I am also leaning towards dropping varints

this decision will be finalized on the next protocol team meeting

TL;DR: I would rather set fixed reasonable-sized types with some slack (you never know) replacing most varints.

varints are somewhat complex types to parse, but I would like to draw your attention to the creation of such message rather than the parsing of it. varints type force the entire message structure to "shift" instead of having fixed offsets. This is quite cumbersome when forming a message where all the parts are not necessarily known before-hand, and therefore the byte-requirements of length varint types is also unknown: when you fill in the data section of the message, you may see yourself push it forward in order to accommodate the corresponding varint size without overflowing into the data section itself. Sure, modern programming languages and libs provide abstract structures for all these, but at a cost of memory reallocation, memory concatenation and other forms of memory juggling. Leveling the participation field ultimately accounts for the flexibility of a solution.

text/0017-message/0017-message.md

thibault-martinez · 2020-08-18T13:38:46Z

text/0017-message/0017-message.md

+
+### Message PoW Hash
+
+This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with Curl-P-81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection.


We usually put the t before the b, so t6b1.

Also, trits, not trytes.

I think this is fundamentally different:

t5b1 encodes ternary data into binary, by converting each group of 5 trits into one byte; it cannot decode arbitrary binary data

b1t6 encodes binary data into ternary, by converting each byte into 6 trits; it cannot decode arbitrary ternary data

In RFC-15 it is also called b1t6.

I think maybe I can delete this entire section. Now we have an RFC dedicated for computing this PoW hash

Yes, the entire section should be deleted. See #17 (comment)

due to me looking at the RFC while you are commenting I repeated your remarks :-P

text/0017-message/0017-message.md

Co-authored-by: Thibault Martinez <thibault.martinez.30@gmail.com>

text/0017-message/0017-message.md

kuhlmannmarkus · 2021-02-18T13:51:32Z

awesome. thanks everyone!

text/0017-message/0017-message.md

Thoralf-M · 2021-03-15T09:21:01Z

Parent mesage ids need to be ordered and unique

text/0017-message/0017-message.md

thibault-martinez · 2021-10-20T10:18:41Z

text/0017-message/0017-message.md

+
+# Summary
+
+A message is the object nodes gossip around in the network. It always references two other messages that are known as _parents_. It is stored as a vertex on the tangle data structure that the nodes maintain.


It always references two other messages that are known as parents.

Not factually correct

Yep, good catch. I'll go over the entire text and make sure that everything is correct.

@thibault-martinez I did a mayor rewrite of the RFC that should hopefully address all the outstanding issues. Please consider reviewing again.

Wollac

Parents are not sorted
Example message is invalid as it contains the parents in the wrong order

add message rfc

3357004

GalRogozinski requested review from hmoog, luca-moser, thibault-martinez and Wollac July 28, 2020 08:37

add payload types

805758f

GalRogozinski changed the title ~~Add message rfc~~ Message RFC Jul 28, 2020

GalRogozinski changed the title ~~Message RFC~~ RFC: Message Jul 28, 2020

corrections

fa24ebb

GalRogozinski commented Jul 28, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

GalRogozinski commented Jul 28, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

add space

c778814

cvarley100 reviewed Jul 31, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

cvarley100 mentioned this pull request Jul 31, 2020

WOTS-Ed25 Wallet Transition Requirements iotaledger/chronicle.rs#70

Closed

Add milestone payload

915cd2b

Wollac reviewed Aug 3, 2020

View reviewed changes

alexsporn reviewed Aug 6, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

Remove dependency on payload validation

b973ccf

Wollac mentioned this pull request Aug 7, 2020

RFC: Transaction Payload #18

Merged

laumair mentioned this pull request Aug 17, 2020

chore: update wallet specification iotaledger/wallet.rs#15

Closed

6 tasks

thibault-martinez reviewed Aug 18, 2020

View reviewed changes

Wollac mentioned this pull request Aug 25, 2020

RFC: Message PoW #24

Merged

Wollac reviewed Aug 25, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

Gal Rogozinski and others added 2 commits August 25, 2020 11:24

change to lower case

5803db6

Co-authored-by: Thibault Martinez <thibault.martinez.30@gmail.com>

change we to it

4ce1327

Co-authored-by: Thibault Martinez <thibault.martinez.30@gmail.com>

Wollac suggested changes Aug 25, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

Wollac reviewed Aug 25, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

luca-moser reviewed Aug 25, 2020

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

thibault-martinez reviewed Feb 18, 2021

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

galrogo added 2 commits March 2, 2021 14:51

change parents rule to be between 1-8

466d328

remove core payload

d4eea48

thibault-martinez reviewed Mar 16, 2021

View reviewed changes

text/0017-message/0017-message.md Outdated Show resolved Hide resolved

m-renaud reviewed Mar 31, 2021

View reviewed changes

Wollac added 4 commits October 20, 2021 11:46

Update 0017-message.md

2ef7705

Update 0017-message.md

e5d02e2

Update 0017-message.md

1ba4de5

Update 0017-message.md

d479e9e

Wollac approved these changes Oct 20, 2021

View reviewed changes

thibault-martinez requested changes Oct 20, 2021

View reviewed changes

Clarify ranges

c3ace55

Wollac suggested changes Oct 20, 2021

View reviewed changes

Wollac added 4 commits October 20, 2021 12:29

Update 0017-message.md

5168461

RFC cleanup

f641685

Merge remote-tracking branch 'origin/master' into message

74a46d1

use correct links

b687e55

Wollac changed the title ~~RFC: Message~~ RFC: Tangle Message Oct 20, 2021

use correct folder

2e2ef6f

Wollac approved these changes Oct 20, 2021

View reviewed changes

Wollac requested review from thibault-martinez, luca-moser, alexsporn and muXxer October 20, 2021 17:41

Add missing data length integer

92cc7ef

thibault-martinez approved these changes Oct 21, 2021

View reviewed changes

alexsporn approved these changes Oct 22, 2021

View reviewed changes

Wollac merged commit 7662b9c into iotaledger:master Oct 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: Tangle Message #17

RFC: Tangle Message #17

GalRogozinski commented Jul 28, 2020 •

edited by Wollac

cvarley100 Jul 31, 2020 •

edited

GalRogozinski Aug 3, 2020

Wollac Aug 3, 2020

alexsporn Aug 6, 2020

GalRogozinski Aug 6, 2020

Wollac Aug 7, 2020

GalRogozinski Aug 25, 2020

Wollac Aug 3, 2020

alexsporn Aug 6, 2020 •

edited

GalRogozinski Aug 6, 2020

wusyong Aug 6, 2020 •

edited

oopsmonk Aug 11, 2020

GalRogozinski Aug 11, 2020

karimodm Aug 27, 2020 •

edited

thibault-martinez Aug 18, 2020

thibault-martinez Aug 18, 2020

Wollac Aug 25, 2020

GalRogozinski Aug 25, 2020

Wollac Aug 25, 2020

GalRogozinski Aug 25, 2020 •

edited

kuhlmannmarkus commented Feb 18, 2021

Thoralf-M commented Mar 15, 2021

thibault-martinez Oct 20, 2021

Wollac Oct 20, 2021

Wollac Oct 20, 2021

Wollac left a comment


		### Message PoW Hash

		This hash is computed by converting all the messages raw bytes into trytes with `b1t6`. Then hash the result with Curl-P-81. The result will be used to verify the proof-of-work done on the message. This will be used for spam protection.


		# Summary

		A message is the object nodes gossip around in the network. It always references two other messages that are known as _parents_. It is stored as a vertex on the tangle data structure that the nodes maintain.

RFC: Tangle Message #17

RFC: Tangle Message #17

Conversation

GalRogozinski commented Jul 28, 2020 • edited by Wollac

cvarley100 Jul 31, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexsporn Aug 6, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wusyong Aug 6, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karimodm Aug 27, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GalRogozinski Aug 25, 2020 • edited

Choose a reason for hiding this comment

kuhlmannmarkus commented Feb 18, 2021

Thoralf-M commented Mar 15, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wollac left a comment

Choose a reason for hiding this comment

GalRogozinski commented Jul 28, 2020 •

edited by Wollac

cvarley100 Jul 31, 2020 •

edited

alexsporn Aug 6, 2020 •

edited

wusyong Aug 6, 2020 •

edited

karimodm Aug 27, 2020 •

edited

GalRogozinski Aug 25, 2020 •

edited