From fd78e876d8b9293d217ee218a71384212e28f491 Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Mon, 7 Dec 2020 10:32:00 +0800 Subject: [PATCH 01/12] docs: Add markers specification draft --- docs/003-markers.md | 439 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 439 insertions(+) create mode 100644 docs/003-markers.md diff --git a/docs/003-markers.md b/docs/003-markers.md new file mode 100644 index 0000000000..15fdd5bd03 --- /dev/null +++ b/docs/003-markers.md @@ -0,0 +1,439 @@ +# Markers +## Summary +To know a message in the Tangle is orphaned or not, we introduce **grades of finality** to interpret the status of a message. However, the higher grades of finality is determined by the **approval weight**, which is the proportion of active consensus mana approving a given message. + +Computing the approaval weight of a given message needs to traverse the Tangle from the message to the tips and sum up the active consensus mana of all the messages in its future cone. And **marker** is a tool to efficiently estimate the approval weight of a message that reduces the portion of the Tangle we need to walk, and finally results in the grade of finality. + +**Note**: Markers is not a core module of the Coordicide project. + +## Motivation +Markers is a tool to infer knowledge about the structure of the Tangle in terms of: ++ past/future cone membership; ++ approximate approval weight of any message; ++ tagging sections of the Tangle (e.g., branches) without having to traverse each message individually. + +## Dependency +Active Consensus Mana + +## Detailed Design + +Let's define the terms related to markers: +* **Marker Sequence:** A Marker Sequence is a sequence of markers with the same Marker Sequence Identifier. Each Marker Sequence corresponds to a UTXO branch, which help us to track the structure independently. +* **Marker Sequence Identifier (MSI):** A Marker Sequence Identifier is the unique identifier of a Marker Sequence. +* **Marker Index (MI):** A Marker Index is the index of the marker in a Marker Sequence +* **marker:** A marker is a pair of numbers: MSI and MI associated to a given message. Markers carrying the same MSI belong to the same Marker Sequence. +* **future marker (FM):** A future marker of a message is the nearest marker in its future cone; this field is updated when the new marker is generated. The future marker has to be the same branch as the message. +* **past marker (PM):** A past marker of a message is the marker in its past cone. It is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. + +### The Marker Sequence +Marker sequences are used to track the UTXO DAG branches, each branch corresponds to a marker sequence with a unique $MSI$. + +#### Marker Sequence Structure + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
iduint64The marker sequence identifier of the marker sequence.
parentReferencesmap[uint64]ThresholdmapThe marker referenced map of each parent marker.
rankuint64The rank of the marker sequence.
highestIndexuint64The highest MI of the marker sequence.
lowestIndexuint64The lowest MI of the marker sequence.
+ +#### Create Marker Sequence +A new marker sequence is created when: +1. there's a conflict in UTXO branch. +2. the UTXO branches are aggregated. +3. UTXO branches are merged. + +Each new marker sequence starts from a new marker. + +### The Markers +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[MSI, MI]$. + +#### Marker Structure + + + + + + + + + + + + + + + + +
NameTypeDescription
MarkerSequenceIDuint64The marker sequence identifier of the marker.
MarkerIndexuint64The index of marker in the marker sequence.
+ +#### Create Markers +A new marker is created when: +1. the default conditions are met: + * **every x messsages**; + + **every t seconds**; + + a mix of the first two! + + Upperbound given by the messages + + Lower temporal bound given by the time + + every x messages that reference (directly or indirectly) the previous marker + + Lower bound given by rank (e.g., how far you are in terms of steps) -> >= 10 or something + + Upper bound given by the amount of messages referencing the previous one -> ~ 200 msgs +2. A new marker sequence is created. +> to be confirmed here. + +A new marker is selected from the tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[MSI, MI]$ is assigned to the new marker. +> to be confirmed here. + +The $MSI$ is set following rules: +* Create a new MSI if it is the first marker of the new marker sequence. +* Inherit the MSI from parents if it references the latest marker of the sequence and meets the requirement to set up a new marker. + +The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies the rule: ++ Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. + +### Past Markers and Future Markers +Each message keeps the marker information in two lists: +* past markers (PMs) +* future markers (FMs) + +PMs and FMs are used to determine whether a message is in the past cone of the other, and FMs also help us efficiently estimate the approval weight of a message. + +The figure below shows an example of applying markers to the Tangle with 1 marker sequence only: +![](https://i.imgur.com/RluZWCJ.png) + +The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in $MS$ $0$ with $MI$ $1$ and $2$ respectively. + +#### StructureDetails Structure +StructureDetails is a structure that will be in message metadata containing marker information. + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Rankuint64The rank of the message.
IsPastMarkerboolA flag to indicate whether a message is a marker.
PastMarkers +
+ Markers + + + + + + + + + + + +
NameTypeDescription
markersmap[SequenceID]IndexA list of PMs from different marker sequences.
+
+
FutureMarkers +
+ Markers + + + + + + + + + + + +
NameTypeDescription
markersmap[SequenceID]IndexA list of FMs from different marker sequences.
+
+
+ +##### Past Markers +PMs of a marker is the marker itself only. +PMs of general messages is inherited from its **strong** parents, with 2 steps: +1. keep the nearest markers (i.e., the marker with the biggest $MI$ ) of each marker sequence, +2. remove those that have been referenced by other markers in the same set. + +##### Future Markers +FMs is empty at start and get updated when the new marker directly or indirectly references it. The FMs propagation ends if: +1. the FMs of a message includes a previous marker of the same marker sequence; +2. the message is the first message we saw in the different marker sequence, we update the FMs of that first message only. + +### Markers and Approval Weight +To approximate the approval weight of a message, we simply retrieve the approval weight of its FMs. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. + +### Marker Management +Messages can have markers from different marker sequences in PMs and FMs, the order and referenced relationship among marker sequences are important when it comes to inheriting the PMs from parents and other usages. Thus, we need to track these marker sequences. + +Here's an example of how markers should look in Tangle: + +![](https://i.imgur.com/yi4E3Ik.png) + +#### Marker Sequences +Whatever reason a marker sequence is created, we assign a new $MSI = 1+max(referenceMarkerSequences)$. To prevent assigning a new $MSI$ when combining same marker sequences again, we build parents-child relation in a map if a new marker sequence is created. + +#### Rank +The rank of marker sequence graph is the number of sequences from the starting point to itself. The marker sequences in the figure above have rank: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColorSequence IDRank
orange01
blue11
yellow22
green31
red42
+ +We check the parent marker sequences of each candidate in order from high to low rank, if the parent sequence is in the candidate list, it will be removed. + +An example is **msg 10** in the figure, the PM candidates are $[0,2], [1,1], [2,3]$. $[2,3]$ is the first marker to check, since it has the highest rank. Then we find its parent marker sequences are $0$ and $1$, and perform futher $MI$ checks. And finally the PMs of **msg 10** is $[2,3]$ only, $[0,2], [1,1]$ are removed. + +This function returns the markers and marker sequences to inherit for a message. +``` +// normalizeMarkers takes a set of Markers and removes each Marker that is already referenced by another Marker in the +// same set (the remaining Markers are the "most special" Markers that reference all Markers in the set grouped by the +// rank of their corresponding Sequence). In addition, the method returns all SequenceIDs of the Markers that were not +// referenced by any of the Markers (the tips of the Sequence DAG). +func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *MarkersByRank, normalizedSequences SequenceIDs) { + rankOfSequencesCache := make(map[SequenceID]uint64) + + normalizedMarkersByRank = NewMarkersByRank() + normalizedSequences = make(SequenceIDs) + markers.ForEach(func(sequenceID SequenceID, index Index) bool { + normalizedSequences[sequenceID] = types.Void + normalizedMarkersByRank.Add(m.rankOfSequence(sequenceID, rankOfSequencesCache), sequenceID, index) + + return true + }) + markersToIterate := normalizedMarkersByRank.Clone() + + for i := markersToIterate.HighestRank() + 1; i > normalizedMarkersByRank.LowestRank(); i-- { + currentRank := i - 1 + markersByRank, rankExists := markersToIterate.Markers(currentRank) + if !rankExists { + continue + } + + if !markersByRank.ForEach(func(sequenceID SequenceID, index Index) bool { + if currentRank <= normalizedMarkersByRank.LowestRank() { + return false + } + + if !m.sequence(sequenceID).Consume(func(sequence *Sequence) { + sequence.HighestReferencedParentMarkers(index).ForEach(func(referencedSequenceID SequenceID, referencedIndex Index) bool { + delete(normalizedSequences, referencedSequenceID) + + rankOfReferencedSequence := m.rankOfSequence(referencedSequenceID, rankOfSequencesCache) + if index, indexExists := normalizedMarkersByRank.Index(rankOfReferencedSequence, referencedSequenceID); indexExists { + if referencedIndex >= index { + normalizedMarkersByRank.Delete(rankOfReferencedSequence, referencedSequenceID) + + if rankOfReferencedSequence > normalizedMarkersByRank.LowestRank() { + markersToIterate.Add(rankOfReferencedSequence, referencedSequenceID, referencedIndex) + } + } + + return true + } + + if rankOfReferencedSequence > normalizedMarkersByRank.LowestRank() { + markersToIterate.Add(rankOfReferencedSequence, referencedSequenceID, referencedIndex) + } + + return true + }) + }) { + panic(fmt.Sprintf("failed to load Sequence with %s", sequenceID)) + } + + return true + }) { + return + } + } + + return +} + +``` + +### Markers Usages: Past Cone Check +By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to walk the Tangle. + +#### Algorithm +``` +// IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. +func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) { + // fast check: if earlier Markers have larger highest Indexes they can't be in the past cone + if earlierMarkers.PastMarkers.HighestIndex() > laterMarkers.PastMarkers.HighestIndex() { + return False + } + + // fast check: if earlier Marker is a past Marker and the later ones reference it we can return early + if earlierMarkers.IsPastMarker { + earlierMarker := earlierMarkers.PastMarkers.FirstMarker() + if earlierMarker == nil { + panic("failed to retrieve Marker") + } + + if laterIndex, sequenceExists := laterMarkers.PastMarkers.Get(earlierMarker.sequenceID); sequenceExists { + if laterIndex >= earlierMarker.index { + return True + } + + return False + } + + if laterMarkers.PastMarkers.HighestIndex() <= earlierMarker.index { + return False + } + } + + if laterMarkers.IsPastMarker { + laterMarker := laterMarkers.PastMarkers.FirstMarker() + if laterMarker == nil { + panic("failed to retrieve Marker") + } + + // if the earlier Marker inherited an Index of the same Sequence that is higher than the later we return false + if earlierIndex, sequenceExists := earlierMarkers.PastMarkers.Get(laterMarker.sequenceID); sequenceExists && earlierIndex >= laterMarker.index { + return False + } + + // if the earlier Markers are referenced by a Marker of the same Sequence that is larger, we are not in the past cone + if earlierFutureIndex, earlierFutureIndexExists := earlierMarkers.FutureMarkers.Get(laterMarker.sequenceID); earlierFutureIndexExists && earlierFutureIndex > laterMarker.index { + return False + } + + // if the earlier Markers were referenced by the same or a higher future Marker we are not in the past cone + // (otherwise we would be the future marker) + if !laterMarkers.FutureMarkers.ForEach(func(sequenceID SequenceID, laterIndex Index) bool { + earlierIndex, similarSequenceExists := earlierMarkers.FutureMarkers.Get(sequenceID) + return !similarSequenceExists || earlierIndex < laterIndex + }) { + return False + } + + if earlierMarkers.PastMarkers.HighestIndex() >= laterMarker.index { + return False + } + } + + // if the highest Indexes of both past Markers are the same ... + if earlierMarkers.PastMarkers.HighestIndex() == laterMarkers.PastMarkers.HighestIndex() { + // ... then the later Markers should contain exact copies of all of the highest earlier Markers because parent + // Markers get inherited and if they would have been captured by a new Marker in between then the highest + // Indexes would no longer be the same + if !earlierMarkers.PastMarkers.ForEach(func(sequenceID SequenceID, earlierIndex Index) bool { + if earlierIndex == earlierMarkers.PastMarkers.HighestIndex() { + laterIndex, sequenceExists := laterMarkers.PastMarkers.Get(sequenceID) + return sequenceExists && laterIndex != earlierIndex + } + + return true + }) { + return False + } + } + + if earlierMarkers.FutureMarkers.HighestIndex() == laterMarkers.FutureMarkers.HighestIndex() && false { + // the earlier future markers need to contain all later ones because if there would be another marker in between that shadows them the later future Marker would have a higher index + if !laterMarkers.FutureMarkers.ForEach(func(sequenceID SequenceID, laterIndex Index) bool { + if laterIndex == laterMarkers.FutureMarkers.highestIndex { + earlierIndex, sequenceExists := earlierMarkers.FutureMarkers.Get(sequenceID) + return sequenceExists && earlierIndex == laterIndex + } + + return true + }) { + return False + } + } + + // detailed check: earlier marker is referenced by something that the later one references + if m.markersReferenceMarkers(laterMarkers.PastMarkers, earlierMarkers.FutureMarkers, false) { + return True + } + + // detailed check: the + if m.markersReferenceMarkers(earlierMarkers.FutureMarkers, laterMarkers.PastMarkers, true) { + return Maybe + } + + return False +} + +``` + +### Markers Usages: Approval Weight Estimation +The approval weight of every marker is easy to compute (and hence its finality too): to check the weight of marker $i$, the node looks at the tip list $X$, and computes the proportion of messages in $X$ whose MRRM is $i$ or greater. + +Store the approval weight +* save the nodeID per marker, and do the union when we want the approval weight -> we don't want to count the mana twice +* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. + * use epoch j-2 mana, if you're in epoch j From c94bd3a5c0b2bf9ad41d3151d6d50cc9bb54c4a1 Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Mon, 7 Dec 2020 10:36:39 +0800 Subject: [PATCH 02/12] docs: Add syntax highlight --- docs/003-markers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 15fdd5bd03..020c8799b6 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -254,7 +254,7 @@ We check the parent marker sequences of each candidate in order from high to low An example is **msg 10** in the figure, the PM candidates are $[0,2], [1,1], [2,3]$. $[2,3]$ is the first marker to check, since it has the highest rank. Then we find its parent marker sequences are $0$ and $1$, and perform futher $MI$ checks. And finally the PMs of **msg 10** is $[2,3]$ only, $[0,2], [1,1]$ are removed. This function returns the markers and marker sequences to inherit for a message. -``` +```go // normalizeMarkers takes a set of Markers and removes each Marker that is already referenced by another Marker in the // same set (the remaining Markers are the "most special" Markers that reference all Markers in the set grouped by the // rank of their corresponding Sequence). In addition, the method returns all SequenceIDs of the Markers that were not @@ -326,7 +326,7 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *M By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to walk the Tangle. #### Algorithm -``` +```go // IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) { // fast check: if earlier Markers have larger highest Indexes they can't be in the past cone From 0c91eb01bec1c87a59650e43747c136cfc8dfd63 Mon Sep 17 00:00:00 2001 From: "Ching-Hua (Vivian) Lin" Date: Tue, 8 Dec 2020 10:15:38 +0800 Subject: [PATCH 03/12] docs: Minor tweaks --- docs/003-markers.md | 70 +++++++++++++++++++++++---------------------- 1 file changed, 36 insertions(+), 34 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 020c8799b6..3d7d96f178 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -1,13 +1,13 @@ -# Markers +# Marker Spec ## Summary -To know a message in the Tangle is orphaned or not, we introduce **grades of finality** to interpret the status of a message. However, the higher grades of finality is determined by the **approval weight**, which is the proportion of active consensus mana approving a given message. +In order to know whether a message in the Tangle is orphaned or not, we introduce **grades of finality** to interpret the status of a message. The higher grade of finality is determined by the **approval weight**, which is the proportion of active consensus mana approving a given message. -Computing the approaval weight of a given message needs to traverse the Tangle from the message to the tips and sum up the active consensus mana of all the messages in its future cone. And **marker** is a tool to efficiently estimate the approval weight of a message that reduces the portion of the Tangle we need to walk, and finally results in the grade of finality. +In order to compute the approval weight of a given message we need to traverse the Tangle from the message to the tips and sum up the active consensus mana of all the messages in its future cone. A **marker** is a tool to efficiently estimate the approval weight of a message and that reduces the portion of the Tangle we need to walk, and which finally results in the grade of finality. -**Note**: Markers is not a core module of the Coordicide project. +**Note**: *Markers* is not a core module of the Coordicide project. ## Motivation -Markers is a tool to infer knowledge about the structure of the Tangle in terms of: +*Markers* is a tool to infer knowledge about the structure of the Tangle in terms of: + past/future cone membership; + approximate approval weight of any message; + tagging sections of the Tangle (e.g., branches) without having to traverse each message individually. @@ -15,6 +15,7 @@ Markers is a tool to infer knowledge about the structure of the Tangle in terms ## Dependency Active Consensus Mana + ## Detailed Design Let's define the terms related to markers: @@ -26,7 +27,7 @@ Let's define the terms related to markers: * **past marker (PM):** A past marker of a message is the marker in its past cone. It is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. ### The Marker Sequence -Marker sequences are used to track the UTXO DAG branches, each branch corresponds to a marker sequence with a unique $MSI$. +Marker sequences are used to track the UTXO DAG branches, each branch corresponds to a marker sequence with a unique *`MSI`*, and the marker sequences form a DAG as well. #### Marker Sequence Structure @@ -49,7 +50,7 @@ Marker sequences are used to track the UTXO DAG branches, each branch correspond rank uint64 - The rank of the marker sequence. + The rank of the marker sequence in the marker sequence DAG. highestIndex @@ -63,16 +64,17 @@ Marker sequences are used to track the UTXO DAG branches, each branch correspond + #### Create Marker Sequence A new marker sequence is created when: -1. there's a conflict in UTXO branch. +1. there's a conflict in a UTXO branch. 2. the UTXO branches are aggregated. 3. UTXO branches are merged. Each new marker sequence starts from a new marker. ### The Markers -Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[MSI, MI]$. +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of *`[MSI, MI]`*. #### Marker Structure @@ -89,7 +91,7 @@ Markers are messages selected from the tip set periodically and assigned unique - +
MarkerIndex uint64The index of marker in the marker sequence.The index of the marker in the marker sequence.
@@ -105,17 +107,18 @@ A new marker is created when: + Lower bound given by rank (e.g., how far you are in terms of steps) -> >= 10 or something + Upper bound given by the amount of messages referencing the previous one -> ~ 200 msgs 2. A new marker sequence is created. -> to be confirmed here. +> :mega: to be confirmed here. -A new marker is selected from the tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[MSI, MI]$ is assigned to the new marker. -> to be confirmed here. +A new marker is selected from the tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of *`[MSI, MI]`* is assigned to the new marker. +> :mega: to be confirmed here. -The $MSI$ is set following rules: + +The *`MSI`* is set according to the following rules: * Create a new MSI if it is the first marker of the new marker sequence. * Inherit the MSI from parents if it references the latest marker of the sequence and meets the requirement to set up a new marker. -The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies the rule: -+ Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. +The *`MI`* is set to *`MI = 1+ max(referencedMI)`*, which complies to the rule: ++ Marker indexes (*`MIs`*) are monotonically increasing such that *`\forall x \in fc(y)`* => *`MI_x > MI_y`*, where *`fc(y)`* is the future cone of *`y`* and *`x`* is any message in that future cone. ### Past Markers and Future Markers Each message keeps the marker information in two lists: @@ -124,13 +127,8 @@ Each message keeps the marker information in two lists: PMs and FMs are used to determine whether a message is in the past cone of the other, and FMs also help us efficiently estimate the approval weight of a message. -The figure below shows an example of applying markers to the Tangle with 1 marker sequence only: -![](https://i.imgur.com/RluZWCJ.png) - -The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in $MS$ $0$ with $MI$ $1$ and $2$ respectively. - #### StructureDetails Structure -StructureDetails is a structure that will be in message metadata containing marker information. +StructureDetails is a structure that will be in the message metadata containing marker information. @@ -191,15 +189,20 @@ StructureDetails is a structure that will be in message metadata containing mark
##### Past Markers -PMs of a marker is the marker itself only. -PMs of general messages is inherited from its **strong** parents, with 2 steps: -1. keep the nearest markers (i.e., the marker with the biggest $MI$ ) of each marker sequence, +A PM of a marker is the marker itself only. +A PM of general messages is inherited from its **strong** parents, with 2 steps: +1. keep the nearest markers (i.e., the marker with the biggest *`MI`* ) of each marker sequence, 2. remove those that have been referenced by other markers in the same set. ##### Future Markers -FMs is empty at start and get updated when the new marker directly or indirectly references it. The FMs propagation ends if: -1. the FMs of a message includes a previous marker of the same marker sequence; -2. the message is the first message we saw in the different marker sequence, we update the FMs of that first message only. +The FM is empty at start and gets updated when the new marker directly or indirectly references it. The FMs propagation ends if: +1. the FM of a message includes a previous marker of the same marker sequence; +2. the message is the first message we saw in the different marker sequence, we update the FM of that first message only. + +The figure below shows an example of applying markers to the Tangle with 1 marker sequence only: +![](https://i.imgur.com/RluZWCJ.png) + +The yellow messages are markers with identifiers: *`[0,1]`* and *`[0,2]`*. Both markers are in *`MS`* *`0`* with *`MI`* *`1`* and *`2`* respectively. ### Markers and Approval Weight To approximate the approval weight of a message, we simply retrieve the approval weight of its FMs. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. @@ -207,12 +210,12 @@ To approximate the approval weight of a message, we simply retrieve the approval ### Marker Management Messages can have markers from different marker sequences in PMs and FMs, the order and referenced relationship among marker sequences are important when it comes to inheriting the PMs from parents and other usages. Thus, we need to track these marker sequences. -Here's an example of how markers should look in Tangle: +Here is an example of how markers would look in the Tangle: ![](https://i.imgur.com/yi4E3Ik.png) #### Marker Sequences -Whatever reason a marker sequence is created, we assign a new $MSI = 1+max(referenceMarkerSequences)$. To prevent assigning a new $MSI$ when combining same marker sequences again, we build parents-child relation in a map if a new marker sequence is created. +Whatever reason a marker sequence is created, we assign a new *`MSI = 1+max(referenceMarkerSequences)`*. To prevent assigning a new *`MSI`* when combining same marker sequences again, we build parents-child relation in a map if a new marker sequence is created. #### Rank The rank of marker sequence graph is the number of sequences from the starting point to itself. The marker sequences in the figure above have rank: @@ -251,7 +254,7 @@ The rank of marker sequence graph is the number of sequences from the starting p We check the parent marker sequences of each candidate in order from high to low rank, if the parent sequence is in the candidate list, it will be removed. -An example is **msg 10** in the figure, the PM candidates are $[0,2], [1,1], [2,3]$. $[2,3]$ is the first marker to check, since it has the highest rank. Then we find its parent marker sequences are $0$ and $1$, and perform futher $MI$ checks. And finally the PMs of **msg 10** is $[2,3]$ only, $[0,2], [1,1]$ are removed. +An example is **msg 10** in the figure, the PM candidates are *`[0,2], [1,1], [2,3]`*. *`[2,3]`* is the first marker to check, since it has the highest rank. Then we find its parent marker sequences are *`0`* and *`1`*, and perform futher *`MI`* checks. And finally the PMs of **msg 10** is *`[2,3]`* only, *`[0,2], [1,1]`* are removed. This function returns the markers and marker sequences to inherit for a message. ```go @@ -431,9 +434,8 @@ func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *Marker ``` ### Markers Usages: Approval Weight Estimation -The approval weight of every marker is easy to compute (and hence its finality too): to check the weight of marker $i$, the node looks at the tip list $X$, and computes the proportion of messages in $X$ whose MRRM is $i$ or greater. - Store the approval weight * save the nodeID per marker, and do the union when we want the approval weight -> we don't want to count the mana twice * bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. - * use epoch j-2 mana, if you're in epoch j + * use epoch j-2 mana, if you're in epoch j + From 603e9a55b3ebb843316c43ee716d4aa8530945d5 Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Thu, 4 Feb 2021 11:55:39 +0800 Subject: [PATCH 04/12] doc: Fix part of comments --- docs/003-markers.md | 204 +++++++++++++++++--------------------------- 1 file changed, 79 insertions(+), 125 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 3d7d96f178..679422c19d 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -1,4 +1,4 @@ -# Marker Spec +# Markers ## Summary In order to know whether a message in the Tangle is orphaned or not, we introduce **grades of finality** to interpret the status of a message. The higher grade of finality is determined by the **approval weight**, which is the proportion of active consensus mana approving a given message. @@ -19,17 +19,19 @@ Active Consensus Mana ## Detailed Design Let's define the terms related to markers: -* **Marker Sequence:** A Marker Sequence is a sequence of markers with the same Marker Sequence Identifier. Each Marker Sequence corresponds to a UTXO branch, which help us to track the structure independently. -* **Marker Sequence Identifier (MSI):** A Marker Sequence Identifier is the unique identifier of a Marker Sequence. -* **Marker Index (MI):** A Marker Index is the index of the marker in a Marker Sequence -* **marker:** A marker is a pair of numbers: MSI and MI associated to a given message. Markers carrying the same MSI belong to the same Marker Sequence. +* **Sequence:** A Sequence is a sequence of markers. Each Sequence corresponds to a UTXO branch, which help us to track the structure independently. +* **Sequence Identifier (SI):** A Sequence Identifier is the unique identifier of a Sequence. +* **Marker Index (MI):** A Marker Index is the marker rank in the marker DAG. Throughout the code the marker rank will be called index. +* **marker:** A marker is a pair of numbers: SI and MI associated to a given message. Markers carrying the same SI belong to the same Sequence. * **future marker (FM):** A future marker of a message is the nearest marker in its future cone; this field is updated when the new marker is generated. The future marker has to be the same branch as the message. * **past marker (PM):** A past marker of a message is the marker in its past cone. It is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. +* **sequence rank:** The rank of a sequence will be simply called rank throughout this code. Bear in mind that for clarity the marker rank is called index. -### The Marker Sequence -Marker sequences are used to track the UTXO DAG branches, each branch corresponds to a marker sequence with a unique *`MSI`*, and the marker sequences form a DAG as well. -#### Marker Sequence Structure +### The Sequence +Sequences are used to track the UTXO DAG branches, each branch corresponds to a sequence with a unique $SI$, and the sequences form a DAG as well. + +#### Sequence Structure @@ -40,7 +42,7 @@ Marker sequences are used to track the UTXO DAG branches, each branch correspond - + @@ -50,7 +52,7 @@ Marker sequences are used to track the UTXO DAG branches, each branch correspond - + @@ -60,21 +62,21 @@ Marker sequences are used to track the UTXO DAG branches, each branch correspond - +
id uint64The marker sequence identifier of the marker sequence.The sequence identifier of the sequence.
parentReferences
rank uint64The rank of the marker sequence in the marker sequence DAG.The rank of the sequence in the marker DAG.
highestIndex
lowestIndex uint64The lowest MI of the marker sequence.The lowest MI of the sequence.
-#### Create Marker Sequence -A new marker sequence is created when: +#### Create Sequence +A new sequence is created when: 1. there's a conflict in a UTXO branch. 2. the UTXO branches are aggregated. 3. UTXO branches are merged. -Each new marker sequence starts from a new marker. +Each new sequence starts from a new marker. ### The Markers -Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of *`[MSI, MI]`*. +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SI, MI]$. #### Marker Structure @@ -84,20 +86,20 @@ Markers are messages selected from the tip set periodically and assigned unique - + - + - + - +
Description
MarkerSequenceIDSequenceID uint64The marker sequence identifier of the marker.The Sequence identifier of the marker.
MarkerIndexIndex uint64The index of the marker in the marker sequence.The index of the marker in the sequence.
#### Create Markers A new marker is created when: -1. the default conditions are met: +1. the default conditions are met, which will be one of these options: * **every x messsages**; + **every t seconds**; + a mix of the first two! @@ -106,26 +108,26 @@ A new marker is created when: + every x messages that reference (directly or indirectly) the previous marker + Lower bound given by rank (e.g., how far you are in terms of steps) -> >= 10 or something + Upper bound given by the amount of messages referencing the previous one -> ~ 200 msgs -2. A new marker sequence is created. +2. A new sequence is created. > :mega: to be confirmed here. -A new marker is selected from the tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of *`[MSI, MI]`* is assigned to the new marker. +A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[SI, MI]$ is assigned to the new marker. > :mega: to be confirmed here. -The *`MSI`* is set according to the following rules: -* Create a new MSI if it is the first marker of the new marker sequence. -* Inherit the MSI from parents if it references the latest marker of the sequence and meets the requirement to set up a new marker. +The $SI$ is set according to the following rules: +* Create a new SI if it is the first marker of the new sequence. +* Inherit the SI from parents if it references the latest marker of the sequence and meets the requirement to set up a new marker. -The *`MI`* is set to *`MI = 1+ max(referencedMI)`*, which complies to the rule: -+ Marker indexes (*`MIs`*) are monotonically increasing such that *`\forall x \in fc(y)`* => *`MI_x > MI_y`*, where *`fc(y)`* is the future cone of *`y`* and *`x`* is any message in that future cone. +The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies to the rule: ++ Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. ### Past Markers and Future Markers Each message keeps the marker information in two lists: -* past markers (PMs) -* future markers (FMs) +* past markers +* future markers -PMs and FMs are used to determine whether a message is in the past cone of the other, and FMs also help us efficiently estimate the approval weight of a message. +These lists for past markers and future markers are used to determine whether a message is in the past cone of the other, and the list for future markers also helps us to efficiently estimate the approval weight of a message. #### StructureDetails Structure StructureDetails is a structure that will be in the message metadata containing marker information. @@ -148,125 +150,67 @@ StructureDetails is a structure that will be in the message metadata containing PastMarkers - -
- Markers - - - - - - - - - - - -
NameTypeDescription
markersmap[SequenceID]IndexA list of PMs from different marker sequences.
-
- + map[SequenceID]Index + PM list, a list of PMs from different sequences. FutureMarkers - -
- Markers - - - - - - - - - - - -
NameTypeDescription
markersmap[SequenceID]IndexA list of FMs from different marker sequences.
-
- + map[SequenceID]Index + FM list, a list of FMs from different sequences. ##### Past Markers -A PM of a marker is the marker itself only. -A PM of general messages is inherited from its **strong** parents, with 2 steps: -1. keep the nearest markers (i.e., the marker with the biggest *`MI`* ) of each marker sequence, -2. remove those that have been referenced by other markers in the same set. +* The PM list of a marker contains the marker itself only. +* The PM list of general messages is inherited from its **strong** parents, with 2 steps: + 1. keep the nearest markers (i.e., the marker with the biggest $MI$ ) of each sequence, + 2. remove those that have been referenced by other markers in the same set. ##### Future Markers -The FM is empty at start and gets updated when the new marker directly or indirectly references it. The FMs propagation ends if: -1. the FM of a message includes a previous marker of the same marker sequence; -2. the message is the first message we saw in the different marker sequence, we update the FM of that first message only. +The FM list of a message is empty at start and gets updated when the new marker directly or indirectly references it. In other words, the new created marker will be propagated to the FM list of its pastcone, and this propagation ends if: +1. the FM list of a message includes a previous marker of the same sequence; +2. the message is the marker in the different sequence, we update the FM list of that marker only. + +The figure below shows an example of applying markers to the Tangle with 1 sequence only. The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in Sequence $0$ with $MI$ $1$ and $2$ respectively. -The figure below shows an example of applying markers to the Tangle with 1 marker sequence only: ![](https://i.imgur.com/RluZWCJ.png) -The yellow messages are markers with identifiers: *`[0,1]`* and *`[0,2]`*. Both markers are in *`MS`* *`0`* with *`MI`* *`1`* and *`2`* respectively. ### Markers and Approval Weight -To approximate the approval weight of a message, we simply retrieve the approval weight of its FMs. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. +To approximate the approval weight of a message, we simply retrieve the approval weight of its FM list. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. ### Marker Management -Messages can have markers from different marker sequences in PMs and FMs, the order and referenced relationship among marker sequences are important when it comes to inheriting the PMs from parents and other usages. Thus, we need to track these marker sequences. +Messages can have markers from different sequences in PM list and FM list, the order and referenced relationship among sequences are important for example when it comes to inheriting the PM list from parents. Thus, we need to track these sequences. -Here is an example of how markers would look in the Tangle: +Here is an example of how the markers and sequences structures would look in the Tangle: + +![](https://i.imgur.com/GENej3O.png) -![](https://i.imgur.com/yi4E3Ik.png) -#### Marker Sequences -Whatever reason a marker sequence is created, we assign a new *`MSI = 1+max(referenceMarkerSequences)`*. To prevent assigning a new *`MSI`* when combining same marker sequences again, we build parents-child relation in a map if a new marker sequence is created. -#### Rank -The rank of marker sequence graph is the number of sequences from the starting point to itself. The marker sequences in the figure above have rank: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ColorSequence IDRank
orange01
blue11
yellow22
green31
red42
-We check the parent marker sequences of each candidate in order from high to low rank, if the parent sequence is in the candidate list, it will be removed. +#### Sequences +Whatever reason a sequence is created, we assign a new $SI = 1+max(referenceSequences)$. To prevent assigning a new $SI$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. -An example is **msg 10** in the figure, the PM candidates are *`[0,2], [1,1], [2,3]`*. *`[2,3]`* is the first marker to check, since it has the highest rank. Then we find its parent marker sequences are *`0`* and *`1`*, and perform futher *`MI`* checks. And finally the PMs of **msg 10** is *`[2,3]`* only, *`[0,2], [1,1]`* are removed. +#### Sequence Rank +The rank of a sequence graph is the number of sequences from the starting point to itself. The sequence ranks are shown in the figure above. -This function returns the markers and marker sequences to inherit for a message. -```go +When a new sequence is created we check the parent marker' sequences with the function `normalizeMarkers()` in order from high to low rank. In this function, we remove those PMs that it's belonging sequence is referenced by others. + +An example is **msg 10** in the figure, $[0,2], [1,1], [2,3]$ are PMs to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced PMs therein. Next any PMs that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. + +In the following we show an implementation of `normalizeMarkers()`, which returns the markers and sequences that will be inherited from a message. +```go= // normalizeMarkers takes a set of Markers and removes each Marker that is already referenced by another Marker in the // same set (the remaining Markers are the "most special" Markers that reference all Markers in the set grouped by the // rank of their corresponding Sequence). In addition, the method returns all SequenceIDs of the Markers that were not // referenced by any of the Markers (the tips of the Sequence DAG). -func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *MarkersByRank, normalizedSequences SequenceIDs) { +func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *markersByRank, normalizedSequences SequenceIDs) { rankOfSequencesCache := make(map[SequenceID]uint64) - normalizedMarkersByRank = NewMarkersByRank() + normalizedMarkersByRank = newMarkersByRank() normalizedSequences = make(SequenceIDs) + // group markers with same sequence rank markers.ForEach(func(sequenceID SequenceID, index Index) bool { normalizedSequences[sequenceID] = types.Void normalizedMarkersByRank.Add(m.rankOfSequence(sequenceID, rankOfSequencesCache), sequenceID, index) @@ -275,6 +219,7 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *M }) markersToIterate := normalizedMarkersByRank.Clone() + //iterate from highest sequence rank to lowest for i := markersToIterate.HighestRank() + 1; i > normalizedMarkersByRank.LowestRank(); i-- { currentRank := i - 1 markersByRank, rankExists := markersToIterate.Markers(currentRank) @@ -282,20 +227,28 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *M continue } + // for each marker from the current sequence rank check if we can remove a marker in normalizedMarkersByRank, + // and add the parent markers to markersToIterate if necessary if !markersByRank.ForEach(func(sequenceID SequenceID, index Index) bool { if currentRank <= normalizedMarkersByRank.LowestRank() { return false } - if !m.sequence(sequenceID).Consume(func(sequence *Sequence) { + if !(&CachedSequence{CachedObject: m.sequenceStore.Load(sequenceID.Bytes())}).Consume(func(sequence *Sequence) { + // for each of the parentMarkers of this particular index sequence.HighestReferencedParentMarkers(index).ForEach(func(referencedSequenceID SequenceID, referencedIndex Index) bool { + // of this marker delete the referenced sequences since they are no sequence tips anymore in the sequence DAG delete(normalizedSequences, referencedSequenceID) rankOfReferencedSequence := m.rankOfSequence(referencedSequenceID, rankOfSequencesCache) + // check whether there is a marker in normalizedMarkersByRank that is from the same sequence if index, indexExists := normalizedMarkersByRank.Index(rankOfReferencedSequence, referencedSequenceID); indexExists { if referencedIndex >= index { + // this referencedParentMarker is from the same sequence as a marker in the list but with higher index - hence remove the index from the Marker list normalizedMarkersByRank.Delete(rankOfReferencedSequence, referencedSequenceID) + // if rankOfReferencedSequence is already the lowest rank of the original markers list, + // no need to add it since parents of the referencedMarker cannot delete any further elements from the list if rankOfReferencedSequence > normalizedMarkersByRank.LowestRank() { markersToIterate.Add(rankOfReferencedSequence, referencedSequenceID, referencedIndex) } @@ -304,6 +257,8 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *M return true } + // if rankOfReferencedSequence is already the lowest rank of the original markers list, + // no need to add it since parents of the referencedMarker cannot delete any further elements from the list if rankOfReferencedSequence > normalizedMarkersByRank.LowestRank() { markersToIterate.Add(rankOfReferencedSequence, referencedSequenceID, referencedIndex) } @@ -322,14 +277,13 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *M return } - ``` ### Markers Usages: Past Cone Check By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to walk the Tangle. #### Algorithm -```go +```go= // IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) { // fast check: if earlier Markers have larger highest Indexes they can't be in the past cone @@ -434,8 +388,8 @@ func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *Marker ``` ### Markers Usages: Approval Weight Estimation +The approval weight of every marker is easy to compute (and hence its finality too): to check the weight of marker $i$, the node looks at the tip list $X$, and computes the proportion of messages in $X$ whose MRRM is $i$ or greater. + Store the approval weight * save the nodeID per marker, and do the union when we want the approval weight -> we don't want to count the mana twice -* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. - * use epoch j-2 mana, if you're in epoch j - +* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. \ No newline at end of file From f7707649f5177a760b205515de5774c46f2a4c2d Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Thu, 4 Feb 2021 13:15:32 +0800 Subject: [PATCH 05/12] doc: Adjust wording --- docs/003-markers.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 679422c19d..810122edb8 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -23,8 +23,8 @@ Let's define the terms related to markers: * **Sequence Identifier (SI):** A Sequence Identifier is the unique identifier of a Sequence. * **Marker Index (MI):** A Marker Index is the marker rank in the marker DAG. Throughout the code the marker rank will be called index. * **marker:** A marker is a pair of numbers: SI and MI associated to a given message. Markers carrying the same SI belong to the same Sequence. -* **future marker (FM):** A future marker of a message is the nearest marker in its future cone; this field is updated when the new marker is generated. The future marker has to be the same branch as the message. -* **past marker (PM):** A past marker of a message is the marker in its past cone. It is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. +* **future marker (FM):** A future marker of a message is the first marker in its future cone from different sequences; this field in the message metadata is updated when the new marker is generated in the future, following the rules defined in (#future-markers). +* **past marker (PM):** A past marker of a message is a marker in its past cone. For a given sequence it is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. * **sequence rank:** The rank of a sequence will be simply called rank throughout this code. Bear in mind that for clarity the marker rank is called index. @@ -123,7 +123,7 @@ The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies to the rule: + Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. ### Past Markers and Future Markers -Each message keeps the marker information in two lists: +Each message keeps its associated marker information in two lists: * past markers * future markers @@ -173,6 +173,7 @@ The FM list of a message is empty at start and gets updated when the new marker The figure below shows an example of applying markers to the Tangle with 1 sequence only. The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in Sequence $0$ with $MI$ $1$ and $2$ respectively. +For a more complete but complex example, including multiple sequences refer to the next figure. ![](https://i.imgur.com/RluZWCJ.png) From 4110ca0803e67ed0000ddb2d300e0a26d83a69e2 Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Thu, 4 Feb 2021 13:33:38 +0800 Subject: [PATCH 06/12] docs: Fix old comments --- docs/003-markers.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 810122edb8..42b23b44e6 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -116,8 +116,8 @@ A new marker is selected from the strong tips set randomly, and selected from th The $SI$ is set according to the following rules: -* Create a new SI if it is the first marker of the new sequence. -* Inherit the SI from parents if it references the latest marker of the sequence and meets the requirement to set up a new marker. +* Inherit the SI from parents if the new marker references the latest marker of a sequence and meets the requirement to set up a new marker without initiating a new MS. +* Create a new SI if it is the first marker of a new sequence. The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies to the rule: + Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. @@ -162,12 +162,12 @@ StructureDetails is a structure that will be in the message metadata containing ##### Past Markers * The PM list of a marker contains the marker itself only. -* The PM list of general messages is inherited from its **strong** parents, with 2 steps: - 1. keep the nearest markers (i.e., the marker with the biggest $MI$ ) of each sequence, - 2. remove those that have been referenced by other markers in the same set. +* The PM list of non-marker messages is inherited from its **strong** parents, with 2 steps: + 1. for a given sequence select only the nearest marker (i.e. the markers with the highest MI). Thus for every sequence from the parents there will be exactly one marker. + 2. remove those that have been referenced by other markers from this set. ##### Future Markers -The FM list of a message is empty at start and gets updated when the new marker directly or indirectly references it. In other words, the new created marker will be propagated to the FM list of its pastcone, and this propagation ends if: +The FM list of a message is empty at start and gets updated when a new marker directly or indirectly references it. The propagation of a FM to its past cone (i.e. the update of the `FutureMarkers` field in the encountered messages) does not continue beyond a message if: 1. the FM list of a message includes a previous marker of the same sequence; 2. the message is the marker in the different sequence, we update the FM list of that marker only. @@ -184,14 +184,13 @@ To approximate the approval weight of a message, we simply retrieve the approval Messages can have markers from different sequences in PM list and FM list, the order and referenced relationship among sequences are important for example when it comes to inheriting the PM list from parents. Thus, we need to track these sequences. Here is an example of how the markers and sequences structures would look in the Tangle: +The purple colored messages are markers. ![](https://i.imgur.com/GENej3O.png) - - #### Sequences -Whatever reason a sequence is created, we assign a new $SI = 1+max(referenceSequences)$. To prevent assigning a new $SI$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. +For whatever reason a sequence is created, we assign a new $SI = 1+max(referenceSequencesIdentifiers)$. To prevent assigning a new $SI$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. #### Sequence Rank The rank of a sequence graph is the number of sequences from the starting point to itself. The sequence ranks are shown in the figure above. From 40b1ba883dc2119e2d1508532ce48d2ffe90e41b Mon Sep 17 00:00:00 2001 From: "Ching-Hua (Vivian) Lin" Date: Thu, 4 Feb 2021 13:37:45 +0800 Subject: [PATCH 07/12] doc: Enable syntax highlight --- docs/003-markers.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 42b23b44e6..68a6346e91 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -200,7 +200,7 @@ When a new sequence is created we check the parent marker' sequences with the fu An example is **msg 10** in the figure, $[0,2], [1,1], [2,3]$ are PMs to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced PMs therein. Next any PMs that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. In the following we show an implementation of `normalizeMarkers()`, which returns the markers and sequences that will be inherited from a message. -```go= +```go // normalizeMarkers takes a set of Markers and removes each Marker that is already referenced by another Marker in the // same set (the remaining Markers are the "most special" Markers that reference all Markers in the set grouped by the // rank of their corresponding Sequence). In addition, the method returns all SequenceIDs of the Markers that were not @@ -283,7 +283,7 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *m By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to walk the Tangle. #### Algorithm -```go= +```go // IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) { // fast check: if earlier Markers have larger highest Indexes they can't be in the past cone @@ -392,4 +392,4 @@ The approval weight of every marker is easy to compute (and hence its finality t Store the approval weight * save the nodeID per marker, and do the union when we want the approval weight -> we don't want to count the mana twice -* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. \ No newline at end of file +* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. From 4aa3239fe07b62fd76b7c08a5f3266eba0910b37 Mon Sep 17 00:00:00 2001 From: apenzk Date: Thu, 11 Mar 2021 10:37:20 +0100 Subject: [PATCH 08/12] Rearange some sections --- docs/003-markers.md | 146 +++++++++++++++++++++++--------------------- 1 file changed, 77 insertions(+), 69 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 68a6346e91..deb856174c 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -16,67 +16,23 @@ In order to compute the approval weight of a given message we need to traverse Active Consensus Mana -## Detailed Design +## Definitions Let's define the terms related to markers: * **Sequence:** A Sequence is a sequence of markers. Each Sequence corresponds to a UTXO branch, which help us to track the structure independently. -* **Sequence Identifier (SI):** A Sequence Identifier is the unique identifier of a Sequence. +* **Sequence Identifier (SID):** A Sequence Identifier is the unique identifier of a Sequence. * **Marker Index (MI):** A Marker Index is the marker rank in the marker DAG. Throughout the code the marker rank will be called index. -* **marker:** A marker is a pair of numbers: SI and MI associated to a given message. Markers carrying the same SI belong to the same Sequence. +* **marker:** A marker is a pair of numbers: SID and MI associated to a given message. Markers carrying the same SID belong to the same Sequence. * **future marker (FM):** A future marker of a message is the first marker in its future cone from different sequences; this field in the message metadata is updated when the new marker is generated in the future, following the rules defined in (#future-markers). * **past marker (PM):** A past marker of a message is a marker in its past cone. For a given sequence it is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. * **sequence rank:** The rank of a sequence will be simply called rank throughout this code. Bear in mind that for clarity the marker rank is called index. +## Detailed Design -### The Sequence -Sequences are used to track the UTXO DAG branches, each branch corresponds to a sequence with a unique $SI$, and the sequences form a DAG as well. - -#### Sequence Structure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NameTypeDescription
iduint64The sequence identifier of the sequence.
parentReferencesmap[uint64]ThresholdmapThe marker referenced map of each parent marker.
rankuint64The rank of the sequence in the marker DAG.
highestIndexuint64The highest MI of the marker sequence.
lowestIndexuint64The lowest MI of the sequence.
- - -#### Create Sequence -A new sequence is created when: -1. there's a conflict in a UTXO branch. -2. the UTXO branches are aggregated. -3. UTXO branches are merged. -Each new sequence starts from a new marker. ### The Markers -Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SI, MI]$. +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SID, MI]$. #### Marker Structure @@ -111,13 +67,13 @@ A new marker is created when: 2. A new sequence is created. > :mega: to be confirmed here. -A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[SI, MI]$ is assigned to the new marker. +A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[SID, MI]$ is assigned to the new marker. > :mega: to be confirmed here. -The $SI$ is set according to the following rules: -* Inherit the SI from parents if the new marker references the latest marker of a sequence and meets the requirement to set up a new marker without initiating a new MS. -* Create a new SI if it is the first marker of a new sequence. +The $SID$ is set according to the following rules: +* Inherit the SID from parents if the new marker references the latest marker of a sequence and meets the requirement to set up a new marker without initiating a new MS. +* Create a new SID if it is the first marker of a new sequence. The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies to the rule: + Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. @@ -174,27 +130,83 @@ The FM list of a message is empty at start and gets updated when a new marker di The figure below shows an example of applying markers to the Tangle with 1 sequence only. The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in Sequence $0$ with $MI$ $1$ and $2$ respectively. For a more complete but complex example, including multiple sequences refer to the next figure. + ![](https://i.imgur.com/RluZWCJ.png) -### Markers and Approval Weight -To approximate the approval weight of a message, we simply retrieve the approval weight of its FM list. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. -### Marker Management -Messages can have markers from different sequences in PM list and FM list, the order and referenced relationship among sequences are important for example when it comes to inheriting the PM list from parents. Thus, we need to track these sequences. +### The Sequence +Sequences are used to track the UTXO DAG branches, each branch corresponds to a sequence with a unique $SID$, and the sequences form a DAG as well. -Here is an example of how the markers and sequences structures would look in the Tangle: -The purple colored messages are markers. +#### Sequence Structure -![](https://i.imgur.com/GENej3O.png) +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
iduint64The sequence identifier of the sequence.
parentReferencesmap[uint64]ThresholdmapThe marker referenced map of each parent marker.
rankuint64The rank of the sequence in the marker DAG.
highestIndexuint64The highest MI of the marker sequence.
lowestIndexuint64The lowest MI of the sequence.
+ + +#### Create Sequence +A new sequence is created when: +1. there's a conflict in a UTXO branch. +2. the UTXO branches are aggregated. +3. UTXO branches are merged. +Each new sequence starts from a new marker. #### Sequences -For whatever reason a sequence is created, we assign a new $SI = 1+max(referenceSequencesIdentifiers)$. To prevent assigning a new $SI$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. +For whatever reason a sequence is created, we assign a new $SID = 1+max(referenceSequencesIdentifiers)$. To prevent assigning a new $SID$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. #### Sequence Rank The rank of a sequence graph is the number of sequences from the starting point to itself. The sequence ranks are shown in the figure above. + + + + +## Example + +Here is an example of how the markers and sequences structures would look in the Tangle: +The purple colored messages are markers. + +![](https://i.imgur.com/GENej3O.png) + + +## Implementation details + +In the following we describe some of the functions in more detail. +### Normalization of the referenced PMs and Sequences + +Messages can have markers from different sequences in PM list and FM list, the order and referenced relationship among sequences are important for example when it comes to inheriting the PM list from parents. Thus, we need to track these sequences. + When a new sequence is created we check the parent marker' sequences with the function `normalizeMarkers()` in order from high to low rank. In this function, we remove those PMs that it's belonging sequence is referenced by others. An example is **msg 10** in the figure, $[0,2], [1,1], [2,3]$ are PMs to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced PMs therein. Next any PMs that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. @@ -279,8 +291,8 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *m } ``` -### Markers Usages: Past Cone Check -By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to walk the Tangle. +### Markers Application: Past Cone Check +By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to perform a search of the Tangle by walking by means of e.g. a Breath-First-Search. #### Algorithm ```go @@ -387,9 +399,5 @@ func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *Marker ``` -### Markers Usages: Approval Weight Estimation -The approval weight of every marker is easy to compute (and hence its finality too): to check the weight of marker $i$, the node looks at the tip list $X$, and computes the proportion of messages in $X$ whose MRRM is $i$ or greater. - -Store the approval weight -* save the nodeID per marker, and do the union when we want the approval weight -> we don't want to count the mana twice -* bitmask, we always have a fixed list of consensus mana of each epoch, we could use a bitmask to represent that. +### Markers Application: Approval Weight Estimation +To approximate the approval weight of a message, we simply retrieve the approval weight of its FM list. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. From 30bf5a5365913fb86ab001319874a58e6fde316a Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Tue, 16 Mar 2021 11:37:43 +0800 Subject: [PATCH 09/12] docs: Minor tweaks in Marker spec --- docs/003-markers.md | 66 ++++++++++++++++++++------------------------- 1 file changed, 29 insertions(+), 37 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index deb856174c..c79ae5bf6b 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -20,19 +20,17 @@ Active Consensus Mana Let's define the terms related to markers: * **Sequence:** A Sequence is a sequence of markers. Each Sequence corresponds to a UTXO branch, which help us to track the structure independently. -* **Sequence Identifier (SID):** A Sequence Identifier is the unique identifier of a Sequence. -* **Marker Index (MI):** A Marker Index is the marker rank in the marker DAG. Throughout the code the marker rank will be called index. -* **marker:** A marker is a pair of numbers: SID and MI associated to a given message. Markers carrying the same SID belong to the same Sequence. -* **future marker (FM):** A future marker of a message is the first marker in its future cone from different sequences; this field in the message metadata is updated when the new marker is generated in the future, following the rules defined in (#future-markers). -* **past marker (PM):** A past marker of a message is a marker in its past cone. For a given sequence it is set to the newest past marker of its parents, that is the one that has the largest MI. The past marker of a marker is set to itself. +* **Sequence Identifier (`SID`):** A Sequence Identifier is the unique identifier of a Sequence. +* **Marker Index (`MI`):** A Marker Index is the marker rank in the marker DAG. Throughout the code the marker rank will be called index. +* **marker:** A marker is a pair of numbers: `SID` and `MI` associated to a given message. Markers carrying the same `SID` belong to the same Sequence. +* **future marker (`FM`):** A future marker of a message is the first marker in its future cone from different sequences; this field in the message metadata is updated when the new marker is generated in the future, following the rules defined in [Future Markers](#future-markers). +* **past marker (`PM`):** A past marker of a message is a marker in its past cone. For a given sequence it is set to the newest past marker of its parents, that is the one that has the largest `MI`. The past marker of a marker is set to itself. * **sequence rank:** The rank of a sequence will be simply called rank throughout this code. Bear in mind that for clarity the marker rank is called index. ## Detailed Design - - ### The Markers -Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SID, MI]$. +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[`SID`, `MI`]$. #### Marker Structure @@ -67,18 +65,18 @@ A new marker is created when: 2. A new sequence is created. > :mega: to be confirmed here. -A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[SID, MI]$ is assigned to the new marker. +A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[`SID`, `MI`]$ is assigned to the new marker. > :mega: to be confirmed here. -The $SID$ is set according to the following rules: -* Inherit the SID from parents if the new marker references the latest marker of a sequence and meets the requirement to set up a new marker without initiating a new MS. -* Create a new SID if it is the first marker of a new sequence. +The `SID` is set according to the following rules: +* Inherit the `SID` from parents if the new marker references the latest marker of a sequence and meets the requirement to set up a new marker without initiating a new MS. +* Create a new `SID` if it is the first marker of a new sequence. -The $MI$ is set to $MI = 1+ max(referencedMI)$, which complies to the rule: -+ Marker indexes ($MIs$) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. +The `MI` is set to $MI = 1+ max(referencedMI)$, which complies to the rule: ++ Marker indexes (`MI`s) are monotonically increasing such that $\forall x \in fc(y)$ => $MI_x > MI_y$, where $fc(y)$ is the future cone of $y$ and $x$ is any message in that future cone. -### Past Markers and Future Markers +### Markers in Messages Each message keeps its associated marker information in two lists: * past markers * future markers @@ -117,26 +115,25 @@ StructureDetails is a structure that will be in the message metadata containing
##### Past Markers -* The PM list of a marker contains the marker itself only. -* The PM list of non-marker messages is inherited from its **strong** parents, with 2 steps: - 1. for a given sequence select only the nearest marker (i.e. the markers with the highest MI). Thus for every sequence from the parents there will be exactly one marker. +* The `PM` list of a marker contains the marker itself only. +* The `PM` list of non-marker messages is inherited from its **strong** parents, with 2 steps: + 1. for a given sequence select only the nearest marker (i.e. the markers with the highest `MI`). Thus for every sequence from the parents there will be exactly one marker. 2. remove those that have been referenced by other markers from this set. ##### Future Markers -The FM list of a message is empty at start and gets updated when a new marker directly or indirectly references it. The propagation of a FM to its past cone (i.e. the update of the `FutureMarkers` field in the encountered messages) does not continue beyond a message if: -1. the FM list of a message includes a previous marker of the same sequence; -2. the message is the marker in the different sequence, we update the FM list of that marker only. +The `FM` list of a message is empty at start and gets updated when a new marker directly or indirectly references it. The propagation of a `FM` to its past cone (i.e. the update of the `FutureMarkers` field in the encountered messages) does not continue beyond a message if: +1. the `FM` list of a message includes a previous marker of the same sequence; +2. the message is the marker in the different sequence, we update the `FM` list of that marker only. -The figure below shows an example of applying markers to the Tangle with 1 sequence only. The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in Sequence $0$ with $MI$ $1$ and $2$ respectively. +The figure below shows an example of applying markers to the Tangle with 1 sequence only. The yellow messages are markers with identifiers: $[0,1]$ and $[0,2]$. Both markers are in Sequence **0** with `MI` **1** and **2** respectively. For a more complete but complex example, including multiple sequences refer to the next figure. ![](https://i.imgur.com/RluZWCJ.png) - ### The Sequence -Sequences are used to track the UTXO DAG branches, each branch corresponds to a sequence with a unique $SID$, and the sequences form a DAG as well. +Sequences are used to track the UTXO DAG branches, each branch corresponds to a sequence with a unique `SID`, and the sequences form a DAG as well. #### Sequence Structure @@ -183,17 +180,13 @@ A new sequence is created when: Each new sequence starts from a new marker. #### Sequences -For whatever reason a sequence is created, we assign a new $SID = 1+max(referenceSequencesIdentifiers)$. To prevent assigning a new $SID$ when combining same sequences again, we build parents-child relation in a map if a new sequence is created. +For whatever reason a sequence is created, we assign a new $SID = 1+max(referenceSequencesIdentifiers)$. To prevent assigning a new `SID` when combining same sequences again, we build parents-child relation in a map if a new sequence is created. #### Sequence Rank The rank of a sequence graph is the number of sequences from the starting point to itself. The sequence ranks are shown in the figure above. - - - ## Example - Here is an example of how the markers and sequences structures would look in the Tangle: The purple colored messages are markers. @@ -201,15 +194,14 @@ The purple colored messages are markers. ## Implementation details - In the following we describe some of the functions in more detail. -### Normalization of the referenced PMs and Sequences -Messages can have markers from different sequences in PM list and FM list, the order and referenced relationship among sequences are important for example when it comes to inheriting the PM list from parents. Thus, we need to track these sequences. +### Normalization of the referenced PMs and Sequences +Messages can have markers from different sequences in `PM` list and `FM` list, the order and referenced relationship among sequences are important for example when it comes to inheriting the `PM` list from parents. Thus, we need to track these sequences. -When a new sequence is created we check the parent marker' sequences with the function `normalizeMarkers()` in order from high to low rank. In this function, we remove those PMs that it's belonging sequence is referenced by others. +When a new sequence is created we check the parent marker' sequences with the function `normalizeMarkers()` in order from high to low rank. In this function, we remove those `PM`s that it's belonging sequence is referenced by others. -An example is **msg 10** in the figure, $[0,2], [1,1], [2,3]$ are PMs to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced PMs therein. Next any PMs that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. +An example is **msg 10** in the figure above, $[0,2], [1,1], [2,3]$ are `PM`s to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced `PM`s therein. Next any `PM`s that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. In the following we show an implementation of `normalizeMarkers()`, which returns the markers and sequences that will be inherited from a message. ```go @@ -292,9 +284,9 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *m ``` ### Markers Application: Past Cone Check -By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to perform a search of the Tangle by walking by means of e.g. a Breath-First-Search. +By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to perform a search of the Tangle by walking by means of e.g. a Breadth-First Search. -#### Algorithm +#### Implementation ```go // IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) { @@ -400,4 +392,4 @@ func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *Marker ``` ### Markers Application: Approval Weight Estimation -To approximate the approval weight of a message, we simply retrieve the approval weight of its FM list. Since the message is in the past cone of its FMs, the approval weight and the finality will be at least the same as its FMs. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. +To approximate the approval weight of a message, we simply retrieve the approval weight of its `FM` list. Since the message is in the past cone of its `FM`s, the approval weight and the finality will be at least the same as its `FM`s. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. From 9f98d5a79b8040a9faab9edd26d741cd34e6c9c4 Mon Sep 17 00:00:00 2001 From: jkrvivian Date: Tue, 16 Mar 2021 11:43:22 +0800 Subject: [PATCH 10/12] docs: Add referal to approval weight spec --- docs/003-markers.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/003-markers.md b/docs/003-markers.md index c79ae5bf6b..970f2b1d1b 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -393,3 +393,6 @@ func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *Marker ### Markers Application: Approval Weight Estimation To approximate the approval weight of a message, we simply retrieve the approval weight of its `FM` list. Since the message is in the past cone of its `FM`s, the approval weight and the finality will be at least the same as its `FM`s. This will of course be a lower bound (which is the “safe” bound), but if the markers are set frequently enough, it should be a good approximation. + +Details of managing approval weight of each marker and approval weight calculation of each messages refer to the Approval Weight Spec. +>:mega: Add link here. From 707d36e1080cd775cecace4e9d337c7b67be1fd3 Mon Sep 17 00:00:00 2001 From: apenzk Date: Wed, 17 Mar 2021 09:37:11 +0100 Subject: [PATCH 11/12] minor correction --- docs/003-markers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 970f2b1d1b..963176e305 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -30,7 +30,7 @@ Let's define the terms related to markers: ## Detailed Design ### The Markers -Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[`SID`, `MI`]$. +Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SID, MI]$. #### Marker Structure From 0b79d0714872876b2ebd9dd9802f660ff2acb274 Mon Sep 17 00:00:00 2001 From: apenzk Date: Wed, 17 Mar 2021 09:44:25 +0100 Subject: [PATCH 12/12] minor change 2 --- docs/003-markers.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/003-markers.md b/docs/003-markers.md index 963176e305..614682e05a 100644 --- a/docs/003-markers.md +++ b/docs/003-markers.md @@ -27,7 +27,7 @@ Let's define the terms related to markers: * **past marker (`PM`):** A past marker of a message is a marker in its past cone. For a given sequence it is set to the newest past marker of its parents, that is the one that has the largest `MI`. The past marker of a marker is set to itself. * **sequence rank:** The rank of a sequence will be simply called rank throughout this code. Bear in mind that for clarity the marker rank is called index. -## Detailed Design +## Design ### The Markers Markers are messages selected from the tip set periodically and assigned unique identifiers, in the form of $[SID, MI]$. @@ -65,7 +65,7 @@ A new marker is created when: 2. A new sequence is created. > :mega: to be confirmed here. -A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[`SID`, `MI`]$ is assigned to the new marker. +A new marker is selected from the strong tips set randomly, and selected from the weak tips set if there's no strong tip. A new pair of $[SID, MI]$ is assigned to the new marker. > :mega: to be confirmed here. @@ -203,7 +203,7 @@ When a new sequence is created we check the parent marker' sequences with the fu An example is **msg 10** in the figure above, $[0,2], [1,1], [2,3]$ are `PM`s to be considered to inherit. $[2,3]$ is the first marker to check, since it has the highest sequence rank. We select the parent sequences of $[2,3]$, which are $0$ and $1$, and the referenced `PM`s therein. Next any `PM`s that are already referenced can be removed. This results in that the PMs of **msg 10** is $[2,3]$ only. -In the following we show an implementation of `normalizeMarkers()`, which returns the markers and sequences that will be inherited from a message. +In the following we show the implementation of `normalizeMarkers()`, which returns the markers and sequences that will be inherited from a message. ```go // normalizeMarkers takes a set of Markers and removes each Marker that is already referenced by another Marker in the // same set (the remaining Markers are the "most special" Markers that reference all Markers in the set grouped by the @@ -286,7 +286,7 @@ func (m *Manager) normalizeMarkers(markers *Markers) (normalizedMarkersByRank *m ### Markers Application: Past Cone Check By comparing the past and future markers of messages, we can easily tell if one is in another's past cone. The function returns a `TriBool` representing the three possible statuses: `True`, `False` and `Maybe`. If `Maybe` is returned, then we need to perform a search of the Tangle by walking by means of e.g. a Breadth-First Search. -#### Implementation +In the following we show the implementation of the past cone check: ```go // IsInPastCone checks if the earlier Markers are directly or indirectly referenced by the later Markers. func (m *Manager) IsInPastCone(earlierMarkers *MarkersPair, laterMarkers *MarkersPair) (referenced TriBool) {