Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add path generation for nmt subroots #621

Merged
merged 14 commits into from
Sep 2, 2022
113 changes: 113 additions & 0 deletions pkg/inclusion/paths.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
package inclusion

// genSubTreeRootPath calculates the path to a given subtree root of a node, given the
// depth and position of the node. note: the root of the tree is depth 0.
// nolint
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved
func genSubTreeRootPath(depth int, pos uint) []bool {
path := make([]bool, depth)
for i := depth; i >= 0; i-- {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the path to the root be the empty set?

Suggested change
for i := depth; i >= 0; i-- {
for i := depth; i > 0; i-- {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! the implementation changed as this was one of the bugs
9cf9f54

there's a test specifically for this now too

if (pos & (1 << i)) == 0 {
path = append(path, false)
} else {
path = append(path, true)
}
}
return path
}

// coord identifies a tree node using the depth and position
type coord struct {
// depth is the typical depth of a tree, 0 being the root
depth uint64
// position is the index of a node of a given depth, 0 being the left most
// node
position uint64
}

// climb is a state transition function to simulate climbing a balanced binary
// tree, using the current node as input and the next highest node as output.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably would be better to add to the docs that we expect canClimbRight() as a precondition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think being able to climb right is a technically a precondition? or perhaps I'm just not clear on what you mean by this? the implementation that uses this code only climbs if it can, but is that different?

func (c coord) climb() coord {
return coord{
depth: c.depth - 1,
position: c.position / 2,
}
}

// canClimbRight uses the current position to calculate the direction of the next
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this part. You can only climb in one direction, right? It is when you're going down that you have right or left path, no?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like you mentioned, with climb terminology, one could think a child node climbs up to a parent node (i.e. only one direction). But another way of thinking about this method: isLeftChild().

Maybe this comment helps? https://github.com/celestiaorg/celestia-app/pull/621/files/d2d645b024464bcbcfd7114b42e9e154f77729cf#r948000052

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Depth       Position
// 0              0
//               / \
//              /   \
// 1           0     1
//            /\     /\
// 2         0  1   2  3
//          /\  /\ /\  /\
// 3       0 1 2 3 4 5 6 7

What canClimbRight() does is, for (depth=3, position=5), check if we can climb from 5 to 2. In this case, canClimbRight() will return false.
For (depth=3, position=6), it can climb right to 3.

I guess we need to explain it with an example in the docs.

// climb. Returns true if the next climb is right (if the position (index) is
// even).
func (c coord) canClimbRight() bool {
return c.position%2 == 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we also check if depth != 0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right, we probably should just to avoid potential leaky abstractions (although I hope others do not use coord for anything 😅) as we're relying on the implementation to do this for us atm

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved

// calculateSubTreeRootCoordinates generates the sub tree root coordinates of a
// set of shares for a balanced binary tree of a given depth. It assumes that
// end does not exceed the range of a tree of the provided depth, and that end
// >= start. This function works by starting at the first index of the msg and
// working our way right.
func calculateSubTreeRootCoordinates(maxDepth, start, end uint64) []coord {
cds := []coord{}
// leafCursor keeps track current leaf that we are starting with when
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved
// finding the subtree root for some set. When leafCursor == end, we are
// finished calculating sub tree roots
leafCursor := start
// nodeCursor keeps track of the current node tree node when finding sub
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved
// tree roots
nodeCursor := coord{
depth: maxDepth,
position: start,
}
// lastNodeCursor keeps track of the last node cursor so that when we climb
// to high, we can use this node as a sub tree root
evan-forbes marked this conversation as resolved.
Show resolved Hide resolved
lastNodeCursor := nodeCursor
lastLeafCursor := leafCursor
// nodeRangeCursor keeps track of the number of leaves that are under the
// current tree node. We could calculate this each time, but this acts as a
// cache
nodeRangeCursor := uint64(1)
// reset is used to reset the above state after finding a subtree root. We
// reset by setting the node cursors to the values equal to the next leaf
// node.
reset := func() {
lastNodeCursor = nodeCursor
lastLeafCursor = leafCursor
nodeCursor = coord{
depth: maxDepth,
position: leafCursor,
}
nodeRangeCursor = uint64(1)
}
// recursively climb the tree starting at the left most leaf node (the
// starting leaf), and save each subtree root as we find it. After finding a
// subtree root, if there's still leaves left in the message, then restart
// the process from that leaf.
for {
switch {
// check if we're finished, if so add the last coord and return
case leafCursor == end:
cds = append(cds, nodeCursor)
return cds
// check if we've climbed too high in the tree. if so, add the last
// highest node and proceed.
case leafCursor > end:
cds = append(cds, lastNodeCursor)
leafCursor = lastLeafCursor + 1
reset()
// check if can climb right again (only even positions will climb
// right). If not, we want to record this coord as it is a subtree
// root, then adjust the cursor and proceed.
case !nodeCursor.canClimbRight():
cds = append(cds, nodeCursor)
leafCursor++
reset()
// proceed to climb higher by incrementing the relevant state and
// progressing through the loop.
default:
lastLeafCursor = leafCursor
lastNodeCursor = nodeCursor
leafCursor = leafCursor + nodeRangeCursor
nodeRangeCursor = nodeRangeCursor * 2
nodeCursor = nodeCursor.climb()
}
}
}
229 changes: 229 additions & 0 deletions pkg/inclusion/paths_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
package inclusion

import (
"testing"

"github.com/stretchr/testify/assert"
)

func Test_calculateSubTreeRootCoordinates(t *testing.T) {
rootulp marked this conversation as resolved.
Show resolved Hide resolved
type test struct {
name string
start, end, maxDepth uint64
expected []coord
}
tests := []test{
{
name: "first four shares of an 8 leaf tree",
start: 0,
end: 3,
maxDepth: 3,
expected: []coord{
{
depth: 1,
position: 0,
},
},
},
{
name: "second set of four shares of an 8 leaf tree",
start: 4,
end: 7,
maxDepth: 3,
expected: []coord{
{
depth: 1,
position: 1,
},
},
},
{
name: "middle 2 shares of an 8 leaf tree",
start: 3,
end: 4,
maxDepth: 3,
expected: []coord{
{
depth: 3,
position: 3,
},
{
depth: 3,
position: 4,
},
},
},
{
name: "third lone share of an 8 leaf tree",
start: 3,
end: 3,
maxDepth: 3,
expected: []coord{
{
depth: 3,
position: 3,
},
},
},
{
name: "middle 3 shares of an 8 leaf tree",
start: 3,
end: 5,
maxDepth: 3,
expected: []coord{
{
depth: 3,
position: 3,
},
{
depth: 2,
position: 2,
},
},
},
{
name: "middle 6 shares of an 8 leaf tree",
start: 1,
end: 6,
maxDepth: 3,
expected: []coord{
{
depth: 3,
position: 1,
},
{
depth: 2,
position: 1,
},
{
depth: 2,
position: 2,
},
{
depth: 3,
position: 6,
},
},
},
{
name: "first 5 shares of an 8 leaf tree",
start: 0,
end: 4,
maxDepth: 3,
expected: []coord{
{
depth: 1,
position: 0,
},
{
depth: 3,
position: 4,
},
},
},
{
name: "first 7 shares of an 8 leaf tree",
start: 0,
end: 6,
maxDepth: 3,
expected: []coord{
{
depth: 1,
position: 0,
},
{
depth: 2,
position: 2,
},
{
depth: 3,
position: 6,
},
},
},
{
name: "all shares of an 8 leaf tree",
start: 0,
end: 7,
maxDepth: 3,
expected: []coord{
{
depth: 0,
position: 0,
},
},
},
{
name: "first 32 shares of a 128 leaf tree",
start: 0,
end: 31,
maxDepth: 7,
expected: []coord{
{
depth: 2,
position: 0,
},
},
},
{
name: "first 33 shares of a 128 leaf tree",
start: 0,
end: 32,
maxDepth: 7,
expected: []coord{
{
depth: 2,
position: 0,
},
{
depth: 7,
position: 32,
},
},
},
{
name: "first 31 shares of a 128 leaf tree",
start: 0,
end: 30,
maxDepth: 7,
expected: []coord{
{
depth: 3,
position: 0,
},
{
depth: 4,
position: 2,
},
{
depth: 5,
position: 6,
},
{
depth: 6,
position: 14,
},
{
depth: 7,
position: 30,
},
},
},
{
name: "first 64 shares of a 128 leaf tree",
start: 0,
end: 63,
maxDepth: 7,
expected: []coord{
{
depth: 1,
position: 0,
},
},
},
}
for _, tt := range tests {
res := calculateSubTreeRootCoordinates(tt.maxDepth, tt.start, tt.end)
assert.Equal(t, tt.expected, res, tt.name)
}
}