Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GetAllLeaves On Channel #2081

Merged
merged 10 commits into from
Jul 10, 2020
Merged

GetAllLeaves On Channel #2081

merged 10 commits into from
Jul 10, 2020

Conversation

BeniaminDrasovean
Copy link
Contributor

@BeniaminDrasovean BeniaminDrasovean commented Jul 8, 2020

Add a new trie functionality, getAllLeavesOnChannel, which returns all the trie leaves on a channel. When all the leaves have been sent, the channel closes.

@BeniaminDrasovean BeniaminDrasovean self-assigned this Jul 8, 2020
Copy link
Contributor

@iulianpascalau iulianpascalau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, pretty much what I wanted. Minor stuff.

@@ -20,6 +20,12 @@ const (
// ModifiedHashes is used to memorize all old hashes and new hashes from when a trie is committed
type ModifiedHashes map[string]struct{}

// TrieLeaf holds the key to a leaf and the value associated with that leaf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly at the same pattern I have worked now. We might have a general implementation in core package (keyValueStorage) that will implement a core.KeyValueHolder interface. That interface will be reused in storage package also.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -602,6 +602,30 @@ func (tr *patriciaMerkleTrie) GetAllLeaves() (map[string][]byte, error) {
return leaves, nil
}

// GetAllLeavesOnChannel adds all the trie leaves to the given channel
func (tr *patriciaMerkleTrie) GetAllLeavesOnChannel(leavesChannel chan *data.TrieLeaf) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would change a little bit this:

func (tr *patriciaMerkleTrie) GetAllLeavesOnChannel() chan *data.TrieLeaf {
     ch := make(chan *data.TrieLeaf)

     //check nil part (including closing the channel)
    
    go func() {
		tr.mutOperation.RLock()
		err := tr.root.getAllLeavesOnChannel(leavesChannel, []byte{}, tr.Database(), tr.marshalizer)
		if err != nil {
			log.Error("could not get all trie leaves: ", "error", err)
		}
		tr.mutOperation.RUnlock()

                close(leavesChannel)
	}()

      return ch
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


err := tr.GetAllLeavesOnChannel(leafChannel)
assert.Nil(t, err)
checkLeaves(t, leafChannel, leaves)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be changed to this:

recovered := make(map[string][]byte)

for leaf := range leafChannel{
      recovered[string(leaf.Key)] = leaf.Value
}
assert.Equal(t, leaves, recovered)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@BeniaminDrasovean BeniaminDrasovean marked this pull request as ready for review July 9, 2020 09:44
@raduchis raduchis self-requested a review July 9, 2020 12:19
raduchis
raduchis previously approved these changes Jul 9, 2020
data/trie/patriciaMerkleTrie.go Show resolved Hide resolved
raduchis
raduchis previously approved these changes Jul 10, 2020
Copy link
Contributor

@iulianpascalau iulianpascalau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few more minor stuff

tr.mutOperation.RUnlock()
close(leavesChannel)

return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return leavesChannel in order to not output a nil chan and break down stream code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

return errNotImplemented
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of returning nil, just return

ch := make(chan core.KeyValueHolder)
close(ch)

return ch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

return errNotImplemented
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of returning nil, just return

ch := make(chan core.KeyValueHolder)
close(ch)

return ch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

return errNotImplemented
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of returning nil, just return

ch := make(chan core.KeyValueHolder)
close(ch)

return ch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

return errNotImplemented
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of returning nil, just return

ch := make(chan core.KeyValueHolder)
close(ch)

return ch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

return errNotImplemented
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of returning nil, just return

ch := make(chan core.KeyValueHolder)
close(ch)

return ch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

@LucianMincu LucianMincu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System tests passed.

@LucianMincu LucianMincu merged commit 1edf6c6 into development Jul 10, 2020
@LucianMincu LucianMincu deleted the getAllLeaves-on-channel branch July 10, 2020 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants