Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Consensus] Own sized leveled forest tech depth #1046

Merged
merged 13 commits into from
Aug 6, 2021

Conversation

durkmurder
Copy link
Member

dapperlabs/flow-go/issues/5670

Context

This PR solves some technical depth around calculating size of leveled forest. Instead of doing this for every class that use/will use LevelledForest we would embed knowledge of own size into leveled forest.

As part of this PR I have also made next changes:

  • removed size calculation from AssignmentCollectorTree
  • added missing locks in ExecutionTree

Note on ExecutionTree: ExecutionTree still has custom logic for calculating size, since it calculates size based on how many receipts are stored but the leveled forest is built using the results. Meaning that one result can have multiple associated receipts. That is the reason why we still need custom logic.

Copy link
Contributor

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +218 to +226
et.RLock()
defer et.RUnlock()
return et.size
}

// LowestHeight returns the lowest height, where results are still stored in the mempool.
func (et *ExecutionTree) LowestHeight() uint64 {
et.RLock()
defer et.RUnlock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@zhangchiqing zhangchiqing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

I'd like to request to add a test cases that verifies the GetSize method is concurrent-safe:

Verify that adding 100 assignment collectors to the tree concurrently and in the end calling GetSize should return 100.
Turning on race flag should not report race condition.

module/forest/leveled_forrest.go Show resolved Hide resolved
module/forest/leveled_forrest.go Outdated Show resolved Hide resolved
module/forest/leveled_forrest.go Outdated Show resolved Hide resolved
module/forest/leveled_forrest_test.go Show resolved Hide resolved
module/forest/leveled_forrest_test.go Show resolved Hide resolved
durkmurder and others added 3 commits August 4, 2021 17:11
@durkmurder
Copy link
Member Author

durkmurder commented Aug 4, 2021

I'd like to request to add a test cases that verifies the GetSize method is concurrent-safe:

@zhangchiqing We currently don't have separate tests for assignment collector tree but I have an issue for it that I will be implementing soon. I would implement that test as part of it if you are ok with it. flow-go/issues/5719

@codecov-commenter
Copy link

codecov-commenter commented Aug 4, 2021

Codecov Report

Merging #1046 (b050362) into master (e1ccd89) will increase coverage by 0.02%.
The diff coverage is 93.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1046      +/-   ##
==========================================
+ Coverage   53.16%   53.18%   +0.02%     
==========================================
  Files         321      321              
  Lines       21734    21741       +7     
==========================================
+ Hits        11554    11563       +9     
+ Misses       8610     8609       -1     
+ Partials     1570     1569       -1     
Flag Coverage Δ
unittests 53.18% <93.33%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...e/consensus/approvals/assignment_collector_tree.go 0.00% <0.00%> (ø)
module/forest/leveled_forrest.go 79.80% <100.00%> (+1.91%) ⬆️
module/mempool/consensus/execution_tree.go 74.35% <100.00%> (+1.38%) ⬆️
module/mempool/stdmap/identifier_map.go 81.25% <0.00%> (-3.13%) ⬇️
...sus/approvals/assignment_collector_statemachine.go 47.11% <0.00%> (-2.89%) ⬇️
cmd/util/ledger/migrations/storage_v4.go 42.16% <0.00%> (+0.60%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1ccd89...b050362. Read the comment docs.

Copy link
Member

@zhangchiqing zhangchiqing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently don't have separate tests for assignment collector tree but I have an issue for it that I will be implementing soon. I would implement that test as part of it if you are ok with it.

Sounds good thanks.

module/forest/leveled_forrest_test.go Outdated Show resolved Hide resolved
Comment on lines 305 to 311
vertexId := strconv.FormatUint(1, 10)
parentId := strconv.FormatUint(parentLevel, 10)
for i := uint64(1); i <= numberOfNodes; i++ {
// add 10 nodes at same level
F.AddVertex(NewVertexMock(vertexId, i, parentId, parentLevel))
}
assert.Equal(t, uint64(1), F.GetSize())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are adding the same nodes 10 times, I was thinking a test case that adds 10 different nodes twice, and verify the size didn't change:

// nodes := createNodes(10)
//
// for each node
//    F.AddVertex(node)
//
// size := F.GetSize() 
//
// for each node
//    F.AddVertex(node)
//
// assert.Equal(size, F.GetSize())

basically I don't care what's the value for GetSize, I only care whether it's changed or not.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, ok

@durkmurder durkmurder merged commit 30f50e2 into master Aug 6, 2021
@durkmurder durkmurder deleted the yurii/5670-own-sized-levelled-forest branch August 6, 2021 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants