Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use headblock for prunePostBlockOperationPools, remove duplicate markInclusionBLStoExecutionChange calls #12085

Merged
merged 24 commits into from Mar 14, 2023

Conversation

james-prysm
Copy link
Contributor

@james-prysm james-prysm commented Mar 6, 2023

What type of PR is this?

Bug fix

What does this PR do? Why is it needed?

We are accidentally looping twice on marking bls changes for inclusion, and also using the current processed block instead of the head block. I migrated this function along with others in handlePostBlockOperations to onBlock to correctly deal with only canonical blocks.

@james-prysm james-prysm requested a review from potuz March 6, 2023 20:20
@james-prysm james-prysm requested a review from a team as a code owner March 6, 2023 20:20
@james-prysm james-prysm added the Bug Something isn't working label Mar 6, 2023
Copy link
Member

@terencechain terencechain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good to me. You probably want to rename the title a little bit

@james-prysm james-prysm changed the title removing duplicate call to blsToExecutionChange mark for inclusion remove duplicate blsToExecutionChange mark for inclusion calls and use head block instead of current block Mar 6, 2023
@james-prysm james-prysm changed the title remove duplicate blsToExecutionChange mark for inclusion calls and use head block instead of current block use headblock for postBlockOperations, remove duplicate markInclusionBLStoExecutionChange calls Mar 8, 2023
Copy link
Contributor

@potuz potuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, only found the race condition that requires head to be locked.

if err := s.forkchoiceUpdateWithExecution(ctx, headRoot, s.CurrentSlot()+1); err != nil {
return err
}

if isNewHead {
// uses the newly saved block from forkchoiceUpdateWithExecution
headBlock, err := s.headBlock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to lock head here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will use HeadBlock(ctx) instead

return err
}
// Handle post block operations such as attestations and exits.
if err := s.handlePostBlockOperations(headBlock.Block()); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the function handlePostBlockOperations only marks as included operations in our pools right? we may as well rename this function to pruneOperationPools or similar.

if err := s.forkchoiceUpdateWithExecution(ctx, headRoot, s.CurrentSlot()+1); err != nil {
return err
}

if isNewHead {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can consider moving this block to ReceiveBlock to alleviate the cognitive value of onBlock

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may need bigger refactors to do this, blockRoot passed in is checked against headRoot first as a comparison that's why this check if afterwards. would need to move more things around to enable this. maybe we can keep it here until I move bigger pieces.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved it based on slack discussion.

@james-prysm james-prysm changed the title use headblock for postBlockOperations, remove duplicate markInclusionBLStoExecutionChange calls use headblock for prunePostBlockOperationPools, remove duplicate markInclusionBLStoExecutionChange calls Mar 8, 2023
@james-prysm james-prysm requested a review from potuz March 8, 2023 17:37
@@ -48,6 +49,9 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig

s.cfg.ForkChoiceStore.Lock()
defer s.cfg.ForkChoiceStore.Unlock()
// Checks if the current blockRoot ( which is also the head root) is new.
// This check must come before forkchoiceUpdateWithExecution because it saves the new head.
isNewHead := s.isNewHead(blockRoot)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do this here versus next to the usage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to get is New Head before forkchoiceUpdateWithExecution saves the head which happens inside the onblock. otherwise it's not new anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work either: an incoming block will always have isNewHead==true here. You need to save the headroot here and then after the call to onBlock check the the new head.

func (s *Service) handlePostBlockOperations(b interfaces.ReadOnlyBeaconBlock) error {
// prunePostBlockOperationPools only runs on new head otherwise should return a nil.
func (s *Service) prunePostBlockOperationPools(ctx context.Context, root [32]byte) error {
headRoot, err := s.HeadRoot(ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there will be a dead lock. HeadRoot requires a read lock

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought what potuz mentioned here #12085 (comment) was to use this lock instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HeadRoot does not require a read lock, the function takes one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. HeadRoot uses headLock from Blockchain pkg not ForkchoiceLock. Confusing how there are two different locks

@@ -48,6 +49,9 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig

s.cfg.ForkChoiceStore.Lock()
defer s.cfg.ForkChoiceStore.Unlock()
// Checks if the current blockRoot ( which is also the head root) is new.
// This check must come before forkchoiceUpdateWithExecution because it saves the new head.
isNewHead := s.isNewHead(blockRoot)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work either: an incoming block will always have isNewHead==true here. You need to save the headroot here and then after the call to onBlock check the the new head.

func (s *Service) handlePostBlockOperations(b interfaces.ReadOnlyBeaconBlock) error {
// prunePostBlockOperationPools only runs on new head otherwise should return a nil.
func (s *Service) prunePostBlockOperationPools(ctx context.Context, root [32]byte) error {
headRoot, err := s.HeadRoot(ctx)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HeadRoot does not require a read lock, the function takes one.

potuz
potuz previously approved these changes Mar 14, 2023
Copy link
Contributor

@potuz potuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 171 to 174
// uses the newly saved block from forkchoiceUpdateWithExecution
headBlock, err := s.HeadBlock(ctx)
if err != nil {
return err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just pass the block as part of the argument? We can save a block copy

@@ -57,7 +57,8 @@ func (s *Service) ReceiveBlock(ctx context.Context, block interfaces.ReadOnlySig
}

// Handle post block operations such as pruning exits and bls messages.
if err := s.prunePostBlockOperationPools(ctx, blockRoot); err != nil {
// Prune service config vlaues based on the block copy if it's the head.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean? Maybe Prune block operations if it's the head

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can rephrase

@james-prysm james-prysm merged commit f92d492 into develop Mar 14, 2023
5 checks passed
@delete-merged-branch delete-merged-branch bot deleted the duplicate-changes-removal branch March 14, 2023 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants