Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proto_array fork choice #804

Merged
merged 88 commits into from Jan 29, 2020
Merged

Add proto_array fork choice #804

merged 88 commits into from Jan 29, 2020

Conversation

paulhauner
Copy link
Member

@paulhauner paulhauner commented Jan 13, 2020

Issue Addressed

NA

Proposed Changes

  • Removes ReducedTree and replaces it with ProtoArray.
  • Adds a Prometheus metrics to count info/warn/erro/crit logs
  • Adds an API endpoint that gets our ProtoArray struct as JSON.

Notes

Spec v0.10.1 introduces changes to fork choice that affect the CheckpointManager. I have not included these changes here yet, they will require a refactor of CheckpointManager. So, when you're reviewing that section, remember that it will soon be refactored (this is why it doesn't have tests).

TODO:

  • Remove Age's sync-bug-finder: 268dd06 (waiting for @AgeManning to come back online before I do this)
  • Ensure that effective balance is used instead of actual balance
  • Ensure that blocks are always imported parent-first

@paulhauner paulhauner added the work-in-progress PR is a work-in-progress label Jan 13, 2020
@paulhauner paulhauner added this to the v0.1.2 milestone Jan 13, 2020
@paulhauner paulhauner self-assigned this Jan 13, 2020
@paulhauner paulhauner added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Jan 28, 2020
@paulhauner paulhauner marked this pull request as ready for review January 28, 2020 01:46
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really tasty! 🍴 😋

Just a few minor changes requested. I haven't had a chance to review the testing code, but have done quite a deep dive into everything else.

beacon_node/beacon_chain/src/fork_choice.rs Outdated Show resolved Hide resolved
/// If some balances are found, they are removed from the cache.
pub fn get(&mut self, block_root: Hash256) -> Option<Vec<u64>> {
let i = self.position(block_root)?;
Some(self.items.remove(i).balances)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was surprised by the remove here, let me see if I have the reasoning right.

Whenever we add a block to fork choice, we call CheckpointManager::process_state, which attempts to fetch the balances from the cache for that block's state's current_justified_checkpoint. In the best case, this current justified checkpoint (CJC) is the same as the CJC for the last block processed, so we remove its entry from the cache (a hit), and then re-add an equivalent entry when we call BalancesCache::process_state. So the only time balances should actually need to be loaded from an on-disk state is on the epoch boundary where the CJC usually updates, and will not be found in the cache?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow your description, so perhaps I can respond with my own:

  • Every time we call process_state with a state that is from the first block in the epoch, we cache it's balances.
  • Whenever we process a block that has a new higher CJC, we try the cache for the balances associated with CJC.root.
    • Since the cache has a size of 4, if there are no forks we should get a cache hit as long as there haven't been 4 epochs since justification. If this is not the case (or there are forks washing out the cache) then we load from an on-disk cache.

WRT to the remove, I think you're right that we might end up doing unnecessary on-disk reads when we don't update current. I'll fix this up in the v0.10.1 update -- in it's current state I don't think it will produce errors, just inefficiency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I agree with your assessment. I'll leave this convo unresolved so we have a pointer back to it for v0.10.1

eth2/utils/remote_beacon_node/src/lib.rs Show resolved Hide resolved
eth2/proto_array_fork_choice/src/proto_array.rs Outdated Show resolved Hide resolved
eth2/proto_array_fork_choice/src/proto_array.rs Outdated Show resolved Hide resolved
eth2/proto_array_fork_choice/src/proto_array.rs Outdated Show resolved Hide resolved
@michaelsproul michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Jan 28, 2020
@paulhauner
Copy link
Member Author

Alrighty, I think we're all good here :) Let see if tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
waiting-on-author The reviewer has suggested changes and awaits thier implementation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants