Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unmarshal Block instead of State #5246

Merged
merged 10 commits into from Mar 31, 2020
Merged

Unmarshal Block instead of State #5246

merged 10 commits into from Mar 31, 2020

Conversation

nisdas
Copy link
Member

@nisdas nisdas commented Mar 30, 2020

When deleting states a bottle neck came up where we were unmarshalling the whole state in order to just find out their slot. This unmarshals the block instead which is a much smaller object

@nisdas nisdas added the Ready For Review A pull request ready for code review label Mar 30, 2020
0xKiwi
0xKiwi previously approved these changes Mar 30, 2020
@@ -308,20 +308,21 @@ func slotByBlockRoot(ctx context.Context, tx *bolt.Tx, blockRoot []byte) (uint64
return stateSummary.Slot, nil
}

bkt := tx.Bucket(stateBucket)
bkt := tx.Bucket(blocksBucket)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually started out by using block. But one very good feedback from @prestonvanloon: what if the block is missing then the node will never be able to delete the state

I know the state is slower, I think the compromise (later) is to add a root->slot storage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an issue during sync, unmarshalling a state just to delete it is quite inefficient. If a block is missing, then how would we save its resulting state ?
i could add a fallback to retrieve the state in the event the block is missing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just preparing for the worst case (ie, user saves the state then mistakenly deletes the block.. etc)

fallback sounds good

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok done, added in a fallback

@codecov
Copy link

codecov bot commented Mar 30, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@3e81afd). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #5246   +/-   ##
=========================================
  Coverage          ?   55.15%           
=========================================
  Files             ?      267           
  Lines             ?    21502           
  Branches          ?        0           
=========================================
  Hits              ?    11860           
  Misses            ?     8032           
  Partials          ?     1610

@prylabs-bulldozer prylabs-bulldozer bot merged commit c8f8e3f into master Mar 31, 2020
@delete-merged-branch delete-merged-branch bot deleted the dontUnmarshal branch March 31, 2020 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready For Review A pull request ready for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants