Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up access to block ancestors #325

Open
Scooletz opened this issue May 9, 2024 · 1 comment
Open

Speed up access to block ancestors #325

Scooletz opened this issue May 9, 2024 · 1 comment
Labels
good first issue Good for newcomers 🐌 performance Perofrmance related issue

Comments

@Scooletz
Copy link
Contributor

Scooletz commented May 9, 2024

Currently, whenever a value is retrieved from Paprika, a list of ancestors is walked through to find whether or not the given value can be provided by it. It's optimized as it uses Xor filter to check whether the given block can serve the read or not. Still, it requires a lot of reads because it always loops over:

private ReadOnlySpanOwnerWithMetadata<byte> TryGetAncestors(scoped in Key key,
scoped ReadOnlySpan<byte> keyWritten, ulong bloom)
{
ushort depth = 1;
var destroyedHash = CommittedBlockState.GetDestroyedHash(key);
// walk all the blocks locally
foreach (var ancestor in _ancestors)
{
var owner = ancestor.TryGetLocal(key, keyWritten, bloom, destroyedHash, out var succeeded);
if (succeeded)
return owner.WithDepth(depth);
depth++;
}
return TryGetDatabase(key);
}

These reads could be sped up using various approaches.

Proposals

  1. squashing the anscestors: when the block is created for processing, squash ancestors so that they can serve values fast (proposed by: @benaadams @LukaszRozmej)
  2. use a Dictionary<ulong, ushort> which would map hash -> ancestor depth that would point to the first ancestor that has the hash set. This would be much easier and lighter to create. If no value for the given ulong, this means the db should be hit.

While the first approach would save lookups making them direct (one lookup against one value) the second could provide removal of Xor filters altogether or put Xor only on top of the map created in this way (this should be measured whether it's worth it)

@Scooletz Scooletz added 🐌 performance Perofrmance related issue good first issue Good for newcomers labels May 9, 2024
@murluki
Copy link

murluki commented May 9, 2024

Could you assign me, please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers 🐌 performance Perofrmance related issue
Projects
Status: Todo
Development

No branches or pull requests

2 participants