Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

external ingest: fix bloom filters when synthetic prefix is used #3627

Closed
RaduBerinde opened this issue May 20, 2024 · 0 comments · Fixed by #3628 or #3629
Closed

external ingest: fix bloom filters when synthetic prefix is used #3627

RaduBerinde opened this issue May 20, 2024 · 0 comments · Fixed by #3628 or #3629

Comments

@RaduBerinde
Copy link
Member

We found that we do not un-apply the synthetic prefix when checking bloom filters. The result is that SeekPrefixGE will more often than not miss keys. This is a major bug that affects Online Restore and is causing corruption on tpcc tests.

I investigated why the metamorphic test did not find this issue. I found that a objDB map wasn't updated properly which lead to the code generating SeekPrefixGE operations not being able to use the keys that were written.

In addition to fixing the bug, I am identifying some key testing misses that got us here:

  • the metamorphic test code is a bit lax in testing its assumptions; for example, we should have asserted when objDB returned a 0 value
  • there aren't enough tests for the metamorphic generation code
  • there is a tendency to over-rely on the metamorphic tests; in this instance we were missing test coverage for SeekPrefixGE in the context of synthetic prefixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment