You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, datanode keeps a full in-memory bloom filter for all segments. This is mainly to check if a delete message hits and to reduce delete data.
I suggest removing the bf from datanode for these reasons:
Maintaining bf only optimizes in two cases:
a) User deletes lots of non-existent primary keys
b) User uses upsert as insert.
We can rely on L0 compaction for the two cases. With batched bf, L0 compaction is fast enough to handle these cases (this can be tested).
Is there an existing issue for this?
What would you like to be added?
Currently, datanode keeps a full in-memory bloom filter for all segments. This is mainly to check if a delete message hits and to reduce delete data.
I suggest removing the bf from datanode for these reasons:
We can rely on L0 compaction for the two cases. With batched bf, L0 compaction is fast enough to handle these cases (this can be tested).
Why is this needed?
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: