storage: add revised plan for loosely-coupled truncation #81113
Conversation
Release note: None
|
This change is |
| // the memtable. This is because of the per-entry overhead. This means | ||
| // there is a decent probability that the state machine memtable will | ||
| // start getting flushed before the corresponding raft engine memtable | ||
| // is flushed. If the flush is fast enough, we would be able to truncate |
There was a problem hiding this comment.
Could we also "just" run the raft instance with a larger memtable?
There was a problem hiding this comment.
This is my thinking as well. With the separate engines, we should be able to configure it specifically for this use-case -- I'm sure there are other tweaks we could make too.
There was a problem hiding this comment.
This is my thinking as well. With the separate engines, we should be able to configure it specifically for this use-case -- I'm sure there are other tweaks we could make too.
|
This can merge, Sumeer. |
I've been running some experiments using sumeerbhola@d186760 to compare the write amp for shared and separated LSMs under both loosely and tightly coupled truncation, in order to understand this better. |
Release note: None