-
-
Notifications
You must be signed in to change notification settings - Fork 77
Open
Labels
Description
Currently the memtable rotation happens in the write hot path when the memtable size threshold is reached:
Lines 747 to 752 in 9395ba1
| pub(crate) fn check_memtable_overflow(&self, size: u32) -> crate::Result<()> { | |
| if size > self.config.max_memtable_size { | |
| self.rotate_memtable()?; | |
| self.check_journal_size(); | |
| self.check_write_halt(); | |
| } |
The performance hit mostly comes from the journal rotation which needs to fsync 2x (?) to disk.
It would be better for performance to send a message via channel, announcing that the partition needs to be flushed, nullifying rotation costs inside the write hot path, which should give quite a performance boost for write heavy applications (currently it is slightly slower than Rocks).
The messages should be picked up by a worker that rotates the partition asynchronously, essentially performing the Partition::rotate_memtable function:
Line 633 in 9395ba1
| pub fn rotate_memtable(&self) -> crate::Result<bool> { |
Reactions are currently unavailable