Join GitHub today
Soft-pressure memtable flusher picks unflushable memtable lists even though flushable ones exist #3716
The flusher picks the memtable list which contains the largest region according to
The latest memtable in the winning list may be small (or empty), in which case the soft pressure flusher will not be able to make much progress. This can lead to writes blocking on dirty.
I observed this for the system memtable group, where it's easy for the memtables to overshoot small soft pressure limits. The flusher kept trying to flush empty memtables, while the previous non-empty memtable was still in the group.
The CPU scheduler makes this worse, because it runs
I think this issue (edit: the issue I am talking about is #3717) may now be more likely to hit due to the CPU scheduler, which spreads related tasks in time more, giving the flusher more time to OOM the memory.