Remove unnecessary intermediate serialization in RaftLog#append #6671
Labels
area/performance
Marks an issue as performance related
kind/toil
Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc.
scope/broker
Marks an issue or PR to appear in the broker section of the changelog
version:8.1.0
Marks an issue as being completely or in parts released in 8.1.0
Description
At the moment, we first serialize the Raft log entry before passing it to the journal. This works fine, but requires us to have an intermediate in memory buffer. This means we have an extra copy in the journal when appending, and we have this expendable, long living buffer in memory.
Proposal:
What we need is a way to serialize only when we have the final destination buffer. One easy way is to amend the Journal interface to add a
public JournalRecord append(int entryLength, Consumer<MutableDirectBuffer> entryWriter);
. Or we we want to obviate the need for lambdas:In the journal, we can then serialize the record data when we need it directly to the writer's buffer instead of using an intermediate write buffer.
NOTE: if we can make it without using generics, even better 🙂
/cc @deepthidevaki
The text was updated successfully, but these errors were encountered: