Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Clean up translog interface #7564
This has two major parts:
Get rid of readSource() entirely, it sucked, Operations should be able
No more TranslogStream headers, you are now required to pass an
The interface now exists of 2 main methods, and one helper:
// read Translog.Operation read(StreamInput in) // write void write(StreamOutput out, Translog.Operation op) // when reading a file, need to be able to read and consume the header void seekPastHeader(StreamInput in)
Previously we handled these integers outside of the translog stream
There is some additional code in LocalIndexShardGateway to handle the
Additionally, we need to handle a case where the header is truncated
I'm hoping this makes working with the translog easier and less error-prone. On top of this (in a separate PR) we can add the size-reading safety to prevent rare OOMEs on corrupted translogs.
@s1monw so I'm kind of concerned about one of the changes I made here, which is that we now serialize the size of the operation when sending translog operations between nodes. If it were just a size it wouldn't be a big deal, but to get the size we create a new byte array for every operation, which seems much slower.
What do you think of keeping