Log entry checksum validation

[killme2008]

Motivation:

LogEntry doesn't have any data checksum validation right now, it's a data integrity risk here. We should check log data's integrity after reading it from network or disk.

Modification:

1. Adds a new log entry codec implementation LogEntryV2CodecFactory that encoding/decoding log entry by protobuf, and it adds a header to the message:

 0   1    2    3  4  5
+-+-+-+-+-+-+-+-++-+-+-+
|Magic|Version|Reserved|
+-+-+-+-+-+-+-+-++-+-+-+

Test the performance between v1 and v2 codec in LogEntryCodecPerfTest, do about 200k times to encoding/decoding log entry in 512 bytes:

V1 codec cost:148 ms.
Total log size:120540000 bytes.
V2 codec cost:302 ms.
Total log size:118014624 bytes.

The encoded log size is reduced about 1%~2% but cost more time, i think it's acceptable.

2. Adds new methods to LogEntry:
   • hasChecksum() return true when has a checksum.
   • checksum() caculate the checksum of the log entry.
   • isCorrupted() returns true when the log entry has checksum and mismatched.

The checksum is caculated by CRC64 and xor.

3. The LogEntryV2CodecFactory uses AutoDetectDecoder to detect the codec version automatically and chooses the right decoder to decode the log entry. We keep log codec compatibility in such way.

4. Adds checksum validation in NodeImpl#handleAppendEntriesRequest,LogManagerImpl#getEntry and LogManagerImpl#getTermFromLogStorage, i introduce a new raft option RaftOptions#enableLogEntryChecksum to control the behaviour, it's false(disabled) by default.

5. Deprecate LogEntry#encode and LogEntry#decode, you should use LogEntryEncoder and LogEntryDecoder instead. Also , deprecate LogStorage#getTerm(index),it can be replaced by getEntry(index).getId().getTerm(), because it don't want to do checksum validation in log storage but do it all in LogManagerImpl.

Result:

Fixes #105

[killme2008]

冲突暂时不需要处理，等 #110 测试通过。

[fengjiachun]

这个 PR 有点大，还没看完，我先歇会

[fengjiachun]

@killme2008 看完了，提了几个意见你先看下

[killme2008]

@fengjiachun 基本都改完了，你可以再看下，但是不着急合并。 V2 codec 这块性能还有改善的空间，我还是坚持用 protobuf，但是可以再想办法降低下损耗。

[fengjiachun]

@fengjiachun 基本都改完了，你可以再看下，但是不着急合并。 V2 codec 这块性能还有改善的空间，我还是坚持用 protobuf，但是可以再想办法降低下损耗。

OK

[killme2008]

@fengjiachun 可以看下最新两个 commit ，优化后， 200 万次（并发20个线程）的 encode/decode 512 个字节数据的 log entry，三次结果为：

V1 codec cost:755 ms.
Total log size:1205400000 bytes.
V2 codec cost:1135 ms.
Total log size:1183706496 bytes.


V1 codec cost:883 ms.
Total log size:1205400000 bytes.
V2 codec cost:1184 ms.
Total log size:1183706496 bytes.


V1 codec cost:836 ms.
Total log size:1205400000 bytes.
V2 codec cost:1197 ms.
Total log size:1183706496 bytes.

这个差距我觉的暂时是可以接受的，大小缩小在 1%~2% 之间， data 是完全随机的数据。

[fengjiachun]

赞👍 节省了一次 toBytesArray 的 memory copy

[killme2008]

这个先合并掉？ @fengjiachun