Prevent segment inconsistencies

[jeroiraz]

• integrity checking into lsm wal files (new versioned wal file format)
• file syncing when a memtable is dumped as a segment
• crash tolerant segment-compaction

Integrity checking was added to WAL files in a backward compatible manner. The new WAL entries data format is defined as follows:

 |---------------------------|
 |  version == 0 (1byte)  |
 |  record (dynamic length)  |
 |---------------------------|

-------------------------------------------------------
|  version == 1 (1byte)  |
|  type (1byte)   |
|  node length (4bytes)    |
|  node (dynamic length)   |
|  checksum (crc32 4bytes non-checksum fields so far)  |
-------------------------------------------------------

What's being changed:

Memtable serialization into a Segment file and Segment compaction process with stronger crash-tolerance

Review checklist

• Documentation has been updated, if necessary. Link to changed documentation:
• Chaos pipeline run or not necessary. Link to pipeline:
• All new code is covered by tests where it is reasonable.
• Performance tests have been run or not necessary.

[parkerduckworth]

LGTM, but I am going to run this through the chaos pipeline now, before we merge this.

Also @amourao are you ok with all the explanations in your unresolved threads?

[sonarcloud]

Quality Gate passed

Issues
7 New issues

Measures
0 Security Hotspots
No data about Coverage
2.6% Duplication on New Code

See analysis details on SonarCloud