Remove versionInfo in Tsfile and get rid of versions in memtable#2445
Remove versionInfo in Tsfile and get rid of versions in memtable#2445HTHou merged 27 commits intoapache:masterfrom
Conversation
SilverNarcissus
left a comment
SilverNarcissus
left a comment
There was a problem hiding this comment.
Fantastic work! I think we need some test for a special situation. There are tests for deletion in memtable and tsfile but no test for deletion in a flushing memtable. Please add some tests for it~
| ChunkMetadata metaData = chunkMetaData.get(metaIndex); | ||
| for (Modification modification : modifications) { | ||
| if (modification.getVersionNum() > metaData.getVersion()) { | ||
| if (modification.getFileOffset() > metaData.getOffsetOfChunkHeader()) { |
There was a problem hiding this comment.
Can modification.getFileOffset() == metaData.getOffsetOfChunkHeader() ? What should we do in this condition
There was a problem hiding this comment.
- metaData.getOffsetOfChunkHeader() marks the byte offset of this chunk header.
- modification.getFileOffset() returns the offset of any chunk in the file after the chunk has been flushed.
So I consider it could be guaranteed that if any chunk is successfully flushed, offset in (2) > (1).
Thats why I didn't add the equality condition.
There was a problem hiding this comment.
Reasonable! So just add some comment here and add test which I said above.
There was a problem hiding this comment.
You can also check sonar code smell. There is an unused parameter.
There was a problem hiding this comment.
Thanks, I'll add some more test scenarios.
HTHou
left a comment
HTHou
left a comment
There was a problem hiding this comment.
I checked the code related to Flushing and TsFile module, just some small log issues. Please fix them.
|
Kudos, SonarCloud Quality Gate passed!
|
4 participants
IoTDB uses an incremental field "version" to keep track of the updating order of multiple Memtables, and this version is also persisted into TsFiles.
This PR targets to remove the "version" field in Memtables and avoid serializing/deserializing it into TsFiles.
Design docs：
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173082421