Fix visible metadata, version, work processor bugs when recovering #966
Conversation
| return StorageEngine.getTimePartition(startTimeMap.values().iterator().next()); | ||
| } | ||
| String[] splits = FilePathUtils.splitTsFilePath(this); | ||
| return Long.parseLong(splits[splits.length - 2]); |
There was a problem hiding this comment.
If we sync a tsfile from other iotdb, will this tsfile has time partition directory? Maybe we should use startTimeMap to get time partition
There was a problem hiding this comment.
Thanks, I use the startTimeMap first, then the path
There was a problem hiding this comment.
It is not guaranteed that the other IoTDB will use the same partition interval, so it is better to check the whole startTimeMap and endTimeMap to see if any device has crossed a partition and report an exception if so.
There was a problem hiding this comment.
restart and load could use different getPartition method, one check partition and the other not. I will add a method with partition check
tsfile/src/main/java/org/apache/iotdb/tsfile/write/writer/TsFileIOWriter.java
Outdated
Show resolved
Hide resolved
|
Kudos, SonarCloud Quality Gate passed!
|
Fix many bugs when recovering:
(1) when recovering the last unclosed SEQ file, the TsFileProcessor is added to workUnsequenceTsFileProcessors.
(2) when recovering the last unclosed UNSEQ file, the TsFileProcessor isn's added to any workTsFileProcessors.
(3) when recovering, the metadata get from TsFile is set to chunkMetadataListMap, however, visibale metadata is recovered from chunkGroupInfoList. Therefore, the persisted Chunk is not visible.
(4) when recovering, the TsFileRecoverPerformer needs to receive a partitionId. However, before recovering, the startTime & endTime maps are empty. So the partitionId is -1, which leads to the wrong version of Deletion and memtable version in recovery. I get the partitionId from resource path. No more -1 partitions in the future.