[IOTDB-587] New TsFile version 2 #855
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to new_tsFile_v2
…to new_tsFile_v2
…to new_tsFile_v2
liutaohua
reviewed
Mar 5, 2020
| timeseriesMetaData.setMeasurementId(path.getMeasurement()); | ||
| timeseriesMetaData.setTSDataType(entry.getValue().get(0).getDataType()); | ||
| timeseriesMetaData.setOffsetOfChunkMetaDataList(out.getPosition()); | ||
| Statistics<?> statistics = entry.getValue().get(0).getStatistics(); |
There was a problem hiding this comment.
Repeat merge the statistics information
| @@ -261,64 +264,55 @@ public void endFile(Schema schema) throws IOException { | |||
| resourceLogger.info("{} writer is closed.", file.getName()); | |||
| } | |||
| canWrite = false; | |||
| chunkMetadataListMap = new TreeMap<>(); | |||
There was a problem hiding this comment.
why need to reapply memory here?
…into new_TsFile
Change to use duplicated path and datatypes in LastQueryExecutor
qiaojialin
approved these changes
Mar 30, 2020
…h TimeseriesMetadata
|
Kudos, SonarCloud Quality Gate passed!
|
liutaohua
approved these changes
Mar 31, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The mean differences of new TsFile are as below:
remove ChunkGroupMetadata
Modified Schema to support same measurement names with different data types
Performance Test:
4G memory.
1 Storage group，1 device，3000 measurements，each timeseries contains 600000 data points
IoTDB configuration:
enable_parameter_adapter=false
tsfile_size_threshold=1024L
memtable_size_threshold=5010241024L
Write cost:
new_TsFile：300569ms，14.76G，184 tsfiles
master：300418ms，14.73G，184 tsfiles
Query time cost:
new_TsFile: 1349ms
master: 2102ms
new_TsFile: 3268ms
master: 4621ms
new_TsFile: 647934ms
master: 814206ms
new_TsFile: 421ms
master: 1654ms
new_TsFile: 1887ms
master: 4231ms
new_TsFile: 3066ms
master: 6653ms
new_TsFile optimize cache: 1243ms
new_TsFile without optimize cache: 313599ms
master: 614638ms