[IOTDB-292]Add load external tsfile feature#597
Conversation
pay attention to the error. is there something not released? |
I will try the ut in the windows environment. |
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
Show resolved
Hide resolved
I have upgraded to antlr4. |
qiaojialin
left a comment
There was a problem hiding this comment.
修改一下加载tsfile的功能吧:
用户有一个 tsfile,放在任意位置,使用 load工具 加载文件,
传入三个参数
tsfile 文件路径
2,是否根据tsfile的schema创建元数据(可选,默认创建)
3,存储组名字(可选,默认以路径前二层创建存储组, root.sg1 这种)
load工具做的功能:
判断这个文件是否为 sequence文件,然后移动到对应文件夹,生成 resource
I don't think so. In a complete tsfile, the data type of timeseries and the storage group are fixed. It cannot change by users. I can add the feature of creating schemas automatically while the schemas don't exist. |
I have added the feature you requested, please check. |
1962de5 to
f4b4b80
Compare
| Map<String, MeasurementSchema> schemaMap = new HashMap<>(); | ||
| List<ChunkGroupMetaData> chunkGroupMetaData = new ArrayList<>(); | ||
| try (TsFileSequenceReader reader = new TsFileSequenceReader(file.getAbsolutePath(), false)) { | ||
| reader.selfCheck(schemaMap, chunkGroupMetaData, false); |
There was a problem hiding this comment.
From my understanding, the reader.selfCheck method with the parameter fastFinish assigned false will sequentially read the whole data part of the tsfile.
If the method here is only used to generate the schemaMap and chunkGroupMetaData, why not just read the metadata at the end of the tsfile, since the tsfile is always complete here?
This pr adds a new feature of loading external tsfile, it allows the following three operations:
Corresponding jira link: https://issues.apache.org/jira/browse/IOTDB-292.