[IOTDB-330] Improve the reading method of unsequence data #619
Conversation
samperson1997
changed the title
| index++; | ||
| if (index < metaDataList.size()) { | ||
| metaData = metaDataList.get(index); | ||
| nextMetaDataStartTime = metaData.getStartTime(); |
There was a problem hiding this comment.
What if here index = metaDataList.size() and the last updated nextMetaDataStartTime <= priorityMergeReader.current(), will the while (priorityMergeReader.current().getTime() >= nextMetaDataStartTime) loop forever?
There was a problem hiding this comment.
Hi, thanks for your reminding! I have modified the code logic in a better way.
| public void close() throws IOException { | ||
| while (!heap.isEmpty()) { | ||
| Element e = heap.poll(); | ||
| e.close(); | ||
| } | ||
| } | ||
|
|
||
| class Element { | ||
| public class Element { | ||
|
|
||
| IPointReader reader; |
There was a problem hiding this comment.
From my current understanding, there is a future TODO:
ChunkReader is going to implement only the IBatchReader operator. In other words, there will be no more IPointReader or related methods such as TimeValuePair timeValuePair = reader.next();.
There was a problem hiding this comment.
The future TODO is out of the scope of this pr. This pr is good.
There was a problem hiding this comment.
Yes, I intend to make DiskChunkReader and MemChunkReader only implement IBatchReader in next step.
| metaData = metaDataList.get(index++); | ||
| if (priorityMergeReader.current().getTime() < metaData.getStartTime()) { | ||
| break; | ||
| } |
There was a problem hiding this comment.
Thanks for your quick reaction.
But ... the modified logic senses more strange to me, especially the two places:
- The
index++logic makes the lastmetaDataofmetaDataListgot at line 133 can never be added when itsgetStartTime() <= priorityMergeReader.current().getTime(), because the program will exit in the next loop at line 129. - the
addReaderWithPriorityis directly used before applying theif (priorityMergeReader.current().getTime() < metaData.getStartTime())check.
You can modify or discuss about this later. Take your time :)
There was a problem hiding this comment.
Sorry for my fault... I have changed the order, how do you think about the codes now? 😂
* add batch reader interfaces * modify docs in SeriesReaderWithoutValueFilter * Add data to TSExecuteStatementResp (#631) * add fill buffer in EngineDataSetWithoutValueFilter (#646) * [IOTDB-330] Improve the reading method of unsequence data (#619) * change UnseqResourceMergeReader to IBatchReader * Fix bug of "Has not execute query" error when querying (#656) * Original query process (#653) * add aggregation reader * Update SeriesReaderWithoutValueFilter.java * fix code-coverage stage does not compile service-rpc and some other modules Co-authored-by: Jackie Tien <JackieTien@foxmail.com> Co-authored-by: Dawei Liu <atoildw@163.com> Co-authored-by: Zesong Sun <szs19@mails.tsinghua.edu.cn> Co-authored-by: Lei Rui <33376433+LeiRui@users.noreply.github.com>
Improve the reading method of unsequence data:
UnseqResourceMergeReaderimplementIBatchReader, and usePriorityMergeReaderas a variable instead of extending it.PriorityMergeReadernot implementIPointReader, which returns Element directly instead of returning TimeValuePairPriorityMergeReaderheap element into time and value