[IOTDB-536] Process about nonExistMeasurement and ConstantMeasurement in AlignByDevice is complex and not intuitive #874
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
qiaojialin
reviewed
Mar 3, 2020
server/src/main/java/org/apache/iotdb/db/qp/physical/crud/AlignByDevicePlan.java
Outdated
Show resolved
Hide resolved
| // update measurementSetOfGivenSuffix and measurement location | ||
| if (measurementSetOfGivenSuffix.add(measurementChecked)) { | ||
| loc++; | ||
| // update measurementSetOfGivenSuffix and Normal measurement |
There was a problem hiding this comment.
rename the pathForDataType to pathWithAggregation
There was a problem hiding this comment.
Sorry? Apparently it may without aggregation.
| * The data type is considered as String and the value is considered is the same with measurement name. | ||
| */ | ||
| public enum measurementType { | ||
| Normal, NonExist, Const; |
There was a problem hiding this comment.
Suggested change
| Normal, NonExist, Const; | |
| Exist, NonExist, Constant |
| * Const: the measurements that have quotation mark. e.g. "abc",'11'. | ||
| * The data type is considered as String and the value is considered is the same with measurement name. | ||
| */ | ||
| public enum measurementType { |
There was a problem hiding this comment.
Suggested change
| public enum measurementType { | |
| public enum MeasurementType { |
…nByDevicePlan.java Co-Authored-By: Jialin Qiao <qjl16@mails.tsinghua.edu.cn>
…tdb into AlignByDeviceLogic
qiaojialin
approved these changes
Mar 5, 2020
|
|
||
| ## 设计原理 | ||
|
|
||
| 按设备对齐查询其实现原理主要是计算出查询中每个设备对应的映射值和过滤条件，然后将查询按设备分别进行，最后将结果集拼装并返回。 |
|
|
||
| 首先解释一下 AlignByDevicePlan 中一些重要字段的含义： | ||
| - measurements：查询中出现的 measurement 列表。 | ||
| - dataTypeMapping: 该变量继承自基类 QueryPlan，其主要作用是在计算每个设备的执行路径时，提供此次查询的 paths 对应的数据类型。 |
| - dataTypeMapping: 该变量继承自基类 QueryPlan，其主要作用是在计算每个设备的执行路径时，提供此次查询的 paths 对应的数据类型。 | ||
| - deviceToMeasurementsMap, deviceToFilterMap: 这两个字段分别用来存储设备对应的测点和过滤条件。 | ||
| - measurementDataTypeMap：AlignByDevicePlan 要求不同设备的同名 sensor 数据类型一致，该字段是一个 `measurementName -> dataType` 的 Map 结构，用来验证同名 sensor 的数据类型一致性。如 `root.sg.d1.s1` 和 `root.sg.d2.s1` 应该是同一数据类型。 | ||
| - measurementType：记录三种 measurement 类型。在任何设备中都不存在的 measurement 为 `NonExist` 类型；有单引号或双引号的 measurement 为 `Constant` 类型；不属于以上两种的正常 measurement 为 `Exist` 类型。 |
There was a problem hiding this comment.
Suggested change
| - measurementType：记录三种 measurement 类型。在任何设备中都不存在的 measurement 为 `NonExist` 类型；有单引号或双引号的 measurement 为 `Constant` 类型；不属于以上两种的正常 measurement 为 `Exist` 类型。 | |
| - measurementType：记录三种 measurement 类型。在任何设备中都不存在的 measurement 为 `NonExist` 类型；有单引号或双引号的 measurement 为 `Constant` 类型；存在的 measurement 为 `Exist` 类型。 |
SilverNarcissus
approved these changes
Mar 5, 2020
| 2. 新建一个添加了时间戳的 `RowRecord`，向其中加入设备列，先根据 `executeColumns` 与得到的结果建立一个由 `measurementName -> Field` 的 Map 结构 `currentColumnMap`. | ||
| 3. 之后只需要遍历去重后的 `measurements` 列表，判断其类型，如果类型为 `Exist` 则根据 measurementName 从 `currentColumnMap` 中取得其对应的结果，如果没有则设为 `null`；如果是 `NonExist`类型，则直接设为 `null`; 如果是 `Constant` 类型，则将 `measureName` 作为该列的值。 | ||
|
|
||
| 在根据变换后的 `RowRecord` 写入输出数据流后，即可将结果集返回。 |
There was a problem hiding this comment.
再根据变换后的 RowRecord 写入输出数据流后
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Optimize the process about nonExistMeasurement and ConstantMeasurement in AlignByDeviceQuery. Modify the data structure to a map recording the kind of measurement, which trim about 150 lines of code in AlignByDeviceQuery.