[pixels-amphi] add a query parser based on calcite.

[bianhq]

The query parser (pixels-parser) validates the query using pixels metadata and parses the valid query into a logical plan in the relnode format in calcite.
Then we extract the critical information from the logical plan to be consumed by the coordinator of pixels-amphi.

JDBC/ODBC server is currently not required.

[jingrongchen]

The relnode json format logical plan doesn't include tree information to differentiate between parent or children node

For example the rel-operation "relOp": "LogicalProject" and "relOp": "LogicalFilter", "relOp": "LogicalAggregate" doesn't include the filed "inputs" to specify the parent node.

Is there any ways to add inputs information(parent information) to those operation node without traverse the tree?

[voidforall]

The relnode json format logical plan doesn't include tree information to differentiate between parent or children node

For example the rel-operation "relOp": "LogicalProject" and "relOp": "LogicalFilter", "relOp": "LogicalAggregate" doesn't include the filed "inputs" to specify the parent node.

Is there any ways to add inputs information(parent information) to those operation node without traverse the tree?

The serialization of RelNode does not include parent information because it's designed as DAG. To analyze the logical plan or add information you needed, the best way is to traverse the tree and process it during the traversal, but not to analyze directly on the json.

[bianhq]

The relnode json format logical plan doesn't include tree information to differentiate between parent or children node
For example the rel-operation "relOp": "LogicalProject" and "relOp": "LogicalFilter", "relOp": "LogicalAggregate" doesn't include the filed "inputs" to specify the parent node.
Is there any ways to add inputs information(parent information) to those operation node without traverse the tree?

The serialization of RelNode does not include parent information because it's designed as DAG. To analyze the logical plan or add information you needed, the best way is to traverse the tree and process it during the traversal, but not to analyze directly on the json.

Do you mean that RelNode only has children's information, so we can only traverse the logical plan from root to leaf?

[voidforall]

Yes. Only support top-down traversal. When traversing the plan, RelVistor can access parent node.

[jingrongchen]

The relnode json format logical plan doesn't include tree information to differentiate between parent or children node
For example the rel-operation "relOp": "LogicalProject" and "relOp": "LogicalFilter", "relOp": "LogicalAggregate" doesn't include the filed "inputs" to specify the parent node.
Is there any ways to add inputs information(parent information) to those operation node without traverse the tree?

The serialization of RelNode does not include parent information because it's designed as DAG. To analyze the logical plan or add information you needed, the best way is to traverse the tree and process it during the traversal, but not to analyze directly on the json.

Many thanks for the advice! I have another question about the Relvisitor, when traversing the plan, I can only get the node information by calling node.getDigest(), It's a string format that has the following structure.

LogicalUnion.NONE.[](input#0=LogicalAggregate#12,input#1=LogicalAggregate#15,all=true) LogicalAggregate.NONE.[](input=LogicalProject#11,group={0},num=SUM($1)) LogicalProject.NONE.[](input=LogicalFilter#2,exprs=[$1, $3]) LogicalFilter.NONE.[](input=LogicalTableScan#1,condition==($5, CAST('3-MEDIUM'):CHAR(18) NOT NULL)) LogicalTableScan.NONE.[](table=[tpch, orders]) LogicalAggregate.NONE.[](input=LogicalProject#14,group={0},num=COUNT()) LogicalProject.NONE.[](input=LogicalFilter#7,exprs=[$1]) LogicalFilter.NONE.[](input=LogicalTableScan#6,condition==($5, CAST('3-MEDIUM'):CHAR(18) NOT NULL)) LogicalTableScan.NONE.[](table=[tpch, orders])
I have to extract information from the string. Do you have any suggestions on traversing the node so that I can get information more easily?

[voidforall]

The relnode json format logical plan doesn't include tree information to differentiate between parent or children node
For example the rel-operation "relOp": "LogicalProject" and "relOp": "LogicalFilter", "relOp": "LogicalAggregate" doesn't include the filed "inputs" to specify the parent node.
Is there any ways to add inputs information(parent information) to those operation node without traverse the tree?

The serialization of RelNode does not include parent information because it's designed as DAG. To analyze the logical plan or add information you needed, the best way is to traverse the tree and process it during the traversal, but not to analyze directly on the json.

Many thanks for the advice! I have another question about the Relvisitor, when traversing the plan, I can only get the node information by calling node.getDigest(), It's a string format that has the following structure.

LogicalUnion.NONE.[](input#0=LogicalAggregate#12,input#1=LogicalAggregate#15,all=true) LogicalAggregate.NONE.[](input=LogicalProject#11,group={0},num=SUM($1)) LogicalProject.NONE.[](input=LogicalFilter#2,exprs=[$1, $3]) LogicalFilter.NONE.[](input=LogicalTableScan#1,condition==($5, CAST('3-MEDIUM'):CHAR(18) NOT NULL)) LogicalTableScan.NONE.[](table=[tpch, orders]) LogicalAggregate.NONE.[](input=LogicalProject#14,group={0},num=COUNT()) LogicalProject.NONE.[](input=LogicalFilter#7,exprs=[$1]) LogicalFilter.NONE.[](input=LogicalTableScan#6,condition==($5, CAST('3-MEDIUM'):CHAR(18) NOT NULL)) LogicalTableScan.NONE.[](table=[tpch, orders]) I have to extract information from the string. Do you have any suggestions on traversing the node so that I can get information more easily?

If you want to get the information, for example, the condition of a LogicalFilter operator, you need to first cast it to Filter node like if (node instanceof Filter) { Filter filter = (Filter) node; }. Then you can call the methods of corresponding node type to fetch information, such as: https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L136

[jingrongchen]

I have another problem regarding that calcite doesn't show the join key for some LogicalJoin

For example in TPCH.Q18. here is the part of the Logical plan

.............
LogicalJoin(condition=[=($8, $33)], joinType=[inner])
LogicalJoin(condition=[true], joinType=[inner])
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[tpch, customer]])
LogicalTableScan(table=[[tpch, orders]])
LogicalTableScan(table=[[tpch, lineitem]])
................

when join customer and orders table, and lineitem table. no join key was indicated.
Is there any way for us to show the join condition, by showing condition=[=($8, $33)] instead of condition=[true]?

[voidforall]

I have another problem regarding that calcite doesn't show the join key for some LogicalJoin

For example in TPCH.Q18. here is the part of the Logical plan

............. LogicalJoin(condition=[=($8, $33)], joinType=[inner]) LogicalJoin(condition=[true], joinType=[inner]) LogicalJoin(condition=[true], joinType=[inner]) LogicalTableScan(table=[[tpch, customer]]) LogicalTableScan(table=[[tpch, orders]]) LogicalTableScan(table=[[tpch, lineitem]]) ................

when join customer and orders table, and lineitem table. no join key was indicated. Is there any way for us to show the join condition, by showing condition=[=($8, $33)] instead of condition=[true]?

Because Calcite cannot find explicit information (join keys) in the query. It should be derived from schema. Not sure if it's possible to specify them inside Calcite, I did not find such functionality in AbstractSchema. If not, it has to be complemented with metadata service