[BEAM-5072][SQL]Disable SortRemoveRule and Remove RelCollationTraitDef configuration in BeamQueryPlanner

[amaliujia]

Right now we configure RelCollationTraitDef in BeamQueryPlanner but seems like we don't use/implement it. However this configuration causes SortRemoveRule to apply sort to SortRel's input and then drop the SorRel (which does not really sort the SortRel's input).

Remove RelCollationTraitDefand disable SortRemoveRule for now.

If don't remove CollationTrait, one example is:

INFO: SQLPlan>
BeamIOSinkRel(table=[[beam, ORDER_ID_TABLE]], operation=[INSERT], flattened=[true])
  LogicalSort(sort0=[$0], dir0=[ASC])
    LogicalProject(order_id=[$0])
      BeamIOSourceRel(table=[[beam, ORDER_DETAILS]])

Aug 03, 2018 1:07:24 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamIOSinkRel(table=[[beam, ORDER_ID_TABLE]], operation=[INSERT], flattened=[true])
  BeamCalcRel(expr#0..3=[{inputs}], order_id=[$t0])
    BeamIOSourceRel(table=[[beam, ORDER_DETAILS]])

---

Follow this checklist to help us incorporate your contribution quickly and easily:

• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---	---	---	---	---
Java
Python		---		---	---	---	---

[amaliujia]

R: @apilloud @akedin

Also created JIRA https://issues.apache.org/jira/browse/BEAM-5073 to track enabling SortRemoveRule.

[amaliujia]

Related calcite code that shows how SortRemoteRule works: https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/SortRemoveRule.java#L47

[apilloud]

What is here so far looks good, you also need to remove it from the JDBC planner here:

beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java

Line 88 in 7ad96d3

public Void apply(RelOptPlanner planner) {

Planner gives you getRelTraitDefs, clearRelTraitDefs, and addRelTraitDef, but nothing to remove a single trait. You'll have to get the list, clear them all, then add back the other ones.

[akedin]

This is only valid for BeamSqlEnv path. Does the JDBC path apply the CollationTrait?

[amaliujia]

I modified JDBC path here: https://github.com/amaliujia/beam/blob/5ab3ed8da33ead0cd3b43da1af8893d761c45a47/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcDriver.java#L100

[amaliujia]

I also removed/commented SortRemoveRule because it does nothing without CollationTrait

[amaliujia]

Green

[akedin]

• @xumingmin @xumingming

[apilloud]

LGTM.