[CALCITE-6650] Optimize the IN sub-query and SOME sub-query by Metada…

[NobiGo]

…ta RowCount

[dssysolyatin]

Are you sure that maxRowCount < 1D guarantees that the subquery will return zero records?

Since maxRowCount is a Double, I think if maxRowCount = 0.7 for example, there is a possibility that it could return more than 0 records.

[julianhyde]

I have the same concern as @dssysolyatin. Could we instead check whether it is an instance of Values and then call Values.isEmpty? (Which is how PruneEmptyRules work.)

[zabetak]

We have rule in PruneEmptyRules that does:

          return maxRowCount != null && maxRowCount == 0.0;

Given that comparisons with doubles are inherently imprecise we may need to revisit that rule as well. Should we employ a threshold comparison?

[zabetak]

In addition, it seems that we employ the maxRowCount < 1D check in other places as well so whatever we decide we may need to propagate the changes everywhere.

[mihaibudiu]

The semantics of this value should be documented. Is it always guaranteed to be less than the actual row count of the table?

[NobiGo]

@zabetak Perhaps we should clarify more clearly that when 'maxRowCount==0D' means no data is returned?

[julianhyde]

Sure. If you can find a better place to put the logic at

calcite/core/src/main/java/org/apache/calcite/rel/rules/SubQueryRemoveRule.java

Line 456 in 3db4dd1

final Double minRowCount = mq.getMinRowCount(e.rel);

then do it. Maybe a method @Nullable Boolean RelMetadataQuery.isEmpty(RelNode) (where true means always empty, false means always non-empty, null for everything else).

I cannot imagine a case where a provider would return a maxRowCount value less than one where the relation was not guaranteed empty, so I wouldn't worry too much whether the condition is < 1D or == 0D.

[NobiGo]

I think add @Nullable Boolean RelMetadataQuery.isEmpty(RelNode) is a good way for users to customize their implementation. The Default behavior will use maxRowCount == 0.0.

[julianhyde]

Note that we don't allow people to override methods in RelMetadataQuery. If we want people to be able to customize we should create a new kind of metadata.

[NobiGo]

I have created an issue CALCITE-6665 to log it.

[mihaibudiu]

This looks fine to me. Unless there are objections in the next few days I plan to merge this.

[mihaibudiu]

Feel free to squash the commits and merge if there are no other suggestions.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
84.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud