feat(optimizer, storage): pushdown range-filter to storage

[wangrunji0408]

Range-filter scan has been supported in storage #589. But the query engine doesn't make use of it. This PR adds an optimization rule to push range predicates from filters down to scans. To identify range predicates on primary keys (e.g. id > 1), we introduced the range analysis. It will extract a KeyRange from =, >, >=, <, <=, and node, which can be passed into storage later.

Another goal of this PR is to decouple storage from the query engine v1. Currently it depends on BoundExpr from v1 to filter data inside storage. But in fact we don't expect storage to support general filters other than range filters on keys. So this PR removed general filter from storage and added a KeyRange for range filter. After that, the v1 engine can be removed completely.

A simple test on TPC-H dataset:

# disable filter scan rule
> select count(O_ORDERKEY) from orders where O_ORDERKEY < 10;
+---+
| 7 |
+---+
in 0.018s

# enable filter scan rule
> select count(O_ORDERKEY) from orders where O_ORDERKEY < 10;
+---+
| 7 |
+---+
in 0.002s

[skyzh]

LGTM and thanks for the PR! Seems that SQLLogicTest needs to be updated with make apply_planner_test. Also it seems that we pass Option<KeyRange> to the filter but the default value for that is true? Shall we convert it to None somewhere so that we can avoid unnecessary expression evaluation?

[skyzh]

what if both range from one side are bound? (i.e., a < 3 && a < 5) In this case we should not return a None here if we don't have filter executor above scan-filter?

[wangrunji0408]

Yes it will return a None. I didn't merge the bounds here because I thought it would be a bit complicated. For these cases I expected them to be handled in rewrite rules later. For example:

(and (< ?a ?v1) (< ?a ?v2)) => (< ?a (min ?v1 ?v2))

[skyzh]

should this be if let Some(ref mut min) = min and *min = min.min(xxx)?

[wangrunji0408]

The whole if expression will be assigned to fetch_size. I think it is fine.

[wangrunji0408]

Looks like this feature is required by range-filter scan now. Should we make it mandatory? or will we support range-filter scan without it? @skyzh

[wangrunji0408]

Do we plan to support range-filter scan (store in order by key) in memory storage?

[wangrunji0408]

Also it seems that we pass Option<KeyRange> to the filter but the default value for that is true? Shall we convert it to None somewhere so that we can avoid unnecessary expression evaluation?

I've changed the default value to null so that it's more intuitive. In fact, there is no evaluation for both cases. It only takes the Option<KeyRange> out from the filter node when building executor.