Skip to content

Commit e1fa536

Browse files
tiancaiamaoyikekelilin90
authored
reference/sql: add description about when partition pruning works (#2328)
Co-authored-by: yikeke <yikeke@pingcap.com> Co-authored-by: Lilian Lee <lilin@pingcap.com>
1 parent 7def406 commit e1fa536

File tree

1 file changed

+102
-0
lines changed

1 file changed

+102
-0
lines changed

reference/sql/partitioning.md

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -507,6 +507,108 @@ The optimizer can prune partitions through `WHERE` conditions in the following t
507507
* partition_column = constant
508508
* partition_column IN (constant1, constant2, ..., constantN)
509509

510+
### Some cases for partition pruning to take effect
511+
512+
1. Partition pruning uses the query conditions on the partitioned table, so if the query conditions can not be pushed down to the partitioned table according to the planner's optimization rules, partition pruning does not apply for this query.
513+
514+
For example:
515+
516+
{{< copyable "sql" >}}
517+
518+
```sql
519+
create table t1 (x int) partition by range (x) (
520+
partition p0 values less than (5),
521+
partition p1 values less than (10));
522+
create table t2 (x int);
523+
```
524+
525+
{{< copyable "sql" >}}
526+
527+
```sql
528+
explain select * from t1 left join t2 on t1.x = t2.x where t2.x > 5;
529+
```
530+
531+
In this query, the left out join is converted to the inner join, and then `t1.x > 5` is derived from `t1.x = t2.x` and `t2.x > 5`, so it could be used in partition pruning and only the partition `p1` remains.
532+
533+
```sql
534+
explain select * from t1 left join t2 on t1.x = t2.x and t2.x > 5;
535+
```
536+
537+
In this query, `t2.x > 5` can not be pushed down to the `t1` partitioned table, so partition pruning would not take effect for this query.
538+
539+
2. Since partition pruning is done during the plan optimizing phase, it does not apply for those cases that filter conditions are unknown until the execution phase.
540+
541+
For example:
542+
543+
{{< copyable "sql" >}}
544+
545+
```sql
546+
create table t1 (x int) partition by range (x) (
547+
partition p0 values less than (5),
548+
partition p1 values less than (10));
549+
```
550+
551+
{{< copyable "sql" >}}
552+
553+
```sql
554+
explain select * from t2 where x < (select * from t1 where t2.x < t1.x and t2.x < 2);
555+
```
556+
557+
This query reads a row from `t2` and uses the result for the subquery on `t1`. Theoretically, partition pruning could benefit from `t1.x > val` expression in the subquery, but it does not take effect there as that happens in the execution phase.
558+
559+
3. As a result of a limitation from current implementation, if a query condition can not be pushed down to TiKV, it can not be used by the partition pruning.
560+
561+
Take the `fn(col)` expression as an example. If the TiKV coprocessor supports this `fn` function, `fn(col)` may be pushed down to the the leaf node (that is, partitioned table) according to the predicate push-down rule during the plan optimizing phase, and partition pruning can use it.
562+
563+
If the TiKV coprocessor does not support this `fn` function, `fn(col)` would not be pushed down to the leaf node. Instead, it becomes a `Selection` node above the leaf node. The current partition pruning implementation does not support this kind of plan tree.
564+
565+
4. For hash partition, the only query supported by partition pruning is the equal condition.
566+
567+
5. For range partition, for partition pruning to take effect, the partition expression must be in those forms: `col` or `fn(col)`, and the query condition must be one of `>`, `<`, `=`, `>=`, and `<=`. If the partition expression is in the form of `fn(col)`, the `fn` function must be monotonous.
568+
569+
If the `fn` function is monotonous, for any `x` and `y`, if `x > y`, then `fn(x) > fn(y)`. Then this `fn` function can be called strictly monotonous. For any `x` and `y`, if `x > y`, then `fn(x) >= fn(y)`. In this case, `fn` could also be called "monotonous". In theory, all monotonous functions are supported by partition pruning.
570+
571+
Currently, partition pruning in TiDB only support those monotonous functions:
572+
573+
```
574+
unix_timestamp
575+
to_days
576+
```
577+
578+
For example, the partition expression is a simple column:
579+
580+
{{< copyable "sql" >}}
581+
582+
```sql
583+
create table t (id int) partition by range (id) (
584+
partition p0 values less than (5),
585+
partition p1 values less than (10));
586+
select * from t where t > 6;
587+
```
588+
589+
Or the partition expression is in the form of `fn(col)` where `fn` is `to_days`:
590+
591+
{{< copyable "sql" >}}
592+
593+
```sql
594+
create table t (dt datetime) partition by range (to_days(id)) (
595+
partition p0 values less than (to_days('2020-04-01')),
596+
partition p1 values less than (to_days('2020-05-01')));
597+
select * from t where t > '2020-04-18';
598+
```
599+
600+
An exception is `floor(unix_timestamp())` as the partition expression. TiDB does some optimization for that case by case, so it is supported by partition pruning.
601+
602+
{{< copyable "sql" >}}
603+
604+
```sql
605+
create table t (ts timestamp(3) not null default current_timestamp(3))
606+
partition by range (floor(unix_timestamp(ts))) (
607+
partition p0 values less than (unix_timestamp('2020-04-01 00:00:00')),
608+
partition p1 values less than (unix_timestamp('2020-05-01 00:00:00')));
609+
select * from t where t > '2020-04-18 02:00:42.123';
610+
```
611+
510612
## Partition selection
511613

512614
`SELECT` statements support partition selection, which is implemented by using a `PARTITION` option.

0 commit comments

Comments
 (0)