Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ranger: refine explain format, again #7041

Merged
merged 8 commits into from Jul 17, 2018

Conversation

@zz-jason
Copy link
Member

commented Jul 12, 2018

What have you changed? (mandatory)

  1. Refine the string format of ranges and constant expressions:

    • use NULL to represent NULL values instead of <nil>
    • wrap string with "
  2. Show whether we use the pseudo statistics for a table or index in the explain result.

  3. Refine the count and range information for inner table of IndexJoin

Before this PR:

TiDB(localhost:4000) > desc select * from t where a = "<nil>";
+-------------------+-------+------+---------------------------------------------------------+
| id                | count | task | operator info                                           |
+-------------------+-------+------+---------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                       |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:[<nil>,<nil>], keep order:false |
+-------------------+-------+------+---------------------------------------------------------+
2 rows in set (0.01 sec)

TiDB(localhost:4000) > desc select * from t where a = "NULL";
+-------------------+-------+------+-------------------------------------------------------+
| id                | count | task | operator info                                         |
+-------------------+-------+------+-------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                     |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:[NULL,NULL], keep order:false |
+-------------------+-------+------+-------------------------------------------------------+
2 rows in set (0.00 sec)

TiDB(localhost:4000) > desc select * from t where a is null;
+-------------------+-------+------+---------------------------------------------------------+
| id                | count | task | operator info                                           |
+-------------------+-------+------+---------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                       |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:[<nil>,<nil>], keep order:false |
+-------------------+-------+------+---------------------------------------------------------+
2 rows in set (0.00 sec)

TiDB(localhost:4000) > desc select /*+ TIDB_INLJ(t1) */ t2.a from t1 join t2 on t1.a=t2.a;
+--------------------------+----------+------+---------------------------------------------------------------------------+
| id                       | count    | task | operator info                                                             |
+--------------------------+----------+------+---------------------------------------------------------------------------+
| Projection_5             | 12500.00 | root | test.t2.a                                                                 |
| └─IndexJoin_9            | 12500.00 | root | inner join, inner:TableReader_8, outer key:test.t1.a, inner key:test.t2.a |
|   ├─TableReader_11       | 10000.00 | root | data:TableScan_10                                                         |
|   │ └─TableScan_10       | 10000.00 | cop  | table:t1, range:[-inf,+inf], keep order:false                             |
|   └─TableReader_8        | 10000.00 | root | data:TableScan_7                                                          |
|     └─TableScan_7        | 10000.00 | cop  | table:t2, range:[-inf,+inf], keep order:false                             |
+--------------------------+----------+------+---------------------------------------------------------------------------+
6 rows in set (0.00 sec)

After this PR:

TiDB(localhost:4000) > desc select * from t where a = "<nil>";
+-------------------+-------+------+---------------------------------------------------------------------------+
| id                | count | task | operator info                                                             |
+-------------------+-------+------+---------------------------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                                         |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:["<nil>","<nil>"], keep order:false, stats:pseudo |
+-------------------+-------+------+---------------------------------------------------------------------------+
2 rows in set (0.00 sec)

TiDB(localhost:4000) > desc select * from t where a = "NULL";
+-------------------+-------+------+-------------------------------------------------------------------------+
| id                | count | task | operator info                                                           |
+-------------------+-------+------+-------------------------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                                       |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:["NULL","NULL"], keep order:false, stats:pseudo |
+-------------------+-------+------+-------------------------------------------------------------------------+
2 rows in set (0.00 sec)

TiDB(localhost:4000) > desc select * from t where a is null;
+-------------------+-------+------+---------------------------------------------------------------------+
| id                | count | task | operator info                                                       |
+-------------------+-------+------+---------------------------------------------------------------------+
| IndexReader_9     | 10.00 | root | index:IndexScan_8                                                   |
| └─IndexScan_8     | 10.00 | cop  | table:t, index:a, range:[NULL,NULL], keep order:false, stats:pseudo |
+-------------------+-------+------+---------------------------------------------------------------------+
2 rows in set (0.00 sec)

TiDB(localhost:4000) > desc select /*+ TIDB_INLJ(t1) */ t2.a from t1 join t2 on t1.a=t2.a;
+--------------------------+----------+------+---------------------------------------------------------------------------+
| id                       | count    | task | operator info                                                             |
+--------------------------+----------+------+---------------------------------------------------------------------------+
| Projection_5             | 12500.00 | root | test.t2.a                                                                 |
| └─IndexJoin_9            | 12500.00 | root | inner join, inner:TableReader_8, outer key:test.t1.a, inner key:test.t2.a |
|   ├─TableReader_11       | 10000.00 | root | data:TableScan_10                                                         |
|   │ └─TableScan_10       | 10000.00 | cop  | table:t1, range:[-inf,+inf], keep order:false, stats:pseudo               |
|   └─TableReader_8        | 10.00    | root | data:TableScan_7                                                          |
|     └─TableScan_7        | 10.00    | cop  | table:t2, range: decided by [test.t1.a], keep order:false, stats:pseudo   |
+--------------------------+----------+------+---------------------------------------------------------------------------+
6 rows in set (0.01 sec)

What is the type of the changes? (mandatory)

  • Improvement (non-breaking change which is an improvement to an existing feature)

How has this PR been tested? (mandatory)

  • unit test
  • explain test

Does this PR affect documentation (docs/docs-cn) update? (mandatory)

Does this PR affect tidb-ansible update? (mandatory)

Does this PR need to be added to the release notes? (mandatory)

Refer to a related PR or issue link (optional)

fix #6935
fix #7038

Benchmark result if necessary (optional)

Add a few positive/negative examples (optional)

@zz-jason

This comment has been minimized.

Copy link
Member Author

commented Jul 12, 2018

@winoros PTAL

@shenli

This comment has been minimized.

Copy link
Member

commented Jul 12, 2018

Please fix the CI.

@@ -130,6 +130,9 @@ func formatDatum(d types.Datum, isLeftSide bool) string {
if d.GetUint64() == math.MaxUint64 && !isLeftSide {
return "+inf"
}
case types.KindString, types.KindBytes, types.KindMysqlEnum, types.KindMysqlSet,
types.KindMysqlJSON, types.KindBinaryLiteral, types.KindMysqlBit:
return fmt.Sprintf("\"%v\"", d.GetValue())
}
return fmt.Sprintf("%v", d.GetValue())

This comment has been minimized.

Copy link
@lamxTyler

lamxTyler Jul 13, 2018

Member

Why not just wrap it here?

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 13, 2018

Author Member

it's leaved for other types, for example decimal/double/datetime/timestamp, I think they'd better not be warped with "

zz-jason added 2 commits Jul 16, 2018
@zz-jason zz-jason changed the title ranger: refine the string format of ranges ranger: refine explain format, again Jul 16, 2018
@zz-jason

This comment has been minimized.

Copy link
Member Author

commented Jul 16, 2018

@zz-jason

This comment has been minimized.

Copy link
Member Author

commented Jul 16, 2018

/run-all-tests

},
{
indexPos: 0,
exprStr: `a LIKE "\\\\a%"`,
accessConds: `[like(test.t.a, \\a%, 92)]`,
filterConds: "[]",
resultStr: `[[\a <nil>,\b <nil>)]`,
resultStr: "[[\"\\a\" NULL,\"\\b\" NULL)]",

This comment has been minimized.

Copy link
@winoros

winoros Jul 16, 2018

Member

Why change the quotation?

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 16, 2018

Author Member

this is due to the modification of formatDatum in util/ranger/types.go

This comment has been minimized.

Copy link
@winoros

winoros Jul 16, 2018

Member

i mean use `, we don't need escape characters

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 16, 2018

Author Member

Oh, the result was directly copied from the test output😂

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 17, 2018

Author Member

The usage of ` and " is not uniformed now, there are many tests use " actually.

@shenli

This comment has been minimized.

Copy link
Member

commented Jul 16, 2018

@lamxTyler PTAL

Copy link
Member

left a comment

LGTM

zz-jason added 2 commits Jul 17, 2018
@zz-jason

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

@winoros PTAL

if ok && !ds.statisticTable.Pseudo {
rowCount = idxHist.AvgCountPerValue(ds.statisticTable.Count)
} else {
rowCount = ds.statisticTable.PseudoAvgCountPerCount()

This comment has been minimized.

Copy link
@coocood

coocood Jul 17, 2018

Member

The method name should be PseudoAvgCountPerValue?

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 17, 2018

Author Member

yes, already changed.

@zz-jason

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

@coocood PTAL

@coocood

This comment has been minimized.

Copy link
Member

commented Jul 17, 2018

LGTM

@zz-jason zz-jason added status/LGT2 and removed status/LGT1 labels Jul 17, 2018
@zz-jason zz-jason merged commit 1bf3f3d into pingcap:master Jul 17, 2018
4 checks passed
4 checks passed
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@zz-jason zz-jason deleted the zz-jason:dev/explain-range branch Jul 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.