parser: allow drop stats of multiple tables #38042

YangKeao · 2022-09-21T07:38:38Z

Signed-off-by: YangKeao yangkeao@chunibyo.icu

What problem does this PR solve?

Issue Number: close #37872

Problem Summary:

DROP STATS doesn't allow dropping stats of multiple tables now.

What is changed and how it works?

This PR modifies the parser and executors to allow dropping multiple tables without partitions.

The grammar should be discussed, as the DROP STATS t1, t2 GLOBAL and DROP STATS t PARTITION xxx is not consistent now. I'm not sure whether it's expected, or I shouldn't allow dropping multiple tables with GLOBAL at the end?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Release note

Allow dropping stats of multiple tables

ti-chi-bot · 2022-09-21T07:38:40Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

mjonss
xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

mjonss · 2022-09-21T08:52:56Z

The grammar should be discussed, as the DROP STATS t1, t2 GLOBAL and DROP STATS t PARTITION xxx is not consistent now. I'm not sure whether it's expected, or I shouldn't allow dropping multiple tables with GLOBAL at the end?

My short answer: Only allow DROP STATS t[,t2[...]]. No need to handle GLOBAL (always table/global level) or PARTITION (since we want to always have table/global stats and all partitions)

I have a patch for always enabling GLOBAL stats for partitioned tables here, which I also see as a part of this discussion :)

Due to the plans of removing static partition prune mode, I would prefer that we also deprecating the support of partition specific configuration of statistics, and not adding support of dropping stats for a specific partition.

Unless there are plans to change the statistic usage for partitioned tables under dynamic prune mode?

I propose that we only support ANALYZE table (global) with non-default configuration (like buckets, samples etc.) and only saves the configuration on table/global level.

My reasoning is that the partition level statistics is aggregated to table/global level and should not differ in configuration which may result in issues when calculating the table/global statistic.

Meaning the following syntax should be supported:
ANALYZE TABLE <list of tables> ... No change.
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> [INDEX <list of indexes>] No additional configuration allowed or used, but still allowed to update the statistics from a subset of partitions.
ALTER TABLE <partitioned table> ANALYZE PARTITION <list of partitions> [INDEX <list of indexes> No additional configuration allowed or used, but still allowed to update the statistics from a subset of partitions.
DROP STATS <list of tables> (no mentioning of partitions allowed or supported)

And this would be deprecated:
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> AllColumnsOrPredicateColumnsOpt AnalyzeOptionListOpt
ALTER TABLE <partitioned table> ANALYZE PARTITION <list of partitions> AllColumnsOrPredicateColumnsOpt AnalyzeOptionListOpt
ANALYZE INCREMENTAL TABLE <partitioned table> PARTITION...
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> COLUMNS...

executor/simple.go

mjonss · 2022-09-21T08:59:57Z

executor/simple.go

@@ -1670,15 +1670,14 @@ func (e *SimpleExec) executeAlterInstance(s *ast.AlterInstanceStmt) error {
 func (e *SimpleExec) executeDropStats(s *ast.DropStatsStmt) (err error) {
 	h := domain.GetDomain(e.ctx).StatsHandle()
 	var statsIDs []int64
-	if s.IsGlobalStats {
-		statsIDs = []int64{s.Table.TableInfo.ID}
+	if s.IsGlobalStats || len(s.PartitionNames) == 0 {


If my proposal that we only support GLOBAL/per table DROP STATS, then I think this should always be done.

YangKeao · 2022-09-22T08:00:58Z

My short answer: Only allow DROP STATS t[,t2[...]]. No need to handle GLOBAL (always table/global level) or PARTITION (since we want to always have table/global stats and all partitions)

@mjonss Good answer! IIUC, DROP STATS ... GLOBAL and DROP STATS ... PARTITION ... will be deprecated? In the latest update of this patch, I add warnings for these two syntax, and don't touch the function of them (before they are useless). In conclusion, this PR becomes the following change:

Deprecate the DROP STATS ... GLOBAL and DROP STATS ... PARTITION ... syntax
Allow dropping stats for multiple tables through DROP STATS a,b,c,d

PTAL

mjonss

LGTM, just a minor suggestion on comment change.

executor/simple.go

xhebox

@YangKeao Rebase please. Also, don't forget pingcap/doc changes.

xhebox · 2022-11-11T10:31:47Z

@YangKeao rebase please, let us merge it.

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>

xhebox · 2022-11-14T05:59:25Z

/merge

ti-chi-bot · 2022-11-14T05:59:28Z

This pull request has been accepted and is ready to merge.

Commit hash: bebee8b

YangKeao · 2022-11-14T06:28:43Z

@YangKeao Rebase please. Also, don't forget pingcap/doc changes.

I have updated the documents in pingcap/docs-cn#11986 and pingcap/docs#11287. Could you please tech-review these documents? @mjonss @xhebox . I only update the syntax (following the DROP TABLE ... style, didn't add more examples about dropping multiple tables). The DROP STATS ... PARTITION ... and DROP STATS ... GLOBAL are not updated there, because they are deprecated (though, have never been on the documents).

sre-bot · 2022-11-14T07:38:17Z

TiDB MergeCI notify

🔴 Bad News! New failing [1] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name	Result	Duration	Compare with Parent commit
idc-jenkins-ci-tidb/mybatis-test	🟥 failed 1, success 0, total 1	11 min	New failing
idc-jenkins-ci-tidb/integration-ddl-test	🔴 failed 1, success 5, total 6	44 min	Existing failure
idc-jenkins-ci/integration-cdc-test	✅ all 39 tests passed	19 min	Fixed
idc-jenkins-ci-tidb/integration-common-test	🟢 all 17 tests passed	13 min	Existing passed
idc-jenkins-ci-tidb/common-test	🟢 all 11 tests passed	10 min	Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2	🟢 all 28 tests passed	6 min 41 sec	Existing passed
idc-jenkins-ci-tidb/tics-test	🟢 all 1 tests passed	6 min 32 sec	Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1	🟢 all 26 tests passed	5 min 34 sec	Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test	🟢 all 1 tests passed	2 min 43 sec	Existing passed
idc-jenkins-ci-tidb/plugin-test	🟢 build success, plugin test success	4min	Existing passed

YangKeao requested review from hawkingrei, mjonss and qw4990 September 21, 2022 07:38

ti-chi-bot added release-note size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 21, 2022

mjonss reviewed Sep 21, 2022

View reviewed changes

YangKeao force-pushed the fix-37872 branch 2 times, most recently from 37e3413 to 356b6f7 Compare September 22, 2022 07:57

YangKeao force-pushed the fix-37872 branch 4 times, most recently from 962ea79 to d313663 Compare September 22, 2022 10:13

mjonss approved these changes Sep 22, 2022

View reviewed changes

executor/simple.go Outdated Show resolved Hide resolved

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 22, 2022

YangKeao force-pushed the fix-37872 branch from d313663 to 4e3c34c Compare September 22, 2022 11:19

YangKeao requested a review from xiongjiwei September 23, 2022 06:16

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 1, 2022

xhebox approved these changes Oct 28, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 28, 2022

allow drop stats of multiple tables

bebee8b

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>

YangKeao force-pushed the fix-37872 branch from 4e3c34c to bebee8b Compare November 14, 2022 05:58

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 14, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 14, 2022

ti-chi-bot added 2 commits November 14, 2022 13:59

Merge branch 'master' into fix-37872

83b2900

Merge branch 'master' into fix-37872

8606693

This was referenced Nov 14, 2022

Synchronise the syntax tree of drop stats pingcap/docs-cn#11986

Merged

Synchronise the syntax tree with drop stats pingcap/docs#11287

Merged

Merge branch 'master' into fix-37872

9879c3b

ti-chi-bot merged commit b0c3381 into pingcap:master Nov 14, 2022

This was referenced Jan 16, 2023

Synchronise the syntax tree of drop stats (#11986) pingcap/docs-cn#12675

Merged

Synchronise the syntax tree with drop stats (#11287) pingcap/docs#12061

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

parser: allow drop stats of multiple tables #38042

parser: allow drop stats of multiple tables #38042

YangKeao commented Sep 21, 2022

ti-chi-bot commented Sep 21, 2022 •

edited

mjonss commented Sep 21, 2022

mjonss Sep 21, 2022

YangKeao commented Sep 22, 2022

mjonss left a comment

xhebox left a comment

xhebox commented Nov 11, 2022

xhebox commented Nov 14, 2022

ti-chi-bot commented Nov 14, 2022

YangKeao commented Nov 14, 2022

sre-bot commented Nov 14, 2022

parser: allow drop stats of multiple tables #38042

parser: allow drop stats of multiple tables #38042

Conversation

YangKeao commented Sep 21, 2022

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Sep 21, 2022 • edited

mjonss commented Sep 21, 2022

mjonss Sep 21, 2022

Choose a reason for hiding this comment

YangKeao commented Sep 22, 2022

mjonss left a comment

Choose a reason for hiding this comment

xhebox left a comment

Choose a reason for hiding this comment

xhebox commented Nov 11, 2022

xhebox commented Nov 14, 2022

ti-chi-bot commented Nov 14, 2022

YangKeao commented Nov 14, 2022

sre-bot commented Nov 14, 2022

TiDB MergeCI notify

ti-chi-bot commented Sep 21, 2022 •

edited