planner, util: fix statement summary history and display correctness

[jiong-nba]

What problem does this PR solve?

Issue Number: ref #70161

Problem Summary:

Statement Summary v1/v2 has several independent read and display correctness issues: open-ended history predicates are narrowed to one hour, history readers return stale intervals, execution averages use the commit count, table names can contain trailing separators, and v2 DIGEST_TEXT ignores tidb_stmt_summary_max_sql_length.

What changed and how does it work?

• Preserve open-ended statement-summary time predicates with MinDatetime/MaxDatetime.
• Return the latest configured history intervals in chronological order for regular and evicted v1 history.
• Use EXEC_COUNT as the denominator for execution-based KV/PD/backoff/write-response averages in v1 and v2.
• Filter empty table entries before joining TABLE_NAMES in v1 and v2.
• Format v2 DIGEST_TEXT with the same configured SQL-length behavior as v1.
• Add focused regressions for every fix.

Each fix is kept in its own commit to make review and selective backporting straightforward.

Check List

Tests

• Unit test
  • ./tools/check/failpoint-go-test.sh pkg/util/stmtsummary -run '^(TestHistoryClearAndResizeKeepsLatestIntervals|TestEvictedHistoryCollectionKeepsLatestIntervals|TestExecutionAverageColumnsUseExecCount|TestTableNamesSkipEmptyTables)$' -tags=intest,deadlock -count=1
  • go test ./pkg/util/stmtsummary/v2 -run '^(TestExecutionAverageColumnsUseExecCount|TestStmtRecordTableNamesSkipEmptyTables|TestStmtRecordFormatsDigestText)$' -tags=intest,deadlock -count=1
  • v1 and v2 focused regressions passed with -race.
  • go test ./pkg/planner/core/operator/logicalop/logicalop_test -run '^TestStatementsSummaryExtractorOpenEndedTimeRange$' -tags=intest,deadlock -count=1
  • make lint
  • make bazel_prepare is blocked locally because bazel is not installed.
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Fix incorrect Statement Summary history ranges, interval selection, average timings, table names, and v2 digest text formatting.

Summary by CodeRabbit

Summary by CodeRabbit

• Bug Fixes

  • Improved statement summary handling of open-ended time predicates (start-only/end-only) and preserved sentinel min/max formatting in EXPLAIN output.
  • Corrected average timing metrics to divide by execution count rather than commit count.
  • History clearing/resizing and evicted-history collection now retain the most recent intervals.
  • Fixed statement table-name formatting (skips empty table entries) and applied consistent normalized SQL length formatting.

• Tests

  • Added/updated unit tests covering open-ended time ranges, average timing logic, and history/table-name behaviors.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign winoros for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS
• pkg/planner/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pingcap-cla-assistant]

All committers have signed the CLA.

[coderabbitai]

📝 Walkthrough

Walkthrough

The changes correct statement-summary time-bound extraction, history retention order, execution-based averages, table-name joining, and v2 digest text formatting. Regression tests cover planner predicates, history resizing, metric calculations, table names, and SQL formatting.

Changes

Statement summary correctness

Layer / File(s)	Summary
Open-ended time-range extraction pkg/planner/core/memtable_predicate_extractor.go, pkg/planner/core/operator/logicalop/logicalop_test/...	Missing predicate bounds use minimum or maximum datetime sentinels, with explanation formatting and tests for both open-ended cases.
Latest history retention pkg/util/stmtsummary/evicted.go, pkg/util/stmtsummary/statement_summary.go, pkg/util/stmtsummary/*_test.go	History collection and clearing retain the newest intervals while preserving chronological result order.
V1 summary calculations and names pkg/util/stmtsummary/reader.go, pkg/util/stmtsummary/statement_summary.go, pkg/util/stmtsummary/statement_summary_test.go	Timing averages use execution counts, and table names omit incomplete entries and separators.
V2 summary record formatting pkg/util/stmtsummary/v2/*	V2 timing averages use execution counts, table names skip empty tables, and normalized SQL uses configured formatting limits.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

• pingcap/tidb#70161: Covers the same statement-summary correctness objectives addressed here.

Possibly related PRs

• pingcap/tidb#70150: Contains overlapping statement-summary history cleanup and eviction handling changes.

Suggested reviewers: nolouch

Poem

A rabbit hops through history’s queue,
Keeping the newest rows in view.
Open bounds stretch wide and true,
Averages count executions too.
Clean names and digests shine—
“Hop!” says the bunny, “All look fine!”

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 29.41% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately summarizes the main statement summary history and display fixes across planner and util packages.
Description check	✅ Passed	The description follows the template well, includes the issue reference, problem summary, changes, tests, side effects, documentation, and release note.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 31.11111% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.8568%. Comparing base (2ddad9e) to head (61e4085).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #70159        +/-   ##
================================================
- Coverage   76.3254%   73.8568%   -2.4686%     
================================================
  Files          2041       2060        +19     
  Lines        559516     582137     +22621     
================================================
+ Hits         427053     429948      +2895     
- Misses       131563     151683     +20120     
+ Partials        900        506       -394     

Flag	Coverage Δ
integration	40.8650% <31.1111%> (+1.2071%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	59.8807% <ø> (ø)
parser	∅ <ø> (∅)
br	46.6707% <ø> (-16.0218%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ti-chi-bot]

@jiong-nba: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/mysql-test	61e4085	link	true	/test mysql-test
pull-integration-realcluster-test-next-gen	61e4085	link	true	/test pull-integration-realcluster-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.