statistics: better benchmark tests for merge topN

[hi-rustin]

Signed-off-by: hi-rustin rustin.liu@gmail.com

What problem does this PR solve?

Issue Number: None

Problem Summary:

What is changed and how it works?

• Added some better benchmark tests for merging topN.
  • Used 500 topN
  • Skip some values for some partitions
• Optimized CheckEmptyTopNs to remove usless count

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[hi-rustin]

Test result:

tidb on  master [$!⇡] via 🐳 orbstack via 🐹 v1.21.3 took 1m26s 
❯ go test -run=^$ -bench=BenchmarkMergePartTopN2GlobalTopNWithHists -benchmem github.com/pingcap/tidb/pkg/statistics/handle/globalstats 
goos: darwin
goarch: arm64
pkg: github.com/pingcap/tidb/pkg/statistics/handle/globalstats
BenchmarkMergePartTopN2GlobalTopNWithHists/Size100-8                 224           6631338 ns/op          177838 B/op         30 allocs/op
BenchmarkMergePartTopN2GlobalTopNWithHists/Size1000-8                 13          89028055 ns/op          177477 B/op         30 allocs/op
BenchmarkMergePartTopN2GlobalTopNWithHists/Size2000-8                  4         274116135 ns/op          176676 B/op         28 allocs/op
BenchmarkMergePartTopN2GlobalTopNWithHists/Size5000-8                  1        1215482458 ns/op          174808 B/op         28 allocs/op
BenchmarkMergePartTopN2GlobalTopNWithHists/Size10000-8                 1        3031611292 ns/op          177688 B/op         30 allocs/op
PASS
ok      github.com/pingcap/tidb/pkg/statistics/handle/globalstats       13.252s

tidb on  master [$!⇡] via 🐳 orbstack via 🐹 v1.21.3 took 27s 
❯ go test -run=^$ -bench=BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists -benchmem github.com/pingcap/tidb/pkg/statistics/handle/globalstats 
goos: darwin
goarch: arm64
pkg: github.com/pingcap/tidb/pkg/statistics/handle/globalstats
BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists/Size100-8                261           6164520 ns/op          932777 B/op        162 allocs/op
BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists/Size1000-8                20          57104685 ns/op          961775 B/op        163 allocs/op
BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists/Size2000-8                 3         392592583 ns/op         2047240 B/op        313 allocs/op
BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists/Size5000-8                 1        3041385250 ns/op         5217016 B/op        723 allocs/op
BenchmarkMergeGlobalStatsTopNByConcurrencyWithHists/Size10000-8                1        21334707667 ns/op       10866232 B/op       1377 allocs/op
PASS
ok      github.com/pingcap/tidb/pkg/statistics/handle/globalstats       33.626s

[codecov]

Codecov Report

Merging #48006 (114610f) into master (a7366f9) will increase coverage by 1.3602%.
Report is 13 commits behind head on master.
The diff coverage is 100.0000%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #48006        +/-   ##
================================================
+ Coverage   71.7365%   73.0967%   +1.3601%     
================================================
  Files          1401       1424        +23     
  Lines        405704     414433      +8729     
================================================
+ Hits         291038     302937     +11899     
+ Misses        94917      92639      -2278     
+ Partials      19749      18857       -892     

Flag	Coverage Δ
integration	42.3941% <100.0000%> (?)
unit	71.9274% <100.0000%> (+0.1909%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	54.0503% <ø> (+0.0589%)	⬆️
parser	∅ <ø> (∅)
br	49.1944% <ø> (-3.8547%)	⬇️

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hawkingrei,qw4990]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-10-26 08:06:20.924790701 +0000 UTC m=+2508378.511900845: ☑️ agreed by hawkingrei.
• 2023-10-27 08:03:47.170364652 +0000 UTC m=+2594624.757474798: ☑️ agreed by qw4990.

[hawkingrei]

/retest

[hi-rustin]

/retest