Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

statistics: use histogram count rather than realtime count to calculate stats healthy #41031

Merged
merged 4 commits into from Feb 3, 2023
Merged

statistics: use histogram count rather than realtime count to calculate stats healthy #41031

merged 4 commits into from Feb 3, 2023

Conversation

xuyifangreeneyes
Copy link
Contributor

@xuyifangreeneyes xuyifangreeneyes commented Feb 3, 2023 •
edited

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

When deciding whether to trigger auto analyze or whether stats is outdated, we use modify_count / histogram_count. However, when calculating stats healthy, we use modify_count / realtime_count. The PR changes the stats healthy formula from (1 - modify_count / realtime_count) * 100 to (1 - modify_count / histogram_count) * 100. In this way, stats healthy is aligned with the threshold of auto analyze and outdated stats.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 3, 2023 •
edited

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • qw4990
  • time-and-fate

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note-none size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 3, 2023
@xuyifangreeneyes xuyifangreeneyes changed the title statistics: use histogram count rather than real time count to calculate stats healthy statistics: use histogram count rather than realtime count to calculate stats healthy Feb 3, 2023
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 3, 2023
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 3, 2023
@qw4990
Copy link
Contributor

qw4990 commented Feb 3, 2023

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 30e1bdd

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 3, 2023
@ti-chi-bot
Copy link
Member

@xuyifangreeneyes: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit 98c22c1 into pingcap:master Feb 3, 2023
@xuyifangreeneyes xuyifangreeneyes deleted the align-stats-healthy branch February 3, 2023 08:22
ghazalfamilyusa pushed a commit to ghazalfamilyusa/tidb that referenced this pull request Feb 6, 2023
@xuyifangreeneyes @ghazalfamilyusa
statistics: use histogram count rather than realtime count to calcula…
3bb8f48 
…te stats healthy (pingcap#41031)
lcwangchao added a commit to lcwangchao/tidb that referenced this pull request Feb 13, 2023
@CbcWestwolf @hawkingrei
@tiancaiamao @fzzf678 @qw4990 @crazycs520 @mjonss @zeminzhou @Yisaer @BornChanger @Dousir9 @v01dstar @Leavrth @lance6716 @YangKeao @tangenta @xiongjiwei @Defined2014 @you06 @djshow832 @HuSharp @hongyunyan @nolouch @joccau @sleepymole @xuyifangreeneyes @lcwangchao @zimulala @lichunzhu @dsdashun @TonsnakeLin @wjhuang2016 @dveeden @MoCuishle28 @guo-shaoge @iosmanthus @winkyao @windtalker @glorv @zhongzc @Reminiscent @JmPotato @solotzg @ystaticy @wshwsh12 @fengou1 @hihihuhu @ghazalfamilyusa @disksing @YuJuncen @okJiang @CabinfeverB @LykxSassinator @MyonKeminta @mornyx @ywqzzy @ti-chi-bot @time-and-fate @AilinKid @Benjamin2037 @alrs @cfzjywxk @dependabot
rebase (#23)
1e0631d 
* statistics: fix data race in the Handle.IsTableLocked (pingcap#40572)

close pingcap#40567

* executor,store/copr: trace the buildCopTask and limitExec close duration when it's slow (pingcap#40579)

ref pingcap#40441

* planner: prepared plan cache support cached plan with placeholder in limit clause (pingcap#40196)

ref pingcap#40219

* planner: refactor to put all plan-cacheability-check functions together (pingcap#40625)

ref pingcap#36598

* testkit: reset the resource manager after test finishes (pingcap#40638)

* planner: disable plan-cache for plans with IndexMerge accessing Multi-Valued Index (pingcap#40646)

ref pingcap#40191

* test: add more foreign key test (pingcap#40650)

close pingcap#40649

* ddl: Block change column on partitioned table if data needs change. (pingcap#40631)

close pingcap#40620

* planner: show un-cacheable reasons for execute statements (pingcap#40651)

ref pingcap#36598

* ddl: cancel `create view` job , when failed to drop old view (pingcap#40353)

close pingcap#40352

* domain: use enhanced waitGroupWrapper in domain pingcap#40606 (pingcap#40626)

ref pingcap#40330

* privilege,planner/core: add dynamic privilege RESOURCE_GROUP_ADMIN for resource group operation (pingcap#40632)

ref pingcap#38825

* *: add a new I_S table and support show create for resource groups (pingcap#40665)

close pingcap#39781

* meta: add lock at autoid.(*allocator).Base() (pingcap#40588)

close pingcap#40584

* *: Support learner replica read (pingcap#39979)

close pingcap#40449

* Revert "planner: show un-cacheable reasons for execute statements" (pingcap#40680)

ref pingcap#36598

* executor: fix buildTableReaderForIndexJoin on partition table (pingcap#40674)

close pingcap#40596

* util: remove empty folder (pingcap#40681)

* domain: let background process unify wg.Run (pingcap#40640)

* BR: add lock with ttl for checkpoint backup (pingcap#40563)

close pingcap#40561

* *: LOAD DATA support load one file from s3 and other OSS (pingcap#40489)

ref pingcap#40499

* ttl: execute scan tasks (pingcap#40564)

close pingcap#40364

* ddl: fix the batch check for unique index (pingcap#40672)

ref pingcap#40464, close pingcap#40592

* resourcemanager: avoid to get nil after task finished when to stop task (pingcap#40699)

close pingcap#40700

* planner: refactor plan cache value code (pingcap#40656)

ref pingcap#40219

* telemetry: resource control telemetry (pingcap#40687)

ref pingcap#38825

* planner: set binary collation for mv column (pingcap#40644)

close pingcap#40558

* ddl: add some tests for flashback cluster with ddl history (pingcap#40695)

ref pingcap#40026

* pitr: move step iterate backup files into worker pool (pingcap#40577)

close pingcap#40576

* ddl, lightning: support detecting duplicate keys in different batch (pingcap#40701)

close pingcap#40698

* json, expression: add json unary not implementation (pingcap#40723)

close pingcap#40683

* ddl: prevent dropping in-use resource group (pingcap#40716)

ref pingcap#38825

* store/copr: move row hint into key range (pingcap#40105)

ref pingcap#39361

* server, privileges: make tidb_auth_session_token compatible with password expiration and resource group (pingcap#40735)

ref pingcap#40614

* mockstore: update pd/client and client-go to repair kvproto compatibility (pingcap#40597)

close pingcap#40598

* Explain : Add new item in TiFlashScanContext to record region num information (pingcap#40666)

ref pingcap#39273

* *: Setup Global Resource Controller (pingcap#40732)

close pingcap#40731

* tests: enable race and flaky for flashback test (pingcap#40752)

* br: split regions before pitr restore (pingcap#39941)

close pingcap#37207

* log-backup: do not resolve locks frequently when log-backup task did not exist (pingcap#40760)

ref pingcap#40759

* lint: update config of deprecated linter (pingcap#40763)

ref pingcap#40786

* lightning: check hasDupe and tableID when resolve duplicate rows (pingcap#40696)

close pingcap#40657

* planner: skip the plan cache if non-int values are converted into int when optimization (pingcap#40686)

close pingcap#40679

* *: Remove unused binary that was accidentally added (pingcap#40782)

close pingcap#40781

* linter: add skywalking-eyes into bazel (pingcap#40788)

close pingcap#40789

* planner: remove pickFromBucket field in LRUPlanCache (pingcap#40668)

ref pingcap#36598

* statistics: move statsReader out of statistics/handle (pingcap#40790)

ref pingcap#40624, ref pingcap#40791

* linter: find inappropriate key in call to context.WithValue (pingcap#40769)

close pingcap#40796

* ttl: notify tidb nodes through etcd notification (pingcap#40705)

close pingcap#40365

* executor: add foreign key and session variable test (pingcap#40785)

close pingcap#40784

* ttl: add table `mysql.ttl_job_history` to store ttl job histories (pingcap#40655)

close pingcap#40654

* planner: forbid the optimizer to generate point-plan upon MVIndexes (pingcap#40804)

ref pingcap#40191

* planner: refactor some code related to IndexMerge and MVIndex (pingcap#40800)

ref pingcap#40191

* ddl: remove useless code (pingcap#40756)

close pingcap#40755

* ttl: use a standalone manager in TriggerScanTask test (pingcap#40816)

close pingcap#40811

* ddl: split some tests into new package (pingcap#40758)

ref pingcap#40802

* resourcemanager: scheduler subtask in the pool's task (pingcap#40670)

close pingcap#40719

* bazel: upgrade rules_go to prepare for Go 1.20 (pingcap#40776)

close pingcap#40798

* store/copr: add a param "limit" to region cache's `SplitRegionRanges` (pingcap#40411)

close pingcap#38436

* br: fix checkpoint panic when BR executes gain (pingcap#40706)

close pingcap#40704

* sessionctx: add validation for tidb_enable_plan_replayer_continues_capture (pingcap#40787)

* planner: show warnings when analyzing multi-valued indexes (pingcap#40827)

ref pingcap#40191

* ddl: move update version function to a proper place (pingcap#40817)

ref pingcap#40217

* lightning: support timeout for tikv/tidb retry (pingcap#40805)

ref pingcap#33714

* ttl: support telemetry for TTL (pingcap#40806)

close pingcap#40519

* ttl: auto delete old rows in `mysql.tidb_ttl_job_history` (pingcap#40808)

close pingcap#40807

* lightning: support data files with bom header (pingcap#40813)

close pingcap#40744

* planner: support `use / force index` for multi-valued indexes (pingcap#40820)

ref pingcap#40191

* lightning/parser: support STARTING BY (pingcap#40821)

ref pingcap#40499

* planner: add a switch control if prepare stmt with param limit can be cached (pingcap#40712)

ref pingcap#40219

* metrics: add goroutine schedule latency to grafana panel (pingcap#40836)

close pingcap#40835

* server: support getting the MVCC of the temporary index (pingcap#40830)

close pingcap#40826

* planner: let optimizer trace support gcSubstitute (pingcap#40815)

close pingcap#40662

* planner: add more test cases for MVIndex (pingcap#40853)

ref pingcap#40191

* planner: supply test cases for binding from history (pingcap#40864)

ref pingcap#39199

* ddl: unsupport flashback cluster cross `alter table attributes` (pingcap#40862)

close pingcap#40861

* bindInfo: fix datarace on bindings (pingcap#40846)

close pingcap#40843

* executor: add historical stats test and make it enabled (pingcap#40664)

* Variables: enhance tidb_enable_gc_tuner (pingcap#40850)

close pingcap#40849

* ttl: add metrics for task manager (pingcap#40819)

close pingcap#40810

* lightning: enable setting conflict max-error (pingcap#40874)

ref pingcap#40743

* DDL: Skip collecting TiFlash status when TiFlash is down (pingcap#40872)

close pingcap#38484

* server: Return error when no password is specified for caching_sha2_password account with password (pingcap#40858)

close pingcap#40831

* br: fix br debug encode panic (pingcap#40880)

close pingcap#40878

* planner: fix HashAgg cannot pushdown to tiflash_compute (pingcap#40828)

close pingcap#40717

* parser: Fix for field name output in multi statements (pingcap#40736)

close pingcap#39662

* expression, cmd: fix ColumnSubstitute and allow some cases to substitute (pingcap#38826)

close pingcap#38736, close pingcap#40536

* planner: forbid using IndexFullScan on multi-valued indexes (pingcap#40907)

ref pingcap#40191

* lightning: local backend support keyspace (pingcap#40628)

ref pingcap#40531

* README: refine the Forum infomation format in community section (pingcap#40924)

* ttl: don't fetch ttl scan task after it finished (pingcap#40919)

close pingcap#40918

* executor: Fix tidb crash on index merge reader (pingcap#40904)

close pingcap#40877

* util: add InTest to check whether to be in test (pingcap#40927)

close pingcap#40929

* *: upgrade mgechev/revive (pingcap#40920)

* ddl: fix create exists resource group (pingcap#40837)

ref pingcap#38825

* handle: support fallback for dump historical stats (pingcap#40889)

close pingcap#40890

* admin: support cleanup multi-valued index (pingcap#40428)

close pingcap#40427

* *: support MockAutoScaler and AWSAutoScaler (pingcap#40729)

close pingcap#40747

* session: fix bad log (pingcap#40948)

close pingcap#40947

* planner: fix dropping non-ScalarFunction when extractCol for memtable (pingcap#40935)

close pingcap#40937

* planner: supply test cases for param limit in plan cache (pingcap#40868)

ref pingcap#40219

* br: disable foreign key check when br create table (pingcap#40951)

close pingcap#40952

* planner: rename keep_order hint to order_index hint for mysql compatibility (pingcap#40945)

* planner: report the warning when use the hint but the SQL has the binding (pingcap#40949)

close pingcap#40910

* *: add ci image (pingcap#40866)

close pingcap#40867

* metrics: introduce the resource control RU Grafana dashboard (pingcap#40922)

ref pingcap#38825

* planner: add MppVersion for mpp task; support data compression in Exchange Operator; (pingcap#40132)

ref pingcap/tiflash#6620, close pingcap#40494

* *: unify wru,rru to ru (pingcap#40884)

close pingcap#40883

* br: adaption for keyspace feature (pingcap#40532)

ref pingcap#40531

* resourcemange: fix delete more workers than expected (pingcap#40894)

close pingcap#40893

* keyspace: gc delete range (pingcap#40639)

ref pingcap#40848

* ddl: add more tests about multi-valued index (pingcap#40973)

* test: stabilize TestAddIndexMergeConflictWithPessimistic (pingcap#40975)

close pingcap#40939

* *: add UseAutoScaler config to disable AutoScaler (pingcap#40966)

close pingcap#40971

* *: add memory tracker for mppIterator (pingcap#40901)

close pingcap#40900

* resource_group: supports burstable for resource group (pingcap#40925)

close pingcap#40380

* util: fix memory leak in chunk allocator (pingcap#40989)

close pingcap#40987

* ddl, statistics: fix stats meta missing when creating many tables at once (pingcap#38301)

close pingcap#36004, close pingcap#38189

* statistics: move TableStatsFromStorage out of handle (pingcap#40822)

ref pingcap#40624, ref pingcap#40791

* br: remove totalFlashbackRegions since task completion region already contains all regions (pingcap#40504)

close pingcap#40391, ref pingcap#41005

* lightning: fix panic when nextKey twice (pingcap#40959)

close pingcap#40934

* ttl: fix data race in TTL manager and test (pingcap#40976)

close pingcap#40944

* session: use `mDDLTableVersion` key to control backfill tables (pingcap#40984)

close pingcap#40980

* planner: not allow the optimizer to use `json_contains(j, '[]')` as access conditions on MVIndex (pingcap#41013)

ref pingcap#40191

* ddl/ingest: add mutex to disk root (pingcap#41029)

close pingcap#40970

* ttl: avoid endless loop in TTL task schedule (pingcap#41020)

close pingcap#41019

* resourcemanager: fix TaskController.Stop() can't make producer exit in spmcpool (pingcap#41016)

close pingcap#41015

* *: add buildtag for ddltest (pingcap#41027)

* statistics: use histogram count rather than realtime count to calculate stats healthy (pingcap#41031)

* pd_client: update client and global config (pingcap#40955)

ref pingcap#38825

* ddl: show more jobs in the tidb_mdl_view (pingcap#40860)

close pingcap#40838

* *: add resource group name into information_schema.processlist (pingcap#40739)

close pingcap#40724

* *: upgrade golangci-linter (pingcap#41046)

* executor: fix IndexMerge handle panic logic (pingcap#41036)

close pingcap#41047

* ttl: fix TTL will delete unexpected rows when timezone changed (pingcap#41044)

close pingcap#41043

* *: fix the sysvar value may be corrupted after set by subquery (pingcap#41003)

close pingcap#40979

* ddl, parser: implement the normal flow of distributed processing in DDL reorg phase for adding index (pingcap#40542)

close pingcap#37125

* session, metrics: refine the resource control metrics (pingcap#41045)

ref pingcap#38825

* meta: fix remove resource group (pingcap#40968)

ref pingcap#38825

* metrics/grafana: add cpu/memory of pd in overview panel (pingcap#41072)

close pingcap#41071

* lightning: precheck should check CDC changefeeed rather than capture (pingcap#41076)

close pingcap#41040

* ttl: fix ttl job manager will panic if the status cache doesn't contain table (pingcap#41069)

close pingcap#41067, close pingcap#41068

* sessionctx: fix hidden and user var problems in encoding session states (pingcap#40754)

close pingcap#40614

* Revert "planner: skip the plan cache if non-int values are converted into int when optimization" (pingcap#41077)

ref pingcap#41032

* schema cache: cache schema version by timestamp (pingcap#40768)

close pingcap#40740

* executor: fix plan replayer dump for tpcds q97 (pingcap#41088)

close pingcap#41064

* *: update util/tracing and remove repeated code (pingcap#40825)

close pingcap#40809

* telemetry: Log error reason for failure to get SQL Metric (pingcap#40778)

close pingcap#40777

* br: add retry for prepare flashback for backup cluster is empty and there are only one region (pingcap#41059)

close pingcap#41058

* server: Implement column defaults for `mysql.ComFieldList` (pingcap#40765)

close pingcap#40764

* resourcemanager: avoid to schedule when no task is in the pool (pingcap#41095)

close pingcap#41094

* *: Update github.com/cloudfoundry/gosigar (pingcap#41111)

close pingcap#41110

* ddl: remove useless error log (pingcap#41107)

close pingcap#41106

* executor: use const varialbe for index merge handleWorkerPanic (pingcap#41066)

close pingcap#41065

* Optimizer: add test cases for predicate simplifications (pingcap#41117)

ref pingcap#40221

* executor: disable paging for small limit (pingcap#41120)

close pingcap#40741

* flashback: retry `getStoreGlobalMinSafeTS` during execute flashback (pingcap#41100)

close pingcap#41086

* metrics: add tidb_rm_pool_concurrency (pingcap#41113)

close pingcap#41114

* handler: fix duplicated log of unanalyzed table for historical stats (pingcap#41129)

* *: update client-go and pd client (pingcap#41124)

close pingcap#41123

* admin: fix recover expression cause index broken (pingcap#41092)

ref pingcap#40430, close pingcap#41087

* statistics/handle: remove unnecessary h.mu.Lock (pingcap#41153)

* parser: move LOAD DATA REMOTE into LOAD DATA (pingcap#41091)

ref pingcap#40499

* planner: support more pattern for index join inner side (pingcap#40999)

ref pingcap#40505

* planner: support more pattern for index join inner side (pingcap#40999)

ref pingcap#40505

* planner: support more pattern for index join inner side (pingcap#40999)

ref pingcap#40505

* makefile: update dumpling build options (pingcap#41165)

* statistics/handle: refine the condition of dumping stats delta (pingcap#41133)

ref pingcap#36004, ref pingcap#38189, ref pingcap#39785

* *: use the latest independent resource group manager client (pingcap#41162)

ref pingcap#38825

* store/copr: set upper limit for extra concurrency (pingcap#41135)

close pingcap#41134

* planner: label plans as over-optimized for plan cache after refining cmp-function arguments (pingcap#41136)

close pingcap#40679, close pingcap#41032

* planner: when pushing `topN` down below `proj`, check whether `topN.ByItems` contains a column(with ID=0) generated by `proj` (pingcap#40593)

close pingcap#40535

* *: update bazel config (pingcap#41127)

close pingcap#41160

* br: fix br ci (pingcap#41149)

* *: fix a timezone data race which may cause wrong row data (pingcap#41146)

close pingcap#40710

* ddl: disable raw mode resource group (pingcap#41050)

close pingcap#41049

* flashback: make `br backup full` works during `flashback cluster` (pingcap#40950)

close pingcap#40404

* placement: supports survival preferences (pingcap#40613)

close pingcap#38605

* log-backup: added more robust error handling for log backup advancer (pingcap#41083)

close pingcap#41082

* lightning: access aws resource by session token (pingcap#40753)

close pingcap#40750

* lightning: introduce param to skip CSV header parsing (pingcap#41128)

close pingcap#40839

* ddl: forbid tiflash while enabling API v2 (pingcap#41167)

ref pingcap#41166

* store/copr: enable store batch by default & add extra copr concurrency and batch nums in stats (pingcap#40711)

ref pingcap#39361, close pingcap#40399

* *: remove ru token in resource group schema and set brust limit (pingcap#41174)

ref pingcap#38825

* ReplicaReadMode: introduce new replica_read mode `PreferLeader`. (pingcap#40906)

close pingcap#40905

* txn: Fix the issue that analyze panics when aggressive locking is enabled globally (pingcap#41196)

close pingcap#41194

* pitr: prevent from restore point to cluster running log backup (pingcap#40871)

ref pingcap#40797

* lightning: do resolve conflict job when other lightning has local dupes (pingcap#41157)

close pingcap#40923

* executor: track the memroy usage in HashJoin probe phase (pingcap#41081)

close pingcap#40500

* *: support recover multi-valued index (pingcap#41181)

close pingcap#41180

* stmtsummary: add persistence implementation (pingcap#40814)

ref pingcap#40812

* br: Support backup replica read (pingcap#40899)

ref pingcap#40898

* store/copr: add log for buildBatchCopTasksConsistentHash (pingcap#41101)

close pingcap#41102

* lightning: support compression when sending kv pairs to tikv (pingcap#41164)

close pingcap#41163

* planner: skip plan cache if the plan contains Shuffle operators (pingcap#41185)

close pingcap#38335

* expression: implement least/greatest for string function pushdown (pingcap#41188)

close pingcap#41189

* go.mod: update client-go to the latest version (pingcap#41212)

ref pingcap#38825

* add metrics for TTL (pingcap#41155)

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: 王超 <cclcwangchao@hotmail.com>

* metrics: Add metrics and telemetry for aggressive locking (pingcap#41038)

close pingcap#41037

* planner: adjust estimated row count when pushing `Limit` and keep order for partitioned table (pingcap#41103)

close pingcap#40986

* planner: fix can't find proper physical plan caused by virtual column (pingcap#41132)

close pingcap#41014

* planner: fix space requirement when binding from history (pingcap#41140)

close pingcap#41126

* ddl: initialize the mock session in advanced (pingcap#41227)

close pingcap#40879

* ddl: fix data race on job.SetWarnings (pingcap#41231)

close pingcap#40751

* *: remove limit (pingcap#41242)

* *: remove limit

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* *: remove limit

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* *: remove limit

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

---------

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* br: skip template system table __TiDB_BR_Temporary_mysql when backup (pingcap#41000)

close pingcap#40797

* ddl, model: support for dist-reorg on partitioned tables (pingcap#41145)

close pingcap#41144

* session: fix processInfo time when using driver to prepare and execute query (pingcap#41151)

close pingcap#41057

* dumpling: fix cannot dump data bug when dumpling fails to check has tikv (pingcap#40977)

close pingcap#40932

* lightning: fix panic when user cancel (pingcap#41236)

close pingcap#41235

* util/gpool/spmc: add a sleep to a simple `for` (pingcap#41240)

close pingcap#41205

* planner: fix calculating TiFlash stream count (pingcap#41221)

ref pingcap#40123

* ddl: check the key existence on original index (pingcap#40749)

close pingcap#40730

* types: use `mathutil.Max/mathutil.Min` instead of `myMax/myMin` (pingcap#41192)

* bazel: set --test_keep_going false (pingcap#41247)

* ddl: Add telementry for distributed reorg tasks. (pingcap#41201)

ref pingcap#41266

* ddl: Improve ErrTooLongKey message (pingcap#41270)

Prefix indexes are specified in characters while this error message
returns a limit in bytes. By adding the byte length in the message this
is makes it easier to understand.

This would deviate a little bit from the message in MySQL.

Before:
```
sql> CREATE TABLE t1(id int primary key, c1 text, key(c1(1000)));
ERROR: 1071 (42000): Specified key was too long; max key length is 3072 bytes
```

After:
```
sql> CREATE TABLE t1(id int primary key, c1 text, key(c1(1000)));
ERROR: 1071 (42000): Specified key was too long (4000 bytes); max key length is 3072 bytes
```

* ddl: fix owner get panic info  with dist-reorg (pingcap#41217)

close pingcap#41208

* planner: recover table/schema only needs create and drop privilege (pingcap#41226)

close pingcap#40633

* Lightning: use **** to represent secret key in unit test (pingcap#41269)

close pingcap#41268

* ddl: improve backfill more general (pingcap#41093)

close pingcap#41002

* executor: fix unreasonable CPU time in HashJoin probe phash (pingcap#41265)

close pingcap#41263

* ttl: optimize bazel for TTL codes (pingcap#41278)

close pingcap#41276

* executor: support plan replayer capture remove task (pingcap#41258)

close pingcap#41262

* expression: return upper bound for enum (pingcap#41021)

close pingcap#40855

* lightning: revert db timeout changes for analyze (pingcap#41012)

close pingcap#40941

* *: REORGANIZE PARTITION (pingcap#38535) (pingcap#41096)

close pingcap#38535

* br: fix backup ci (pingcap#41295)

ref pingcap#40404, ref pingcap#40898

* *: reduce the allocation of error constructing in `DecodeTableID` (pingcap#41301)

close pingcap#41300

* util: fix dropped error (pingcap#41254)

* metric: seperate the internal and external transaction metrics (pingcap#41206)

ref pingcap#41203

* planner: fix a bug when pushing streamAgg down (pingcap#41056)

close pingcap#40857

* build(deps): bump golang.org/x/oauth2 from 0.3.0 to 0.5.0 (pingcap#41283)

* Optimizer: derive TopN from filter on row number (pingcap#41209)

ref pingcap#39792

* domain: fix data race in the ttlJobManager (pingcap#41289)

close pingcap#41288

* stmtsummary: fix data race in test (pingcap#41287)

close pingcap#41275

* ddl: return panic when to get tiflash sync progress failed in test (pingcap#41292)

close pingcap#39949

* infoschema: fix is.partitions.partition_expression for range columns (pingcap#41285)

close pingcap#40931

* sysvar: introduce variable tidb_enable_inl_join_inner_multi_pattern (pingcap#41319)

ref pingcap#40505

* executore: fix logic error in HashJoin probe (pingcap#41320)

* *: update bazel config (pingcap#41296)

* infoschema: fix unstable test TestSelectClusterTable (pingcap#41321)

close pingcap#41307

* log-backup: added cache for sst_importer client (pingcap#41305)

close pingcap#41328

* stmtsummary: fix panic in stmts test (pingcap#41334)

close pingcap#41315

* *: fix goleak in the test (pingcap#41339)

close pingcap#41274

* planner: fix tiflash cannot find generated column (pingcap#41261)

close pingcap#40663

* server, execute: set text for execute command (pingcap#41340)

close pingcap#41156

* br: increase the sleep time in the `waitForSend` function (pingcap#41322)

close pingcap#41210

* cgroup: improve test to find cgroup compatibility issues (pingcap#41347)

close pingcap#39786

---------

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: tiancaiamao <tiancaiamao@gmail.com>
Co-authored-by: fzzf678 <108643977+fzzf678@users.noreply.github.com>
Co-authored-by: Yuanjia Zhang <zhangyuanjia@pingcap.com>
Co-authored-by: crazycs <crazycs520@gmail.com>
Co-authored-by: Mattias Jonsson <mjonss@users.noreply.github.com>
Co-authored-by: zzm <zhouzemin@pingcap.com>
Co-authored-by: Song Gao <disxiaofei@163.com>
Co-authored-by: BornChanger <97348524+BornChanger@users.noreply.github.com>
Co-authored-by: Jk Xu <54522439+Dousir9@users.noreply.github.com>
Co-authored-by: Yang Zhang <yang.zhang@pingcap.com>
Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com>
Co-authored-by: lance6716 <lance6716@gmail.com>
Co-authored-by: YangKeao <yangkeao@chunibyo.icu>
Co-authored-by: tangenta <tangenta@126.com>
Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Hangjie Mo <mohangjie1995@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
Co-authored-by: djshow832 <zhangming@pingcap.com>
Co-authored-by: Hu# <jinhao.hu@pingcap.com>
Co-authored-by: hongyunyan <649330952@qq.com>
Co-authored-by: ShuNing <nolouch@gmail.com>
Co-authored-by: Zak Zhao <57036248+joccau@users.noreply.github.com>
Co-authored-by: Yujie Xia <xiayjchn@gmail.com>
Co-authored-by: Yifan Xu <30385241+xuyifangreeneyes@users.noreply.github.com>
Co-authored-by: 王超 <cclcwangchao@hotmail.com>
Co-authored-by: Lynn <zimu_xia@126.com>
Co-authored-by: Chunzhu Li <lichunzhu@stu.xjtu.edu.cn>
Co-authored-by: dsdashun <dsdashun@qq.com>
Co-authored-by: TonsnakeLin <87681388+TonsnakeLin@users.noreply.github.com>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
Co-authored-by: hehechen <chentongli@pingcap.com>
Co-authored-by: Daniël van Eeden <git@myname.nl>
Co-authored-by: MoCuishle28 <32541204+MoCuishle28@users.noreply.github.com>
Co-authored-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: iosmanthus <MyOsmanthusTree@gmail.com>
Co-authored-by: winkyao <golangwink@gmail.com>
Co-authored-by: xufei <xufeixw@mail.ustc.edu.cn>
Co-authored-by: glorv <glorvs@163.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Chengpeng Yan <41809508+Reminiscent@users.noreply.github.com>
Co-authored-by: JmPotato <ghzpotato@gmail.com>
Co-authored-by: Zhigao Tong <tongzhigao@pingcap.com>
Co-authored-by: ystaticy <y_static_y@sina.com>
Co-authored-by: Shenghui Wu <793703860@qq.com>
Co-authored-by: fengou1 <85682690+fengou1@users.noreply.github.com>
Co-authored-by: Chen Ding <dingc05@gmail.com>
Co-authored-by: ghazalfamilyusa <113858565+ghazalfamilyusa@users.noreply.github.com>
Co-authored-by: disksing <i@disksing.com>
Co-authored-by: 山岚 <36239017+YuJuncen@users.noreply.github.com>
Co-authored-by: okJiang <819421878@qq.com>
Co-authored-by: Yongbo Jiang <cabinfeveroier@gmail.com>
Co-authored-by: Lucas <nkcs_lykx@hotmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: Yexiang Zhang <mornyx.z@gmail.com>
Co-authored-by: yanweiqi <592838129@qq.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: Zhou Kunqin <25057648+time-and-fate@users.noreply.github.com>
Co-authored-by: Arenatlx <314806019@qq.com>
Co-authored-by: Benjamin2037 <bear.c@pingcap.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qw4990 qw4990 qw4990 approved these changes

@time-and-fate time-and-fate time-and-fate approved these changes

@Yisaer Yisaer Awaiting requested review from Yisaer

Assignees
No one assigned
Labels
release-note-none size/M Denotes a PR that changes 30-99 lines, ignoring generated files. skip-issue-check status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@xuyifangreeneyes @ti-chi-bot @qw4990 @time-and-fate