autoid_service,owner: change autoid service's etcd lease to 10s #46455

tiancaiamao · 2023-08-28T09:10:49Z

What problem does this PR solve?

Issue Number: close #46454

Problem Summary:

What is changed and how it works?

Change the lease value from 60s to 10s
When autoid service panic unexpectedly, it wait lease timeout to elect a new leader to recover the service.
60s is a bit too long.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

start a tiup cluster

tiup playground  --tiflash 0

start a new tidb at 4001

./bin/tidb-server -store=tikv -path=127.0.0.1:2379 --status=10081 -P 4001

close the old tidb, so the autoid service will be fallover to the new tidb 4001

kill  1469782

restart the old tidb at 4000

./bin/tidb-server -P 4000 --store=tikv --host=127.0.0.1 --status=10080 --path=127.0.0.1:2379

run sysbench test

sysbench --config-file=config oltp_insert --tables=4 --auto_inc=on --mysql_storage_engine='innodb */ AUTO_ID_CACHE 1 /*' prepare 
sysbench --config-file=config oltp_insert --tables=4 --auto_inc=on --mysql_storage_engine='innodb */ AUTO_ID_CACHE 1 /*' run

when the test is running, force kill tidb 4001, so the autoid service leader killed, see how long it takes to recover

kill -9 1470126

Here is the sysbench log:

sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 4
Report intermediate results every 3 second(s)
Initializing random number generator from current time


Initializing worker threads...

Threads started!

[ 3s ] thds: 4 tps: 1528.89 qps: 1528.89 (r/w/o: 0.00/1528.89/0.00) lat (ms,95%): 3.55 err/s: 0.00 reconn/s: 0.00
[ 6s ] thds: 4 tps: 1568.76 qps: 1568.76 (r/w/o: 0.00/1568.76/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 9s ] thds: 4 tps: 1574.96 qps: 1574.96 (r/w/o: 0.00/1574.96/0.00) lat (ms,95%): 3.43 err/s: 0.00 reconn/s: 0.00
[ 12s ] thds: 4 tps: 1497.56 qps: 1497.56 (r/w/o: 0.00/1497.56/0.00) lat (ms,95%): 3.68 err/s: 0.00 reconn/s: 0.00
[ 15s ] thds: 4 tps: 1458.44 qps: 1458.44 (r/w/o: 0.00/1458.44/0.00) lat (ms,95%): 3.89 err/s: 0.00 reconn/s: 0.00
[ 18s ] thds: 4 tps: 1544.67 qps: 1544.67 (r/w/o: 0.00/1544.67/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 21s ] thds: 4 tps: 1544.66 qps: 1544.66 (r/w/o: 0.00/1544.66/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 24s ] thds: 4 tps: 1534.67 qps: 1534.67 (r/w/o: 0.00/1534.67/0.00) lat (ms,95%): 3.55 err/s: 0.00 reconn/s: 0.00
[ 27s ] thds: 4 tps: 1538.00 qps: 1538.00 (r/w/o: 0.00/1538.00/0.00) lat (ms,95%): 3.55 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 4 tps: 1549.69 qps: 1549.69 (r/w/o: 0.00/1549.69/0.00) lat (ms,95%): 3.43 err/s: 0.00 reconn/s: 0.00
[ 33s ] thds: 4 tps: 1540.64 qps: 1540.64 (r/w/o: 0.00/1540.64/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 36s ] thds: 4 tps: 1547.01 qps: 1547.01 (r/w/o: 0.00/1547.01/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 39s ] thds: 4 tps: 1546.99 qps: 1546.99 (r/w/o: 0.00/1546.99/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 42s ] thds: 4 tps: 1548.34 qps: 1548.34 (r/w/o: 0.00/1548.34/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 45s ] thds: 4 tps: 1546.66 qps: 1546.66 (r/w/o: 0.00/1546.66/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 48s ] thds: 4 tps: 1502.00 qps: 1502.00 (r/w/o: 0.00/1502.00/0.00) lat (ms,95%): 3.68 err/s: 0.00 reconn/s: 0.00
[ 51s ] thds: 4 tps: 222.00 qps: 222.00 (r/w/o: 0.00/222.00/0.00) lat (ms,95%): 3.30 err/s: 0.00 reconn/s: 0.00
[ 54s ] thds: 4 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 57s ] thds: 4 tps: 496.67 qps: 496.67 (r/w/o: 0.00/496.67/0.00) lat (ms,95%): 3.30 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 4 tps: 1631.66 qps: 1631.66 (r/w/o: 0.00/1631.66/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 63s ] thds: 4 tps: 1640.00 qps: 1640.00 (r/w/o: 0.00/1640.00/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 66s ] thds: 4 tps: 1606.67 qps: 1606.67 (r/w/o: 0.00/1606.67/0.00) lat (ms,95%): 3.43 err/s: 0.00 reconn/s: 0.00
[ 69s ] thds: 4 tps: 1637.34 qps: 1637.34 (r/w/o: 0.00/1637.34/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 72s ] thds: 4 tps: 1606.43 qps: 1606.43 (r/w/o: 0.00/1606.43/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 75s ] thds: 4 tps: 1611.90 qps: 1611.90 (r/w/o: 0.00/1611.90/0.00) lat (ms,95%): 3.55 err/s: 0.00 reconn/s: 0.00
[ 78s ] thds: 4 tps: 1638.99 qps: 1638.99 (r/w/o: 0.00/1638.99/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 81s ] thds: 4 tps: 1603.35 qps: 1603.35 (r/w/o: 0.00/1603.35/0.00) lat (ms,95%): 3.43 err/s: 0.00 reconn/s: 0.00
[ 84s ] thds: 4 tps: 1609.00 qps: 1609.00 (r/w/o: 0.00/1609.00/0.00) lat (ms,95%): 3.43 err/s: 0.00 reconn/s: 0.00
[ 87s ] thds: 4 tps: 1657.99 qps: 1657.99 (r/w/o: 0.00/1657.99/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 90s ] thds: 4 tps: 1636.33 qps: 1636.33 (r/w/o: 0.00/1636.33/0.00) lat (ms,95%): 3.36 err/s: 0.00 reconn/s: 0.00
[ 93s ] thds: 4 tps: 1583.99 qps: 1583.99 (r/w/o: 0.00/1583.99/0.00) lat (ms,95%): 3.49 err/s: 0.00 reconn/s: 0.00
[ 96s ] thds: 4 tps: 1527.01 qps: 1527.01 (r/w/o: 0.00/1527.01/0.00) lat (ms,95%): 3.68 err/s: 0.00 reconn/s: 0.00
[ 99s ] thds: 4 tps: 1518.00 qps: 1518.00 (r/w/o: 0.00/1518.00/0.00) lat (ms,95%): 3.62 err/s: 0.00 reconn/s: 0.00
[ 102s ] thds: 4 tps: 1510.94 qps: 1510.94 (r/w/o: 0.00/1510.94/0.00) lat (ms,95%): 3.75 err/s: 0.00 reconn/s: 0.00
[ 105s ] thds: 4 tps: 1441.71 qps: 1441.71 (r/w/o: 0.00/1441.71/0.00) lat (ms,95%): 3.96 err/s: 0.00 reconn/s: 0.00
[ 108s ] thds: 4 tps: 1449.67 qps: 1449.67 (r/w/o: 0.00/1449.67/0.00) lat (ms,95%): 3.96 err/s: 0.00 reconn/s: 0.00
[ 111s ] thds: 4 tps: 1522.21 qps: 1522.21 (r/w/o: 0.00/1522.21/0.00) lat (ms,95%): 3.62 err/s: 0.00 reconn/s: 0.00
^C

Before this fix, I once tested and found it took around 45s to recover.
Now the unavaiable time is within 10s. (see the log from 51s - 57s)

No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2023-08-28T09:11:14Z

Hi @tiancaiamao. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2023-08-28T09:21:06Z

Codecov Report

Merging #46455 (aeb9349) into master (5fbe25e) will decrease coverage by 1.1384%.
Report is 35 commits behind head on master.
The diff coverage is 100.0000%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #46455        +/-   ##
================================================
- Coverage   73.3441%   72.2057%   -1.1384%     
================================================
  Files          1309       1336        +27     
  Lines        394540     407887     +13347     
================================================
+ Hits         289372     294518      +5146     
- Misses        86753      94790      +8037     
- Partials      18415      18579       +164

Flag	Coverage Δ
integration	`25.9377% <12.5000%> (?)`
unit	`73.3972% <100.0000%> (+0.0530%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`54.0444% <ø> (ø)`
parser	`85.0500% <ø> (+0.0727%)`	⬆️
br	`46.9307% <ø> (-5.4364%)`	⬇️

autoid_service/autoid.go

ti-chi-bot · 2023-09-04T09:02:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, zimulala

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hawkingrei,zimulala]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2023-09-04T09:02:31Z

[LGTM Timeline notifier]

Timeline:

2023-08-29 02:35:04.513013541 +0000 UTC m=+1808069.062029528: ☑️ agreed by zimulala.
2023-09-04 09:02:29.792747182 +0000 UTC m=+2349714.341763168: ☑️ agreed by hawkingrei.

tiancaiamao · 2023-09-04T11:48:30Z

/retest-required

tiprow · 2023-09-04T11:49:03Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest-required

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tiancaiamao · 2023-09-04T13:20:46Z

/test unit-test

tiprow · 2023-09-04T13:21:20Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hawkingrei · 2023-09-04T15:51:02Z

/retest

tiancaiamao · 2023-09-05T01:28:21Z

/run-cherry-picker

ti-chi-bot · 2023-09-05T01:28:26Z

In response to a cherrypick label: new pull request created to branch release-7.1: #46653.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

…) (#46653) close #46454

dulao5 · 2023-09-06T11:13:56Z

I devised a test method for this scenario, which proved to be as expected.

Expected effect

When the autoid owner job crashes, the recovery time of OLTP (insert) is optimized from around 60s to 10s
DDL owner job crash, DDL recovery time is not affected (60s or so)

Verification method

Modify sysbench, use AUTO_CACHE_ID = 1 option when creating tables
Verify OLTP (insert) scenarios with sysbench oltp_insert
Use the shell to send one ddl statement per second to verify the ddl job scenario
Use v7.1.1 and nightly(7.4-nightly) respectively, start multiple tidb processes with tiup playground, use kill -9 to kill the tidb process where the owner job is located to confirm the recovery time.

Verification results

✅ as expected

	v7.1.1 (current status)	v7.4.0-alpha-nightly-20230905	results
DDL recovery time	82s	85s	✅ unchanged
OLTP (insert) recovery time	52s	10s	✅ been sped up

Details

OLTP (insert) recovery time -- before

OLTP (insert) recovery time -- after

DDL recovery time -- before

DDL recovery time -- after

…cap#46455) close pingcap#46454

* lightning: fix lightning failed to log encoding error (pingcap#45241) (pingcap#45365) close pingcap#44321 * *: Fix data race between getting labels and setting labels in config (pingcap#45563) (pingcap#45568) close pingcap#45561 * store: Don't set limit for GetRegionsInfoByRange function (pingcap#45556) (pingcap#45571) close pingcap#45531 * parser: support more join hints on parser (pingcap#45525) (pingcap#45574) ref pingcap#45520 * *: Fix data race in TestSetLabelsConcurrentWithGetLabel (pingcap#45579) (pingcap#45590) close pingcap#45578 * planner: support `no_hash_join` hint on optimizer (pingcap#45538) (pingcap#45635) ref pingcap#45520 * planner: support `no_merge_join` hint on optimizer (pingcap#45562) (pingcap#45705) ref pingcap#45520 * planner: refine `tryToGetIndexJoin` (pingcap#45587) (pingcap#45713) ref pingcap#45520 * ddl: create temp dir automatically for adding index (pingcap#45457) (pingcap#45475) close pingcap#45456 * statistics: kill in time when to mergeGlobalTopN with partition (pingcap#45707) (pingcap#45709) close pingcap#45706 * executor, util: make tmp-storage-quota take affect (pingcap#45549) (pingcap#45732) close pingcap#45161 * planner: refactor `tryToGetIndexJoin` (pingcap#45617) (pingcap#45841) ref pingcap#45520 * lightning: fix routes panic for csv data load (pingcap#45405) (pingcap#45537) close pingcap#43284 * planner: support `no_index_join`, `no_index_hash_join`, `no_index_merge_join` hints (pingcap#45633) (pingcap#45882) ref pingcap#45520 * parser: reset `lexer.identifierDot` correctly when reusing parsers (pingcap#45903) (pingcap#45910) close pingcap#45898 * executor, planner: add extraProj for indexLookUp with order by + limit + static prune (pingcap#45771) close pingcap#45757 * statistics: remove useless code in the MergePartTopN2GlobalTopN (pingcap#45755) (pingcap#45769) close pingcap#45706 * statistics: kill in time when to run topnStatsMergeWorker and improve memory (pingcap#45765) (pingcap#45768) close pingcap#45706 * snap_restore: resend `recover_region` while there are TiKV restarts (pingcap#45361) (pingcap#45722) close pingcap#45206 * statistics: reduce memory usage when to MergePartTopN2GlobalTopN (pingcap#45718) (pingcap#45971) close pingcap#45727 * calibrate: shorten the minimum calibrate duration (pingcap#44972) (pingcap#46007) ref pingcap#43212, close pingcap#46006 * *: set go1.20.7 (pingcap#46027) close pingcap#46026 * ttl: make ttl manually trigger stable (pingcap#45869) (pingcap#45896) close pingcap#45868 * statistics: add bench for MergeGlobalStatsTopNByConcurrency (pingcap#45998) (pingcap#46031) ref pingcap#45281 * statistics: improve memory for mergeGlobalStatsTopNByConcurrency (pingcap#45993) (pingcap#46058) close pingcap#45727 * ttl: fix some memory leak in TTL (pingcap#45512) (pingcap#45514) close pingcap#45510 * lightning: make OpLevelOptional suppress the error of DoChecksum (pingcap#45486) (pingcap#45867) close pingcap#45382 * expression: Fix different behaviors with MySQL when comparing datetime column with numeric constant | tidb-test=pr/2198 (pingcap#46129) close pingcap#38361 * br: make download metadata concurrency adjustable (pingcap#45639) (pingcap#45845) close pingcap#45511 * executor: Fix coroutine leak after kill remote connection (pingcap#46035) (pingcap#46067) close pingcap#46034 * br: Fix RangeTree.GetIncompleteRange when the rangeTree is empty. (pingcap#37086) (pingcap#46168) * *: modify headers of some source files in `privilege` (pingcap#46222) (pingcap#46236) * ddl: use the correct timezone to encode record for adding index (pingcap#46055) (pingcap#46103) close pingcap#46033 * parser: fix parser identifier with dot (pingcap#46314) (pingcap#46323) close pingcap#35031 * lightning: "no leader" should be a retryable error (pingcap#46300) (pingcap#46317) close pingcap#45673 * lightning: save NEXT_GLOBAL_ROW_ID as uint64 instead of int64 (pingcap#45614) (pingcap#45940) close pingcap#45427 * executor: optimize load data assignment expression (pingcap#46563) close pingcap#46081 * br: fix restore metakv without default cf files (pingcap#46589) (pingcap#46643) close pingcap#46578 * autoid_service,owner: change autoid service's etcd lease to 10s (pingcap#46455) (pingcap#46653) close pingcap#46454 * executor: fix hang when index join newInnerworker panic (pingcap#45890) (pingcap#46114) close pingcap#45716 * br: adjust default tidb global config for br (pingcap#45794) (pingcap#46192) close pingcap#45793 * br: configure the httpclient for external storage (pingcap#46040) (pingcap#46141) close pingcap#46011 * planner: refactor Join and Limit's ResolveIndices (pingcap#45831) (pingcap#45864) close pingcap#45758, close pingcap#45805 * expression: fix wrong result for unsigned non-const int cmp const duration (pingcap#46620) (pingcap#46703) close pingcap#45410 * executor: fix batchPoint for partition table with duplicate rows (pingcap#46808) (pingcap#46885) close pingcap#46779 * statistics: add historgram bench for MergePartitionHist2GlobalHist (pingcap#46788) (pingcap#47038) close pingcap#46805 * br: not use the custom http client for gcs (pingcap#47026) (pingcap#47058) close pingcap#47022 * planner: don't recompute the hashcode when generated column substitution doesn't happen (pingcap#46450) (pingcap#46630) close pingcap#42788 * *: add bench for analyze (pingcap#46898) (pingcap#47249) close pingcap#46914 * hint: fix panic when query block not found in prepare statement (pingcap#46818) (pingcap#46843) close pingcap#46817 * br: fix unstable test br_300_small_tables (pingcap#46560) (pingcap#46568) close pingcap#46561 * executor: fix plan replayer for sql file input wrongly fetch startTS after `OnTxnEnd` (pingcap#46201) (pingcap#46345) close pingcap#46197 * statistics, planner: use the correct `IndexInfo` in `GenerateHistCollFromColumnInfo()` | tidb-test=pr/2218 (pingcap#44441) (pingcap#46730) close pingcap#44369 * lightning: fix incorrect _tidb_rowid allocator value after import for table with AUTO_ID_CACHE=1 (pingcap#46171) (pingcap#46184) close pingcap#46100 * br: check the correct changefeed info when restore/import data (pingcap#47322) (pingcap#47370) close pingcap/tiflow#9807 * *: fix the duplicate entry error when using BR to restore a NONCLUSTERED AUTO_ID_CACHE=1 table (pingcap#46127) (pingcap#46338) close pingcap#46093 * br: wait tiflash replicas ready && fix unstable test (pingcap#46301) (pingcap#46341) close pingcap#46302 * ddl, session: using table ID instead of partition ID when calling `SplitRegions` (pingcap#46156) (pingcap#46161) close pingcap#46135 * util: change log level to debug calculating stale read ts (pingcap#47309) (pingcap#47453) close pingcap#47308 * ddl: fix DST times for adding index (pingcap#47425) (pingcap#47448) ref pingcap#46033, close pingcap#47426 * ddl: Exchange partition rollback (pingcap#45877) (pingcap#45981) close pingcap#45791, close pingcap#45920 * br: avoid possible infinite loop checking tikv gc and scheduling status (pingcap#46078) (pingcap#46196) close pingcap#46073 * br: fix the split issue in txn restore (pingcap#45441) (pingcap#45548) close pingcap#45476 * *: Exchange partition, fix LIST COLUMNs validation as well as NULL validation (pingcap#46533) (pingcap#46697) close pingcap#46492 * domain: use dedicated lock for expiredTimeStamp4PC (pingcap#45802) (pingcap#47519) close pingcap#45400 * autoid_service: fix potential 'duplicated entry' error when tidb exit for AUTO_ID_CACHE=1 tables (pingcap#46445) (pingcap#46500) close pingcap#46444 * domain: fix a bug when reloading take a long time in domain.Init (pingcap#45170) (pingcap#45391) close pingcap#45176 * autoid_service: add some debug log for rebase related operation (pingcap#46526) (pingcap#46530) ref pingcap#46444 * ddl: relax the check in ownerCheckAllVersions (pingcap#46752) (pingcap#46794) close pingcap#46751 * planner: clean the one side filters for outer join reorder which is not done in pingcap#44409 (pingcap#45565) (pingcap#45586) ref pingcap#44409, close pingcap#45564 * planner: fix the issue that TIDB_INLJ hint cannot take effect when left joining two sub-queries (pingcap#46271) (pingcap#46291) close pingcap#46160 * planner: DML queries might panic when CTE exists (pingcap#46092) (pingcap#46097) close pingcap#46083 * snap_restore: added retry for recovery (pingcap#46094) (pingcap#46225) close pingcap#46181 * planner: fix Uncertain Results caused by MERGE_JOIN (pingcap#47078) (pingcap#47081) close pingcap#46580 * br: fix add ingest index as sub job (pingcap#47484) (pingcap#47556) close pingcap#47482 * planner: Adjust the log level and returned value when `cacheableChecker` check `*ast.TableName` nodes (pingcap#46831) (pingcap#46877) close pingcap#46760 * planner: disallow split clustered index with SPLIT TABLE .. INDEX (pingcap#47412) (pingcap#47436) close pingcap#47350 * planner: fix cast(col) = range couldn't build range when cast function doesn't contain any precision loss in some cases | tidb-test=pr/2224 (pingcap#46303) (pingcap#46548) close pingcap#45199 * planner: fix group_concat function couldn't resolve the index of its order-by item (pingcap#46419) (pingcap#47121) close pingcap#41986 * planner: adjust N used in TopN cost formula based on the total number of rows (pingcap#46368) (pingcap#46571) close pingcap#43285 * planner: increase the maximum number limit of TopN when analyzing tables (pingcap#45962) (pingcap#45973) ref pingcap#45919 * ddl: fix a bug that MDL may progress unexpectedly or block forever (pingcap#46921) (pingcap#47069) close pingcap#46920 * ddl: fix duplicate inforSchema information of rename tables (pingcap#47087) (pingcap#47142) close pingcap#47064 * OWNERS: Auto Sync OWNERS files from community membership (pingcap#47615) * planner: don't force set the Column in physical proj elimination (pingcap#45824) (pingcap#46132) close pingcap#45804 * *: update dependency for uber-go/automaxprocs (pingcap#47605) close pingcap#47601 * planner: store the hints of session variable (pingcap#45814) (pingcap#46047) close pingcap#45812 * planner: consider Unspecified length when setting length for unhex (pingcap#45449) (pingcap#45451) close pingcap#45378 * planner: fix index heuristic rule will prune out hint preferred tiflash path (pingcap#46102) (pingcap#46145) close pingcap#40146 * planner: fix group concat projection elimination logic (pingcap#47612) (pingcap#47628) close pingcap#41957 * planner: log the reason why the sample-rate is chosen when analyzing table (pingcap#45938) (pingcap#45968) close pingcap#45936 * session: set request source to type of the prepared stmt for execute stmt (pingcap#46348) (pingcap#46354) close pingcap#46349 * *: add MVCC version ratio to slow log metrics (pingcap#44897) (pingcap#47645) close pingcap#44896 * extension: fix enterprise git commit hash display in `tidb-server -V` (pingcap#47647) (pingcap#47652) close pingcap#47646 * executor: Fix crash during sort spill (pingcap#47581) (pingcap#47625) close pingcap#47538 * binlog: fix the wrong fatal error when binlog is too large (pingcap#46365) (pingcap#46380) close pingcap#28659 * dumpling: fix dumpling ignore file writer close error (pingcap#45374) (pingcap#45404) close pingcap#45353 * planner: output a warning if plan rebuilding fails when reusing a cached plan (pingcap#46278) (pingcap#46281) close pingcap#46159 * executor, planner: fix plan_replayer zip format (pingcap#47474) (pingcap#47713) close pingcap#46474 * statistics: check Killed in the TableStatsFromStorage (pingcap#47568) (pingcap#47641) close pingcap#47570 * lightning/backend/local: fix `buildIndexDupTasks` (pingcap#44442) (pingcap#47894) close pingcap#44439 * executor: fix an issue that index_lookup_hash_join incorrectly estimate memory usage. (pingcap#47795) (pingcap#47925) close pingcap#47788 * br: avoid retry if it's ec2metadata (pingcap#47651) (pingcap#47790) close pingcap#47650 * br: add integration test for pitr (pingcap#47740) (pingcap#47906) ref pingcap#47738 * planner: do not convert update to point get if the expr has sub-query (pingcap#47454) (pingcap#47735) close pingcap#47445 * statistic: fix panic when building topN (pingcap#47928) (pingcap#47990) close pingcap#35948 * local backend: fix log been eaten when failed on execute region job (pingcap#47782) (pingcap#47814) close pingcap#47781 * local backend: fix worker err overriden by job generation err (pingcap#48185) (pingcap#48198) close pingcap#47992 * executor: reuse chunk for GetMatchedRowsAndPtrs calls (pingcap#48090) (pingcap#48105) close pingcap#48082 * pdutil/backend: enlarge max retry time and fix nested retriable error (pingcap#48210) (pingcap#48227) close pingcap#46950 * lightning: every HTTP retry should use its own request (pingcap#47959) (pingcap#47968) close pingcap#47930 * statistic: improve ipv6 for analyze job meta (pingcap#45177) (pingcap#45185) ref pingcap#43260 * statistics: check Killed in the GenJSONTableFromStats (pingcap#47778) (pingcap#47824) close pingcap#47779 * util/cgroup: fix InContainer for cgroup v2 (pingcap#48344) (pingcap#48345) close pingcap#48342 * *: upgrade 1.20.11 (pingcap#48385) ref pingcap#48383 * resource_control: add duration buffer for calibrate time window (pingcap#48172) (pingcap#48353) close pingcap#48162 * statistics: avoid oom when to gc large stats_history (pingcap#48430) (pingcap#48489) close pingcap#48431 * planner, CTE: Fix default inline CTE which contains agg or window function and refactor inline CTE strategy (pingcap#48436) close pingcap#47711 * copr: fix copr cache panic when `tidb_enable_collect_execution_info` is off (pingcap#48340) (pingcap#48475) close pingcap#48212 * executor: AnalyzePartitionConcurrency is not more than number of task (pingcap#47463) (pingcap#48547) close pingcap#47464 * domain: randomly trigger the DumpStatsDeltaToKV (pingcap#48692) (pingcap#48704) close pingcap#48693 * ddl: fix issue of alter last partition failed when partition column is datetime (pingcap#48815) (pingcap#48847) close pingcap#48814 * planner: fix nil pointer at expression.(*CorrelatedColumn).Eval (pingcap#42789) (pingcap#48977) close pingcap#42739 * *: handle region error for GetMvccByEncodedKey API (pingcap#47811) (pingcap#47822) close pingcap#47807 * pkg/util: refine tidb_server_memory_limit to make the cpu usage more stable (pingcap#48927) (pingcap#49061) close pingcap#48741 * session: fix select for update statement can't get stmt-count-limit error (pingcap#48412) (pingcap#48468) close pingcap#48411 * domain: add resolve lock logic for mvcc get key loading schema diff (pingcap#48330) (pingcap#48417) close pingcap#48281 * *: exclude `**/.git/**` in licenserc.yml (pingcap#49157) (pingcap#49158) close pingcap#49156 * br: add more retryable error for br_import (pingcap#48529) (pingcap#48543) close pingcap#48528 * extension: update submodule for release-7.1 (pingcap#49175) close pingcap#49133 * executor: fix index out of range panic of cte when max_chunk_size is samll (pingcap#48839) (pingcap#49004) close pingcap#48808 * executor: fix deadlock in dml statement with cte when oom panic action was triggered (pingcap#49192) (pingcap#49202) close pingcap#49096 * errmsg: fix example s3 url in error msg (pingcap#49207) (pingcap#49208) close pingcap#49196 * *: support concurrent write for S3 writer (pingcap#45723) (pingcap#49185) ref pingcap#45719, close pingcap#48607 * planner: fixing wrong result after applying predicate push down for CTEs (pingcap#47891) (pingcap#48193) close pingcap#47881 * util/ranger: don't exclude start key for range from `_` in `like` function | tidb-test=pr/2258 (pingcap#48984) (pingcap#49065) ref pingcap#48181, close pingcap#48983 * br: retry to scatter the regions if status is timeout or cancel (pingcap#46471) (pingcap#48378) close pingcap#47236 * planner/core: keep sort operator when ordered by tablesample (pingcap#48315) (pingcap#48450) close pingcap#48253 * s3: fix s3 concurrent uploader will overwrite error (pingcap#48163) (pingcap#49221) close pingcap#48164 * lightning: add timeout for "write" RPC (pingcap#48355) (pingcap#48397) close pingcap#46321, close pingcap#48352 * *: fix data race of Column.GeneratedExpr (pingcap#48888) (pingcap#48923) close pingcap#44919, close pingcap#48191 * planner: Fix the issue that may generate many plandigests when the inner table is clustered (pingcap#47952) (pingcap#48112) close pingcap#47634 * planner: fix the issue that the optimizer terminates the optimization process for `DataSource` too early (pingcap#48186) (pingcap#48266) close pingcap#46177 * executor/brie: use the default value from flags (pingcap#48025) (pingcap#48442) close pingcap#48000 * planner: fix possible inconsistent output cols among union's children (pingcap#48775) (pingcap#48895) close pingcap#48755 * executor, codec: hash join build wrong hash key for ENUM/SET value (pingcap#49031) (pingcap#49055) close pingcap#48991 * br: fix incorrect uri for the file storage (pingcap#48453) (pingcap#48722) close pingcap#48452 * BR: Support encryption for restored ebs volumes (pingcap#48900) (pingcap#48937) close pingcap#48899 * plugin: fix bug that watch loop will refresh frequently when channel closed (pingcap#49275) (pingcap#49289) close pingcap#49273 * planner: fix a panic during column pruning (pingcap#47883) (pingcap#48809) close pingcap#47331 * ddl: fix recover table by JobID bug when JobID is set to 0 tidb-server panic (pingcap#46343) (pingcap#48086) close pingcap#46296 * pkg/lightning : remove get_regions call in physical backend (pingcap#46202) (pingcap#46337) close pingcap#45507 * planner: check nil pointer in rule_result_reorder to avoid panic (pingcap#48100) (pingcap#48116) close pingcap#45044 * planner: plan cache supports Batch/PointGet converted from (primary keys) in ((...), ...) (pingcap#44838) (pingcap#49380) close pingcap#44830 * executor: fill correlated column value in late materialization filter conditions (pingcap#49244) (pingcap#49431) close pingcap#49241 * executor: set OverflowAsWarning for insert statement in non-strict sql mode (pingcap#49383) (pingcap#49459) close pingcap#49369 * planner: fix tiflash path will be identified as table dual (pingcap#49338) (pingcap#49395) close pingcap#49285 * expression: enum/set could be invalid during evaluation (pingcap#49543) (pingcap#49550) close pingcap#49487 * pkg/executor: fix the hang issue in indexHashJoin (pingcap#49218) (pingcap#49411) close pingcap#49033 * cherry-pick: mannual cherry-pick 49421 back to release 7.1 (pingcap#49592) close pingcap#49377 * executor: use EncodeKey to encode groupkey in StreamAggExec (pingcap#49974) (pingcap#50026) close pingcap#49902 * executor: avoid concurrently update for `SimpleLRUCache` in `ApplyCache` (pingcap#50356) (pingcap#50385) close pingcap#50347 * fix ut Signed-off-by: zzm <zhouzemin@pingcap.com> * fix ut Signed-off-by: zzm <zhouzemin@pingcap.com> * tag this branch image as v7.1 for e2e test Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 79a02af Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick a52c366 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 3652a93 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick d121f46 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick a1ea04c Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick e07cf40 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick e5004d8 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick fc4edaf Signed-off-by: zzm <zhouzemin@pingcap.com> * planner: fix issue 48643 that aggDesc modification will change the referrence (pingcap#48662) close pingcap#48643 * cherry-pick 72e5323 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick eac9cd0 Signed-off-by: zzm <zhouzemin@pingcap.com> * resource_control: fetch cpu quota metrics from store instead of prometheus (pingcap#49176) (pingcap#49255) close pingcap#49174 * cherry-pick 6b72ccd Signed-off-by: zzm <zhouzemin@pingcap.com> * pkg: support the TSO format for asof expression (pingcap#48371) (pingcap#48380) close pingcap#48372 * ebs br: control the snapshots batch size for fsr enable/disable (pingcap#48506) (pingcap#48526) close pingcap#48505 * cherry-pick eacc493 Signed-off-by: zzm <zhouzemin@pingcap.com> * ebs br: make sure fsr credit is full filled (pingcap#48627) (pingcap#48743) close pingcap#48629 * cherry-pick 950521e Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 36c74e6 Signed-off-by: zzm <zhouzemin@pingcap.com> * planner: fix leading hint cannot take effect in UNION ALL statements (pingcap#50277) (pingcap#50322) close pingcap#50067 * make check Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 38fb0eb Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 0312e78 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 0d9903c Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 359aea8 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick a56af1b Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 536545d Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 9da9d4d Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 14a07f2 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 126cf52 Signed-off-by: zzm <zhouzemin@pingcap.com> * make check Signed-off-by: zzm <zhouzemin@pingcap.com> * update kvproto Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick b003558 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick acdac74 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 49e9daf Signed-off-by: zzm <zhouzemin@pingcap.com> * pdutil: fix retry reusing body reader (pingcap#48312) (pingcap#48320) close pingcap#48307 * cherry-pick f249304 Signed-off-by: zzm <zhouzemin@pingcap.com> * cherry-pick 575a4e6 Signed-off-by: zzm <zhouzemin@pingcap.com> * parser: fix limit lost in SetOprStmt (pingcap#49898) (pingcap#49931) close pingcap#49874 * build br/lightning Signed-off-by: zzm <zhouzemin@pingcap.com> * fix lightning panic Signed-off-by: zzm <zhouzemin@pingcap.com> * make check Signed-off-by: zzm <zhouzemin@pingcap.com> * fix build br Signed-off-by: zzm <zhouzemin@pingcap.com> * revert cd Signed-off-by: zzm <zhouzemin@pingcap.com> --------- Signed-off-by: zzm <zhouzemin@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: Hangjie Mo <mohangjie1995@gmail.com> Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com> Co-authored-by: yibin <huyibin@pingcap.com> Co-authored-by: lance6716 <lance6716@gmail.com> Co-authored-by: 王超 <cclcwangchao@hotmail.com> Co-authored-by: Yuanjia Zhang <zhangyuanjia@pingcap.com> Co-authored-by: Arenatlx <314806019@qq.com> Co-authored-by: Lynn <zimu_xia@126.com> Co-authored-by: crazycs <crazycs520@gmail.com> Co-authored-by: ShuNing <nolouch@gmail.com> Co-authored-by: BornChanger <97348524+BornChanger@users.noreply.github.com>

autoid_service,owner: change autoid service's etcd lease to 10s

9ce58a1

ti-chi-bot bot added release-note-none size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 28, 2023

zimulala approved these changes Aug 29, 2023

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm labels Aug 29, 2023

hawkingrei reviewed Aug 29, 2023

View reviewed changes

autoid_service/autoid.go Show resolved Hide resolved

tiancaiamao added 2 commits September 1, 2023 09:21

address comment, Merge branch 'master' into autoid-lease

e50cf33

fix lint check

aeb9349

tiancaiamao requested a review from hawkingrei September 1, 2023 11:46

hawkingrei approved these changes Sep 4, 2023

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm labels Sep 4, 2023

ti-chi-bot bot merged commit 18ecfba into pingcap:master Sep 4, 2023
15 of 18 checks passed

tiancaiamao deleted the autoid-lease branch September 5, 2023 01:16

tiancaiamao added the needs-cherry-pick-release-7.1 label Sep 5, 2023

ti-chi-bot mentioned this pull request Sep 5, 2023

autoid_service,owner: change autoid service's etcd lease to 10s (#46455) #46653

Merged

12 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Sep 5, 2023

This is an automated cherry-pick of pingcap#46455

9e4fe0c

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot bot pushed a commit that referenced this pull request Sep 6, 2023

autoid_service,owner: change autoid service's etcd lease to 10s (#46455…

2d63b90

…) (#46653) close #46454

yibin87 pushed a commit to yibin87/tidb that referenced this pull request Oct 31, 2023

autoid_service,owner: change autoid service's etcd lease to 10s (ping…

ee5b77f

…cap#46455) close pingcap#46454

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

autoid_service,owner: change autoid service's etcd lease to 10s #46455

autoid_service,owner: change autoid service's etcd lease to 10s #46455

tiancaiamao commented Aug 28, 2023 •

edited

Loading

tiprow bot commented Aug 28, 2023

codecov bot commented Aug 28, 2023 •

edited

Loading

ti-chi-bot bot commented Sep 4, 2023

ti-chi-bot bot commented Sep 4, 2023

tiancaiamao commented Sep 4, 2023

tiprow bot commented Sep 4, 2023

tiancaiamao commented Sep 4, 2023

tiprow bot commented Sep 4, 2023

hawkingrei commented Sep 4, 2023

tiancaiamao commented Sep 5, 2023

ti-chi-bot commented Sep 5, 2023

dulao5 commented Sep 6, 2023 •

edited

Loading

autoid_service,owner: change autoid service's etcd lease to 10s #46455

autoid_service,owner: change autoid service's etcd lease to 10s #46455

Conversation

tiancaiamao commented Aug 28, 2023 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

tiprow bot commented Aug 28, 2023

codecov bot commented Aug 28, 2023 • edited Loading

Codecov Report

ti-chi-bot bot commented Sep 4, 2023

ti-chi-bot bot commented Sep 4, 2023

[LGTM Timeline notifier]

tiancaiamao commented Sep 4, 2023

tiprow bot commented Sep 4, 2023

tiancaiamao commented Sep 4, 2023

tiprow bot commented Sep 4, 2023

hawkingrei commented Sep 4, 2023

tiancaiamao commented Sep 5, 2023

ti-chi-bot commented Sep 5, 2023

dulao5 commented Sep 6, 2023 • edited Loading

Expected effect

Verification method

Verification results

Details

tiancaiamao commented Aug 28, 2023 •

edited

Loading

codecov bot commented Aug 28, 2023 •

edited

Loading

dulao5 commented Sep 6, 2023 •

edited

Loading