Skip to content

Commit

Permalink
# This is a combination of 58 commits.
Browse files Browse the repository at this point in the history
# This is the 1st commit message:

Add latency stats for kv get and kv batch get

Signed-off-by: mornyx <mornyx.z@gmail.com>

# This is the commit message #2:

tikv_util: upgrade `gag` from 0.1 to 1.0 (tikv#11277)

ref tikv#11276

Signed-off-by: kennytm <kennytm@gmail.com>
# This is the commit message tikv#3:

codec: add compile time assert to guarantee safety (tikv#11357)

* codec: add compile time assert to guarantee safety

close tikv#11356.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* add more explanation.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#4:

rafstore: fix commit state check in apply thread (tikv#11401)

* fix apply check & close tikv#11396

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* make format

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* format

Signed-off-by: gengliqi <gengliqiii@gmail.com>
# This is the commit message tikv#5:

backup: create external storage per-backup reqeust. (tikv#11113)

* backup: create external storage per-backup reqeust.

fixed tikv#11112

Signed-off-by: yujuncen <yujuncen@pingcap.com>

* backup: make clippy happy

Signed-off-by: yujuncen <yujuncen@pingcap.com>

* backup: fix build

Signed-off-by: yujuncen <yujuncen@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#6:

Remove duplicate enum and macro about numberic_enum_mod (tikv#11347)

* remove duplicated and useless `remove duplicated and useless`

close tikv#11297

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* remove duplicate macro rocksdb_enum_mod

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* fix format problem

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* resolve discussion

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* fix test error

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* move macro `numeric_enum_serializing_mod` to a proper location

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* move `perf_level_serde` of mod config in engine trait to mod perf_context

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

* Add test about `rate-limiter-mode` and fix compile error in integration test

Signed-off-by: yuqi1129 <yuqi4733@gmail.com>

Co-authored-by: Xinye Tao <xy.tao@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#7:

ctl: detect raft db correctly (tikv#11395)

Close tikv#11393.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#8:

 Support to iterate table properties for KvEngine  (tikv#11388)

* support to iterate table properties for KvEngine
* close tikv#11387

Signed-off-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#9:

gc: Fix GC scan effectiveness to avoid OOM (tikv#11416)

* close tikv#11410 limit the pending task for gc worker

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* avoid using prefix seek

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* rename test

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* address comment

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* fix build

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#10:

fix a panic about parsing memory.limit_in_bytes (tikv#11354)

* handle int overflow for memory.limit_in_bytes

clsoe tikv#11353.
Signed-off-by: qupeng <qupeng@pingcap.com>

* close tikv#11353

Signed-off-by: qupeng <qupeng@pingcap.com>

* address comments

Signed-off-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#11:

build: fix make docker on Linux (tikv#11349)

* fix make docker

fix tikv#11312

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix cargo

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#12:

engine_rocks: ignore NoSpace error during flush or compaction (tikv#11225)

* ignore NoSpace error during flush or compaction

fix tikv#11224

Signed-off-by: tabokie <xy.tao@outlook.com>

* use constant

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix typo

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#13:

gc: set pending capacity for gc worker (tikv#11429)

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
# This is the commit message tikv#14:

raftstore: support synchronous write (tikv#11418)

* support sync write & ref tikv#11165

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* add tests

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* change default store-io-pool-size to 0

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* remove unnecessary test

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* merge raftstore: separate read write ready(tikv#10592)

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* address comments

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* fix test

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* make clippy happy

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* address comment

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* remove offset in NormalFsm

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* make format

Signed-off-by: gengliqi <gengliqiii@gmail.com>
# This is the commit message tikv#15:

metrics: add Raft Engine panel (tikv#11415)

Signed-off-by: tabokie <xy.tao@outlook.com>
# This is the commit message tikv#16:

resource-metering: add some metrics (tikv#11414)

* resource-metering: add some metrics

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* more log

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* polish description, ref tikv#11436

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#17:

metrics: add asynchronous write io panels (tikv#11408)

* update metrics name

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* update tikv-details & close tikv#10540

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* rename Raft GC to Raft Log GC

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* address comment

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* fix datasource

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* add more metrics

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* update metrics

Signed-off-by: gengliqi <gengliqiii@gmail.com>

* update

Signed-off-by: gengliqi <gengliqiii@gmail.com>
# This is the commit message tikv#18:

storage: fix reverse scan check memory locks (tikv#11447)

close tikv#11440

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
# This is the commit message tikv#19:

test: make test_sending_fail_with_net_error more statable (tikv#11366)

* sending count should deregister on time

Signed-off-by: bufferflies <1045931706@qq.com>

* make test statable

Signed-off-by: bufferflies <1045931706@qq.com>

* remove unused change

Signed-off-by: bufferflies <1045931706@qq.com>

* remove unused change

Signed-off-by: bufferflies <1045931706@qq.com>

* close tikv#11365

Signed-off-by: bufferflies <1045931706@qq.com>

* reformat annotation

Signed-off-by: bufferflies <1045931706@qq.com>

* add send once filter

Signed-off-by: bufferflies <1045931706@qq.com>

* lint pass

Signed-off-by: bufferflies <1045931706@qq.com>

* lint pass

Signed-off-by: bufferflies <1045931706@qq.com>

* replace sendOnceMsgFilter

Signed-off-by: bufferflies <1045931706@qq.com>

* remove unnecessary change

Signed-off-by: bufferflies <1045931706@qq.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#20:

raftstore: increase batch for raftlog-gc-worker (tikv#11405)

* increase batch

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* do not change seek

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* fix fmt

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* fix test

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* fix number overflow

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* increase batch

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* revert time

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* set last_compacted_idx for split region

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* log large range gc task

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* address comment

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

* close tikv#11404

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

Co-authored-by: 5kbpers <tangminghua@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#21:

Make tikv-server fully generic over kv engines (tikv#10945)

* server: Allow flow_info_receiver to be None

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Move init_flow_receiver call into init_raw_engines

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract debug service creation to CreateKvEngine::start_debug_service

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract backup service creation to CreateKvEngine::start_backup_service

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract rocks cfg controller registration to CreateKvEngine::register_kv_config

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract create_raftstore_compaction_listener to free function

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract create_kv_engine to CreateKvEngine trait

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Parameterize EngineResourceInfo over EK

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Parameterize TikvServer over EK

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Document CreateKvEngine trait

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* clippy

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract raft engine creation to CreateRaftEngine::create_raft_engine

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract CreateRaftEngine::register_raft_config method

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract CreateRaftEngine::create_engine_infos method

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Unspecialize init_raw_engines method

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* server: Extract setup code to generic run_server function

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* clippy

ref tikv#6402

Signed-off-by: Brian Anderson <andersrb@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#22:

Revert "Make tikv-server fully generic over kv engines (tikv#10945)" (tikv#11462)

This reverts commit df2e921.

Ref tikv#11461.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
# This is the commit message tikv#23:

raftstore: flush waterfall metrics when using synchronous write (tikv#11451)

Signed-off-by: gengliqi <gengliqiii@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#24:

fix one disk full unstable test case on assert condition. (tikv#11391)

close tikv#11389

Signed-off-by: tier-cap <zhengxiaojin@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#25:

resolved_ts: return early after regions already had quorum (tikv#11352)

* close tikv#11351

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix tests

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* fix tests

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Apply suggestions from code review

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

Co-authored-by: Zixiong Liu <liuzixiong@pingcap.com>

* address comments

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* make clippy happy

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* close tikv#11400

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

Co-authored-by: qupeng <qupeng@pingcap.com>
Co-authored-by: Zixiong Liu <liuzixiong@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#26:

support to get approximate keys and size for a given range from UserCollectedProperties (tikv#11465)

Signed-off-by: qupeng <qupeng@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#27:

copr: add paging API support for streaming-like processling (tikv#11300)

* copr response with range

Signed-off-by: you06 <you1474600@gmail.com>

* switch to new kvproto

Signed-off-by: you06 <you1474600@gmail.com>

* fix bugs...

Signed-off-by: you06 <you1474600@gmail.com>

* format

Signed-off-by: you06 <you1474600@gmail.com>

* remove unsed field

Signed-off-by: you06 <you1474600@gmail.com>

* add comments

Signed-off-by: you06 <you1474600@gmail.com>

* update kvproto

Signed-off-by: you06 <you1474600@gmail.com>

* fix bench

Signed-off-by: you06 <you1474600@gmail.com>

* update cargo.lock

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

* add test for copr paging

Signed-off-by: you06 <you1474600@gmail.com>

* close tikv#11448

Signed-off-by: you06 <you1474600@gmail.com>

* Update components/tidb_query_executors/src/runner.rs

Co-authored-by: Lei Zhao <zlwgx1023@gmail.com>
Signed-off-by: you06 <you1474600@gmail.com>

* check range in response

Signed-off-by: you06 <you1474600@gmail.com>

* format code

Signed-off-by: you06 <you1474600@gmail.com>

* add test for reverse scan

Signed-off-by: you06 <you1474600@gmail.com>

* add test for multi ranges

Signed-off-by: you06 <you1474600@gmail.com>

* test drained with multi ranges

Signed-off-by: you06 <you1474600@gmail.com>

* fix clippy

Signed-off-by: you06 <you1474600@gmail.com>

* fix lint

Signed-off-by: you06 <you1474600@gmail.com>

* paging by all record count

Signed-off-by: you06 <you1474600@gmail.com>

* format

Signed-off-by: you06 <you1474600@gmail.com>

Co-authored-by: Lei Zhao <zlwgx1023@gmail.com>
# This is the commit message tikv#28:

encryption: support seek for encryption writer (tikv#10929)

* support seek for encryption writer

Signed-off-by: tabokie <xy.tao@outlook.com>

* minor fix, ref: tikv#11119

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix error from merge

Signed-off-by: tabokie <xy.tao@outlook.com>

* fight format checker, ref tikv#11119

Signed-off-by: tabokie <xy.tao@outlook.com>

* add wrappers

Signed-off-by: tabokie <xy.tao@outlook.com>

* remove pub quantifiers and fix lint

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix name export

Signed-off-by: tabokie <xy.tao@outlook.com>

* revert naming changes

Signed-off-by: tabokie <xy.tao@outlook.com>

* address comment on unit test

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix test and inline wrapper calls

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix comments

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#29:

server: remove register PD process (tikv#10985)

* remove register PD process

Signed-off-by: Ryan Leung <rleungx@gmail.com>

* resolve conflicts

close tikv#11472

Signed-off-by: Ryan Leung <rleungx@gmail.com>
# This is the commit message tikv#30:

engine: provide customized file builder to raft engine (tikv#10937)

* open raft engine with customized file builder

ref tikv#11119

Signed-off-by: tabokie <xy.tao@outlook.com>

* overhaul configurations

Signed-off-by: tabokie <xy.tao@outlook.com>

* update default config and address comment

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#31:

*: update some dependencies (tikv#11458)

close tikv#10997, ref tikv#11276

Signed-off-by: tabokie <xy.tao@outlook.com>
# This is the commit message tikv#32:

coprocessor: add `to_binary` and `from_binary` function (tikv#11397)

Signed-off-by: xiongjiwei <xiongjiwei1996@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#33:

server: fix IOType configure for recving snapshot (tikv#11122)

* fix IOType for recving snapshot

Signed-off-by: MuZhou233 <muzhou233@outlook.com>

* Set io_type before Write::write_all. close tikv#10659

Signed-off-by: MuZhou233 <MuZhou233@outlook.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#34:

raftstore: add pessimistic lock table to peer (tikv#11459)

ref tikv#11452

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#35:

engine: fix backward compatibility issue of raft-engine.recovery-mode (tikv#11490)

* add test case

Signed-off-by: tabokie <xy.tao@outlook.com>

* update raft-engine, fix tikv#11489

Signed-off-by: tabokie <xy.tao@outlook.com>

* handle timeout error

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#36:

metrics: add duration composition panel (tikv#11491)

* metrics: add duration composition pannel

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>

* close tikv#11492

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#37:

*: support read-through lock (tikv#11238)

* storage: get/batch_get support read-through lock

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* storage: scan support read-through lock

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* coprocessor: support read-through lock

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* don't check access_locks when checking memory locks

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* fix clippy
close tikv#11402

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* update kvproto and add some comments

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* test desc scan

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#38:

storage: support raw value encode for API V2 (tikv#11139)

* implement api v2 raw value

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* storage: support raw value encode for API V2

ref tikv#10938 close tikv#11041
Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix clippy

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix clippy

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* remove println

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* add test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* disable test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test again

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test again

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test again

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix overflow

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix overflow

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve comment

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* add case

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve comment

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix clippy

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* improve

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

* fix test

Signed-off-by: Andy Lok <andylokandy@hotmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#39:

*: replace snapshot `CbContext` with `SnapshotExt` (tikv#11471)

ref tikv#11452

Signed-off-by: Yilin Chen <sticnarf@gmail.com>
# This is the commit message tikv#40:

Add comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

# This is the commit message tikv#41:

Extract one stats struct for KvGet/KvBatchGet

Signed-off-by: mornyx <mornyx.z@gmail.com>

# This is the commit message tikv#42:

raftstore: relax merge result check (tikv#11478)

* add test case

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* raftstore: relax merge result check

Close tikv#11475.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* fix unstable transfer leader case

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
# This is the commit message tikv#43:

engine: move raft engine purge job to a dedicated thread (tikv#11497)

* update raft engine and implement batch_gc, ref tikv#11119

Signed-off-by: tabokie <xy.tao@outlook.com>

* introduce purge-worker

Signed-off-by: tabokie <xy.tao@outlook.com>

* clean up old worker

Signed-off-by: tabokie <xy.tao@outlook.com>

* update raft engine

Signed-off-by: tabokie <xy.tao@outlook.com>
# This is the commit message tikv#44:

Update max_ts for Prewrite and AcquirePessimisticLock with should_not_exist flag. (tikv#11488)

* update max_ts for prewrite with should_not_exist flag

Signed-off-by: dwangxxx <dwang_slam@163.com>

* close tikv#11315

Signed-off-by: dwangxxx <dwang_slam@163.com>

* close tikv#11315

Signed-off-by: dwangxxx <dwang_slam@163.com>

* fix unit test

Signed-off-by: dwangxxx <dwang_slam@163.com>

* fix format

Signed-off-by: dwangxxx <dwang_slam@163.com>

* fix test

Signed-off-by: dwangxxx <dwang_slam@163.com>

* add test.

Signed-off-by: dwangxxx <dwang_slam@163.com>

* fix clippy error

Signed-off-by: dwangxxx <dwang_slam@163.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#45:

Rename KvGetLatencyStats

Signed-off-by: mornyx <mornyx.z@gmail.com>

# This is the commit message tikv#46:

Rename

Signed-off-by: mornyx <mornyx.z@gmail.com>

# This is the commit message tikv#47:

*: simplify pull request template (tikv#11504)

We only needs to explain what's change. Problem description should be
traced as issue.

This PR also polishes contributing guide to explain the new flow.

Close tikv#11503.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#48:

cdc: introduce TsFilter into incremental scan (tikv#11385)

* close tikv#11384

Signed-off-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#49:

metric: add stale read count metric (tikv#11051)

Signed-off-by: linning <linningde25@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#50:

storage:change configuration storage.max-key-size default value from 4096 to 8192 (tikv#11331)

close tikv#11241

Signed-off-by: TonsnakeLin <lpbgytong@163.com>
# This is the commit message tikv#51:

storage: make it cheaper to clone Scheduler (tikv#11501)

closes tikv#11500

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#52:

raftclient: count term and index in estimated size (tikv#11493)

* add test case

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* count term and index tag

If there are many entries in a message, the estimated size of message
can be way smaller than the actual size. This PR fixes the error by
also counting index and term in estimation.

It also remove the hard limit as the estimation is closed enough.

Close tikv#9714.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#53:

scanner/forward: move to next key if accesses a lock with delete type (tikv#11544)

close tikv#11541

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
# This is the commit message tikv#54:

*: add configuration for in-memory pessimistic lock (tikv#11481)

ref tikv#11452

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
# This is the commit message tikv#55:

backup: fix background panic in `endpoint::tests::test_adjust_thread_pool_size` (tikv#11556)

* backup: fix tikv#11548

Signed-off-by: yujuncen <yujuncen@pingcap.com>
# This is the commit message tikv#56:

tikv_util: reimplement poll_future_notify (tikv#11550)

The current implementation is very easy to get deadlock if the future is
woken immediately during polling. This PR resolves the issue by
maintaining states instead.

Note, if the future is implemented in the wrong way that always
notifying the future without checking if it has readiness, the new
implementation can run into dead loop.

Close tikv#11549.

Co-authored-by: qupeng <qupeng@pingcap.com>
# This is the commit message tikv#57:

fix taking rootfs as filesystem by mistake (tikv#11554)

* fix adding rootfs as filesystem by mistake

Signed-off-by: onlyacat <huangzhe8263@gmail.com>

* reformat by lints

Signed-off-by: onlyacat <huangzhe8263@gmail.com>

* close tikv#11552

Signed-off-by: onlyacat <huangzhe8263@gmail.com>
# This is the commit message tikv#58:

raftstore: set term when catch up logs for merge  (tikv#11545)

* add test case

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* raftstore: set term when catch up logs for merge

Prepare merge may includes logs from different terms, which may be even
larger than the term of a follower that is lag behind. So when catching
up logs by commit merge, its term should also be set to get a correct
metadata.

Close tikv#11526.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* add log for becoming follower

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* correct commit log check

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
  • Loading branch information
mornyx and ti-chi-bot committed Dec 7, 2021
1 parent 59d15cc commit 3c5ae18
Show file tree
Hide file tree
Showing 219 changed files with 16,081 additions and 8,243 deletions.
11 changes: 3 additions & 8 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,9 @@ PR Title Format:
2. *: what's changed
-->

### What problem does this PR solve?

Issue Number: close #xxx <!-- REMOVE this line if no issue to close -->

Problem Summary:

### What is changed and how it works?

Proposal: [xxx](url) <!-- REMOVE this line if not applicable -->
Close #xxx <!-- Associate issue that describes the problem the PR tries to solve. -->

What's Changed:

Expand Down Expand Up @@ -49,4 +43,5 @@ Side effects
```release-note
Please add a release note.
If you don't think this PR needs a release note then fill it with None.
```
If this PR will be picked to release branch, then a release note is probably required.
```
26 changes: 14 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ To build TiKV you'll need to at least have the following installed:
* `awk` - Pattern scanning/processing language
* C++ compiler - gcc 4.9+ (required for gRPC)

If you are targeting platforms other than x86_64 linux, you'll also need:
If you are targeting platforms other than x86_64/aarch64 linux, you'll also need:

* [`llvm` and `clang`](http://releases.llvm.org/download.html) - Used to generate bindings for different platforms and build native libraries (required for grpcio, rocksdb).

Expand Down Expand Up @@ -60,13 +60,6 @@ During interactive development, you may prefer using `cargo check`, which will p
cargo check --all
```

It is particularly handy alongside `cargo-watch`, which runs a command each time you change a file.

```bash
cargo install cargo-watch
cargo watch -s "cargo check --all"
```

When you're ready to test out your changes, use the `dev` task. It will format your codebase, build with `clippy` enabled, and run tests. This should run without failure before you create a PR. Unfortunately, some tests will fail intermittently and others don't pass on all platforms. If you're unsure, just ask!

```bash
Expand All @@ -80,13 +73,15 @@ You can run the test suite alone, or just run a specific test:
make test
# Run a specific test
./scripts/test $TESTNAME -- --nocapture
# Or using make
env EXTRA_CARGO_ARGS=$TESTNAME make test
```

TiKV follows the Rust community coding style. We use Rustfmt and [Clippy](https://github.com/Manishearth/rust-clippy) to automatically format and lint our code. Using these tools is checked in our CI. These are as part of `make dev`, you can also run them alone:

```bash
# Run Rustfmt
cargo fmt
make format
# Run Clippy (note that some lints are ignored, so `cargo clippy` will give many false positives)
make clippy
```
Expand Down Expand Up @@ -120,14 +115,17 @@ Read our configuration guide to learn about various [configuration options](http

This is a rough outline of what a contributor's workflow looks like:

- Make sure what you want to contribute is already traced as an issue.
* We may discuss the problem and solution in the issue.
- Create a Git branch from where you want to base your work. This is usually master.
- Write code, add test cases, and commit your work (see below for message format).
- Run tests and make sure all tests pass.
- Push your changes to a branch in your fork of the repository and submit a pull request.
- Your PR will be reviewed by two maintainers, who may request some changes.
* Make sure mention the issue, which is created at step 1, in the commit meesage.
- Your PR will be reviewed and may be requested some changes.
* Once you've made changes, your PR must be re-reviewed and approved.
* If the PR becomes out of date, you can use GitHub's 'update branch' button.
* If there are conflicts, you can rebase (or merge) and resolve them locally. Then force push to your PR branch.
* If there are conflicts, you can merge and resolve them locally. Then push to your PR branch.
You do not need to get re-review just for resolving conflicts, but you should request re-review if there are significant changes.
- Our CI system automatically tests all pull requests.
- Our [bot](https://github.com/ti-chi-bot) will merge your PR. It can be summoned by commenting `/merge` (requires tests to pass and two approvals. You might have to ask your reviewer to do this).
Expand Down Expand Up @@ -162,6 +160,8 @@ the body of the commit should describe the why.
engine/raftkv: add comment for variable declaration.
Improve documentation.
Close #1234.
```

The format can be described more formally as follows:
Expand All @@ -171,6 +171,8 @@ The format can be described more formally as follows:
<BLANK LINE>
<why this change was made>
<BLANK LINE>
<issue reference>
<BLANK LINE>
Signed-off-by: <Name> <email address>
```

Expand All @@ -180,7 +182,7 @@ If the change affects more than one subsystem, you can use comma to separate the

If the change affects many subsystems, you can use ```*``` instead, like ```*:```.

The body of the commit message should describe why the change was made and at a high level, how the code works.
The body of the commit message should describe why the change was made and at a high level, how the code works. It should also reference the issue for problem context.

### Signing off the Commit

Expand Down
91 changes: 65 additions & 26 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 3c5ae18

Please sign in to comment.