Skip to content

Test cloud 5.3#3

Open
bufferflies wants to merge 35 commits into
bufferflies:masterfrom
BusyJay:test-cloud-5.3
Open

Test cloud 5.3#3
bufferflies wants to merge 35 commits into
bufferflies:masterfrom
BusyJay:test-cloud-5.3

Conversation

@bufferflies
Copy link
Copy Markdown
Owner

@bufferflies bufferflies commented Oct 19, 2022

What is changed and how it works?

Issue Number: Close #xxx

What's Changed:

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note 

Please add a release note.

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with None.
If this PR will be picked to release branch, then a release note is probably required.

tabokie and others added 30 commits November 5, 2021 12:55
@tabokie
engine_rocks: switch rust-rocksdb to branch tikv-5.2 (tikv#11246)
304c278 
Signed-off-by: tabokie <xy.tao@outlook.com>
@Little-Wallace
import: remove verify checksum in apply thread (tikv#10950) (tikv#11260)
b1f4852 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

Co-authored-by: Wallace <bupt2013211450@gmail.com>
@tier-cap @ti-chi-bot
cherry pick tikv#10833 to release-5.3 (tikv#11240)
f98a247 
close tikv#10960

Co-authored-by: tier-cap <zhengxiaojin@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@Little-Wallace @ti-chi-bot
rocksdb: fix panic caused by ingest multiple files (tikv#10948) (tikv…
0a8aa2d 
…#11255)

* cherry pick tikv#10948 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

* close tikv#10947

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

Co-authored-by: Wallace <bupt2013211450@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@gengliqi
raftstore: call proposed callback in advance (tikv#11147) (tikv#11275)
def151c 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
@YangKeao @ti-chi-bot
status_server: add glibc, pthread, libgcc to the blocklist (tikv#11205)…
099ae16 
… (tikv#11215)

close tikv#11108

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@MrCroxx @ti-chi-bot
raftstore: refine raft log gc tick. (tikv#10599) (tikv#11278)
fd66095 
* cherry pick tikv#10599 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

* skip sync when no raft log gc task ref tikv#10599

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* ref tikv#10540

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

Co-authored-by: Croxx <mrcroxx@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@Connor1996 @ti-chi-bot
gc: Fix GcKeys task doesn't work when called with multiple keys (tikv…
f264d5b 
…#11248) (tikv#11286)

* cherry pick tikv#11248 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* raftstore: Fix compaction guard panic on non-data key (tikv#11294)

* close tikv#11290 fix compaction guard panic on non-data key

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* add comment

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Co-authored-by: Connor <zbk602423539@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
tests: speed up tests with batch put and mem disk (tikv#11249) (tikv#…
99c7216 
…11271)

* cherry pick tikv#11249 to release-5.3

There are many heavy write in raftstore tests, I have seen some write IO
task takes several seconds to finished, which will certainly make most
test cases timeout. This PR utilizes memory disk to speed up those
operations.

Ref tikv#11177.
Ref tikv#5637.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
tests: fix slow snapshot tests and failed split operation (tikv#11212) (
500ead0 
tikv#11237)

* cherry pick tikv#11212 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

* Ref tikv#11186 and tikv#11182

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Jay <BusyJay@users.noreply.github.com>
Co-authored-by: Jay Lee <BusyJayLee@gmail.com>
@gengliqi @ti-chi-bot
raftstore: reduce some cpu waste of apply thread (tikv#11252) (tikv#1…
97c8754 
…1298)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@hicqu
cdc: remove hint_min_ts for cdc incremental scanner (tikv#11314) (tik…
d628f0a 
…v#11318)

close tikv#11313

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>

Co-authored-by: qupeng <qupeng@pingcap.com>
@lzmhhh123
cherry pick tikv#11302 to release-5.3 (tikv#11336)
e16bf9c 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Zhuomin Liu <lzmhhh123@gmail.com>
@gengliqi @ti-chi-bot
raftstore: reduce the frequency of flushing raft messages (tikv#11310) (
b02d159 
tikv#11323)

* cherry pick tikv#11310 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

* fix test & close tikv#11309

Signed-off-by: gengliqi <gengliqiii@gmail.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@MyonKeminta @ti-chi-bot
txn: Check constraint when retrying prewrite of non-pessimistic-locke…
2784a88 
…d keys in pessimistic transactions (tikv#11264) (tikv#11293)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@gengliqi
tikv_util: support microsecond unit for ReadableDuration (tikv#11338) (…
46b3921 
…tikv#11361)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
@gengliqi
raftstore: batch apply msg to reduce apply state keys (tikv#11304) (t…
795e9be 
…ikv#11362)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
@bufferflies @lhy1024
@zhouqiang-cl @ti-chi-bot
server: deregister should be call if stream revc error after inject n…
e6081e3 
…et error (tikv#11342) (tikv#11355)

* cherry pick tikv#11342 to release-5.3

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

* ref tikv#11365

Signed-off-by: lhy1024 <admin@liudos.us>

* temp using v5.2.3

Signed-off-by: Qiang Zhou <zhouqiang.cl@gmail.com>

* Update Cargo.toml

Signed-off-by: nolouch <nolouch@gmail.com>

Co-authored-by: buffer <doufuxiaowangzi@gmail.com>
Co-authored-by: lhy1024 <admin@liudos.us>
Co-authored-by: Qiang Zhou <zhouqiang.cl@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@gengliqi
cherry pick tikv#11401 to release-5.3 (tikv#11406)
62b92fa 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
@Connor1996
gc: Fix GC scan effectiveness to avoid OOM (tikv#11416) (tikv#11422)
5829245 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Connor <zbk602423539@gmail.com>
@Connor1996
gc: set pending capacity for gc worker(tikv#11429) (tikv#11430)
c20024a 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@gengliqi
cherry pick tikv#11418 to release-5.3 (tikv#11432)
b6cad4e 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
@zhouqiang-cl
Bump v5.3.0
6c14247 
Signed-off-by: Qiang Zhou <zhouqiang.cl@gmail.com>
@BusyJay
update kvproto
e5e445f 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
pick engine trait changes
fded947 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
pick raftstore changes
341dcdc 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
pick tikv change
07a4034 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
pick all rest
4546642 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
raftclient: delay flush
4cf9e24 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
remove raftstore wait
68a2f0c 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
support new flush algorithm
b3816d2 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
disable stats history
fd4fe17 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
allow customize atomic flush
89bec65 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
support blob request
019ed21 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@BusyJay
remove assert for singleton
cde68c1 
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
bufferflies pushed a commit that referenced this pull request Sep 19, 2025
@youjiali1995
Roll up cloud engine (#3)
271e0d4 
* rfengine: introduce DmaBuffer for direct I/O (tikv#12611)

* rfengine: introduce DmaBuffer

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* rotate before write

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

* rfengine: add missing sync_dir (tikv#12628)

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@bufferflies @tabokie @ti-srebot @YangKeao @Connor1996 @zhouqiang-cl @BusyJay