cdc: Unstable unit test clientSuite.TestConcurrentProcessRangeRequest #179
Description
Bug Report
1. Describe the bug
CI link: https://github.com/tikv/migration/runs/7512462840?check_suite_focus=true
Logs
=== Failed
=== FAIL: cdc/kv Test (276.63s)
Error: 7/26 02:42:20.489 +00:00] [ERROR] [client.go:1041] ["tikv reported the request cluster ID mismatch error, which is not expected"] [tikvCurrentClusterID=0] [requestClusterID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1041\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:574\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:42:21.384 +00:00] [ERROR] [region_worker.go:686] ["The CommitTs must be greater than the resolvedTs"] ["Event Type"=COMMIT] [CommitTs=98] [resolvedTs=100] [regionID=3] [stack="github.com/tikv/migration/cdc/cdc/kv.(*regionWorker).handleEventEntry\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:686\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).processEvent\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:386\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).eventHandler\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:478\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).run.func3\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:601\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:42:22.341 +00:00] [ERROR] [client.go:1036] ["tikv reported compatibility error, which is not expected"] [rpcCtx="region ID: 3, meta: id:3 region_epoch:<> peers:<id:4 store_id:1 > , peer: id:4 store_id:1 , addr: 127.0.0.1:45399, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv"] [error="required_version:\"v4.0.7\" "] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1036\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:574\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
----------------------------------------------------------------------
FAIL: client_test.go:[307](https://github.com/tikv/migration/runs/7512462840?check_suite_focus=true#step:4:308)1: clientSuite.TestConcurrentProcessRangeRequest
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:[318](https://github.com/tikv/migration/runs/7512462840?check_suite_focus=true#step:4:319)4:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
client_test.go:3184:
c.Errorf("no more events received")
... Error: no more events received
Error: 7/26 02:44:38.627 +00:00] [ERROR] [client.go:1032] ["tikv reported duplicated request to the same region, which is not expected"] [regionID=3] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1032\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:574\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:44:48.628 +00:00] [ERROR] [client.go:365] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:363\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:365\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:44:48.738 +00:00] [ERROR] [client.go:365] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.1.0 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.1.0 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:363\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:365\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:44:48.841 +00:00] [ERROR] [client.go:365] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 2.1.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 2.1.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:363\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:365\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:44:48.944 +00:00] [ERROR] [client.go:365] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:363\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:365\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:351\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:747\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:515\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f/errgroup/errgroup.go:75"]
Error: 7/26 02:44:50.581 +00:00] [ERROR] [client_test.go:1714] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestIncompatibleTiKV.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1714\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:00.455 +00:00] [ERROR] [client_test.go:2939] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).testKVClientForceReconnect.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:2939\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:01.221 +00:00] [ERROR] [client_test.go:2996] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).testKVClientForceReconnect.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:2996\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:04.415 +00:00] [ERROR] [client_test.go:3423] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestPrewriteNotMatchError.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3423\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:05.506 +00:00] [ERROR] [client_test.go:3509] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestPrewriteNotMatchError.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3509\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:17.478 +00:00] [ERROR] [client_test.go:3328] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestRegionWorkerExitWhenIsIdle.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3328\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:17.483 +00:00] [ERROR] [client_test.go:3328] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestRegionWorkerExitWhenIsIdle.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3328\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:36.062 +00:00] [ERROR] [client_test.go:1509] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamRecvWithErrorAndResolvedGoBack.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1509\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:36.232 +00:00] [ERROR] [client_test.go:1509] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamRecvWithErrorAndResolvedGoBack.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1509\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 7/26 02:46:42.236 +00:00] [ERROR] [client_test.go:1[320](https://github.com/tikv/migration/runs/7512462840?check_suite_focus=true#step:4:321)] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamSendWithError.func3\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1320\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
OOPS: 47 passed, 1 skipped, 1 FAILED
DONE 227 tests, 1 failure in 565.011s
make: *** [Makefile:119: unit_test_in_verify_ci] Error 1
Error: Process completed with exit code 2.
2. Minimal reproduce step (Required)
3. What did you see instead (Required)
4. What did you expect to see? (Required)
5. What is your migration tool and TiKV version? (Required)
- TiKV:
- TiKV CDC:
- TiKV BR:
- TiKV Online Bulk Load: