[bug][tidb-connector]checkpoint is not updated long after a task has been running #1686
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wangxiaojing
changed the title
wangxiaojing
changed the title
|
@wangxiaojing Thanks for your contribution ,can you add unit test to make sure it's worked. |
GOODBOY008
approved these changes
Nov 3, 2022
|
This pr will hang until master |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the bug(Please use English)
Environment :
Flink version : 1.14
Flink CDC version: 2.3
Database and version: 5.1
To Reproduce
Steps to reproduce the behavior:
The error :
``2022-10-27 12:48:34,981 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953707961122840, regionId: 1721083
2022-10-27 12:48:35,469 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953708092194817, regionId: 10236245
2022-10-27 12:48:35,469 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953708092194817, regionId: 10237829
2022-10-27 12:48:35,469 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953708092194817, regionId: 8419115
2022-10-27 12:48:35,477 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953708092194820, regionId: 7318373
2022-10-27 12:48:35,477 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953708092194820, regionId: 11085331
2022-10-27 12:48:35,631 INFO com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction [] - snapshotState checkpoint: 9 at resolvedTs: 436952756312866827
2022-10-27 12:58:35,594 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953865404809241, regionId: 8684959 2022-10-27 12:58:35,594 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953865404809241, regionId: 8309437 2022-10-27 12:58:35,594 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953865404809241, regionId: 1721083 2022-10-27 12:58:35,594 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436953865404809241, regionId: 11090547 2022-10-27 12:58:35,635 INFO com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction [] - snapshotState checkpoint: 10 at resolvedTs: 4369527563128668272022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 8424191 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 8343109 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 3287795 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 8221803 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 11120919 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 10296761 2022-10-2 7 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 8309437 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 8684959 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 11090547 2022-10-27 13:08:35,197 INFO org.tikv.cdc.CDCClient [] - handle resolvedTs: 436954022586351623, regionId: 1721083 2022-10-27 13:08:35,632 INFO com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction [] - snapshotState checkpoint: 11 at resolvedTs: 436952756312866827
Additional Description
update the resolvedTs from the max ResolvedTs