Repository navigation
🎉 fix-mysql-cdc: poll for 5 minutes only when we have not received a single record - #3789
Merged
Subodh Chaturvedi (subodh1810) merged 3 commits intoJun 2, 2021
Merged
Conversation
auto-assign
Bot
requested review from
Charles (cgardens) and
Jared Rhizor (jrhizor)
June 1, 2021 18:42
Contributor
Author
|
/test connector=source-mysql
|
Sherif A. Nada (sherifnada)
approved these changes
Jun 1, 2021
Sherif A. Nada (sherifnada)
left a comment
Sherif A. Nada (sherifnada)
left a comment
Contributor
There was a problem hiding this comment.
LGTM but one style comment
| * continue to run for 5 minutes | ||
| */ | ||
| static TimeUnit sleepTimeUnit = TimeUnit.MINUTES; | ||
| private static final int SLEEP_TIME_AMOUNT = 5; |
Contributor
There was a problem hiding this comment.
this would 100% trip me up because I would first go: "what is the time unit for this" and then realize it's actually being used as two different constants. Could we instead make it two different constants e.g:
private static final int FIRST_RECORD_WAIT_TIME_MINUTES = 5;
private static final int SUBESQUENT_RECORD_WAIT_TIME_SECONDS = 5;
Contributor
Author
|
/publish connector=connectors/source-mysql
|
Subodh Chaturvedi (subodh1810)
deleted the
add-first-record-logic-mysql-cdc
branch
June 2, 2021 06:25
Abhi Vaidyanatha (avaidyanatha)
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue : #3788
In this PR #3757 we increased the poll duration from 5 seconds to 5 minutes cause 5 seconds was too low and debezium was not returning any records in that duration. This was mainly happening cause the first time the connector runs, it creates a snapshot and the snapshot creation was taking more than 5 seconds.
But now 5 minutes is too much and if a table is receiving continous insert/update/delete events within 5 minutes, the sync process would never finish.
Tested this locally against a MySQL Aurora instance
Pre-merge Checklist
Recommended reading order
test.javacomponent.ts