🎉 fix-mysql-cdc: poll for 5 minutes only when we have not received a single record #3789
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/test connector=source-mysql
|
sherifnada
approved these changes
Jun 1, 2021
| * This is not private and final because we need to override in tests otherwise each test would | ||
| * continue to run for 5 minutes | ||
| */ | ||
| static TimeUnit sleepTimeUnit = TimeUnit.MINUTES; | ||
| private static final int SLEEP_TIME_AMOUNT = 5; |
There was a problem hiding this comment.
this would 100% trip me up because I would first go: "what is the time unit for this" and then realize it's actually being used as two different constants. Could we instead make it two different constants e.g:
private static final int FIRST_RECORD_WAIT_TIME_MINUTES = 5;
private static final int SUBESQUENT_RECORD_WAIT_TIME_SECONDS = 5;
|
/publish connector=connectors/source-mysql
|
avaidyanatha
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue : #3788
In this PR #3757 we increased the poll duration from 5 seconds to 5 minutes cause 5 seconds was too low and debezium was not returning any records in that duration. This was mainly happening cause the first time the connector runs, it creates a snapshot and the snapshot creation was taking more than 5 seconds.
But now 5 minutes is too much and if a table is receiving continous insert/update/delete events within 5 minutes, the sync process would never finish.
Tested this locally against a MySQL Aurora instance
Pre-merge Checklist
Recommended reading order
test.javacomponent.ts