New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DBZ-6510 Use event processing failure handling mode in VitessReplicationConnection #138
Conversation
Hi @twthorn, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@twthorn Thanks a lot. I left two comments related to removal of unnecessary code and change in tests. The functionality itself LGTM.
@@ -175,6 +175,15 @@ public static BigIntUnsignedHandlingMode parse(String value, String defaultValue | |||
} | |||
} | |||
|
|||
public EventProcessingFailureHandlingMode getEventProcessingFailureHandlingMode() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be unnecessary as io.debezium.config.CommonConnectorConfig.EVENT_PROCESSING_FAILURE_HANDLING_MODE
declares FAIL
as the default value.
consumedMessages.add(new MessageAndVgtid(message, vgtid)); | ||
}, | ||
error); | ||
// Since we are using the "current" as the starting position, there is a race here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a bit unsafe. Could you please take a look at io.debezium.connector.mysql.StreamingSourceIT.inconsistentSchema(EventProcessingFailureHandlingMode)
to see how to intercept log messages and wait for them?
@jpechane Thank you for the review. I've updated with the feedback to refactor the test to wait on log messages, and to remove the unnecessary config method. |
@twthorn Applied, thanks! |
Summary
Use the event processing failure handling mode for handling errors appropriately in the VitessReplicationConnection. This is similar to what is done for Postgres. We follow the standard behaviors for error/skip/warn/ignore.
Verification
Add a few integration tests for each branch of error/skip/warn/ignore. Check that the error throwable is set/not set appropriately. Use the example use case of an oversized grpc message to be the guaranteed failure (by setting byte max very low).