Improve the separation between the Connect and MM2 operators #9955
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of change
Description
The
AbstractConnectOperator
,KafkaConnectAssemblyOperator
andKafkaMirrorMaker2AssemblyOperator
classes are currently badly designed in how they separate the functionality specific to Connect and MM2. TheAbstractConnectOperator
does not really contain only the abstract and shared parts. But it seems to contain also many parts related only to Connect. This applies specifically for the parts related to theKafkaConnector
management and operations ->KafkaConnector
resources are used only with Connect as MM2 has the connector configuration directly inside theKafkaMirrorMaker2
CR. These parts are then later overridden in theKafkaMirrorMaker2AssemblyOperator
. This makes it very hard to see:This PR refactors these classes to improve the separation between them. The
AbstractConnectOperator
now contains only the parts that are really shared and does not contain any parts unique to either Connect or MM2. The other parts were moved to the corresponding assembly operator, in most cases toKafkaConnectAssemblyOperator
. This refactoring should help us to better understand what is shared and what is not shared. And in the future it should allow us to follow up with another refactoring that will try to reuse more code betweenKafkaConnectAssemblyOperator
andKafkaMirrorMaker2AssemblyOperator
=> these classes have now each its own reconciliation flow. But they really differ only in how the connectors are managed. So there should be space to further improve this. But this is not done in this PR to keep the PR scope under review and keep the reviews easier. Where possible, this PR also makes methods private or static.Note: While this PR moves a lot of methods around, it usually only changes their placement, access modifiers etc. It doesn't change any content inside these methods with the exception of few places where very simple methods were inlined.
This contributes to #8158.
Checklist