Problem
Kafka sink has two validation gaps and several low-quality operational logs:
- After
CreateTopic succeeds, waitUntilTopicVisible treats an empty metadata map with no error as proof that the topic is visible. The sink can proceed before brokers expose the requested topic.
- Kafka sink accepts explicitly configured non-positive
dial-timeout, read-timeout, and write-timeout values.
- Kafka logging contains noisy normal-path messages, duplicated error logs, inconsistent messages and fields, and per-message payload diagnostics. Some integration tests depend on exact log text instead of validating behavior.
Expected behavior
- Topic creation retries until metadata contains the requested topic, and success is logged only after visibility is confirmed.
- Explicit Kafka timeout values must be greater than zero and invalid values return
CDC:ErrKafkaInvalidConfig. Omitted values keep the existing 10-second defaults.
- Kafka sink logs focus on lifecycle transitions, final configuration, and actionable failures:
- normal control-flow and duplicated errors remain quiet;
- producer failures preserve the original Kafka error and event context;
- version detection emits one fallback warning without changing version selection;
- messages, fields, and durations use consistent names and types;
- DDL logs describe the specific child event that was skipped or sent.
- Integration tests validate end-to-end behavior rather than grep implementation-specific Kafka log messages.
Related PR: #5818
Problem
Kafka sink has two validation gaps and several low-quality operational logs:
CreateTopicsucceeds,waitUntilTopicVisibletreats an empty metadata map with no error as proof that the topic is visible. The sink can proceed before brokers expose the requested topic.dial-timeout,read-timeout, andwrite-timeoutvalues.Expected behavior
CDC:ErrKafkaInvalidConfig. Omitted values keep the existing 10-second defaults.Related PR: #5818