[Feature] Looking to increase kafa support Routine Load kafka_topic_list support #30110
Open
2 of 3 tasks
Labels
kind/feature
Categorizes issue or PR as related to a new feature.
Search before asking
Description
Routine Load 导入Kafka中的数据功能,现在是只支持1个topic。
对于分库分表场景,dts订阅会有许多topic,创建许多个Routine Load可能导致写入频次过高影响效率,配置也比较麻烦。
参考 ClickHouse 的Kafka表引擎,它是支持配置多个topic的,并行导入速度是非常快的。
Use case
Kafka引擎表配置
kafka_topic_list = 'APP_LOG',
kafka_group_name = 'toClickhouse',
kafka_format = 'JSONAsString',
kafka_num_consumers = 16,
kafka_thread_per_consumer = 1;
Related issues
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: