Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

没有平衡的写入数据 #162

Closed
shouziws opened this issue Sep 21, 2022 · 2 comments
Closed

没有平衡的写入数据 #162

shouziws opened this issue Sep 21, 2022 · 2 comments

Comments

@shouziws
Copy link

{
"clickhouse": {
"hosts": [
[
"10.242.44.95"
],
[
"10.242.168.107"
]
],
"port": 9000,
"db": "default",
"username": "default",
"password": "root",
"retryTimes": 0
},
"kafka": {
"brokers": "192.168.1.4:9092",
"version": "2.5.0"
},
"task": {
"name": "test_auto_schema",
"topic": "topic1",
"consumerGroup": "test_auto_schema",
"earliest": true,
"parser": "json",
"autoSchema": true,
"tableName": "test_auto_schema",
"excludeColumns": [
"day"
],
"bufferSize": 50000
},
"logLevel": "info"
}

上面是我的sinker配置,我手动测试写kafka,他只会向一个host clickhouse服务写数据,没有做到平衡的写数据

@yuzhichang
Copy link
Collaborator

sinker可以平衡写,batch级别的。你kafka里数据不够两个batch,看不出来效果

@shouziws
Copy link
Author

shouziws commented Sep 22, 2022

如作者所说,是batch级别的,是我数据量不够多看不出效果,已测试是平衡写入的,已解决

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants