feat: add clickhouse flusher (#554) #580
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
messixukejia
requested review from
henryzhx8,
yyuuttaaoo and
messixukejia
and removed request for
henryzhx8 and
yyuuttaaoo
henryzhx8
requested changes
Dec 20, 2022
henryzhx8
reviewed
Dec 20, 2022
好好休息，保重身体，祝早日康复。 |
shalousun
reviewed
Dec 21, 2022
kl7sn
force-pushed
the
flusher-clickhouse
branch
from
088612d
to
6fbdf67
Compare
henryzhx8
requested changes
Jan 16, 2023
kl7sn
force-pushed
the
flusher-clickhouse
branch
from
525c622
to
ee12739
Compare
|
我在本地执行
但是该依赖是由 #603 合并到主分支产生的
我这边是有什么操作不对吗？执行 |
kl7sn
force-pushed
the
flusher-clickhouse
branch
from
79f027a
to
7dc4906
Compare
test目录下go mod tidy一下先 |
|
本地运行 当前报错 在增加 converter 之前未出现该报错，现有的 init 方法这边有什么写错的地方吗？ https://github.com/kl7sn/ilogtail/blob/flusher-clickhouse/test/engine/subscriber/clickhouse.go#L185 |
henryzhx8
requested changes
Feb 7, 2023
kl7sn
force-pushed
the
flusher-clickhouse
branch
from
f0c3252
to
7478b85
Compare
henryzhx8
approved these changes
Feb 7, 2023
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
Signed-off-by: kl7sn <mex7.0828@gmail.com>
kl7sn
force-pushed
the
flusher-clickhouse
branch
from
84f4580
to
8bebe12
Compare
yyuuttaaoo
approved these changes
Feb 13, 2023
| // post them to db all at once, build statements | ||
| batch, err := f.conn.PrepareBatch(ctx, sql) | ||
| if err != nil { | ||
| return err |
There was a problem hiding this comment.
是否需要打印error，外层是FLUSH_DATA_ALARM，但如果这里不打日志会缺少具体错误调用点
| } | ||
| } | ||
| // commit and record metrics | ||
| if err = batch.Send(); err != nil { |
| func (f *FlusherClickHouse) BufferFlush(projectName string, logstoreName string, configName string, logGroupList []*protocol.LogGroup) error { | ||
| ctx := context.Background() | ||
| sql := fmt.Sprintf(insertSQL, f.Authentication.PlainText.Database, f.Table) | ||
| for _, logGroup := range logGroupList { |
There was a problem hiding this comment.
如果一部分flush成功一部分失败了，似乎不能处理得很好
messixukejia
approved these changes
Feb 13, 2023
commit 66daff1 Author: Tom Yu <yyuuttaaoo@gmail.com> Date: Mon Feb 13 16:49:29 2023 +0800 Grok processor report match errors by default (alibaba#645) * close alibaba#644, grok report match errors on default meanwhile change pure plugin LogtailSysConfDir to current dir fix plugin version in logs by using ldflag to set go plugin version
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.