Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]Clickhouse sink can cause data loss when checkpointing #2174

Closed
2 tasks done
ApocalypseWan opened this issue Jan 17, 2022 · 0 comments · Fixed by #2175
Closed
2 tasks done

[Bug]Clickhouse sink can cause data loss when checkpointing #2174

ApocalypseWan opened this issue Jan 17, 2022 · 0 comments · Fixed by #2175
Labels
component/sort type/bug Something is wrong
Milestone
1.0.0

Comments

@ApocalypseWan
Copy link
Contributor

What happened

Currently, clickhouse sink flush data asynchrony when doing checkpoint, which may cause data loss

What you expected to happen

Flush data synchrony when doing checkpoint in clickhouse sink

How to reproduce

Use inlong-sort to write data to clickhouse, and failover the job over and over.

Environment

No response

InLong version

master

InLong Component

InLong Sort

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/sort type/bug Something is wrong
Projects
None yet
Milestone
1.0.0
2 participants
@dockerzhang @ApocalypseWan