-
Notifications
You must be signed in to change notification settings - Fork 518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clickhouse: set mutations_sync 2 at delete version to avoid apply down migration twice #454
Conversation
Given the nature of async in ClickHouse, is it normal for ClickHouse migrations to be run one at a time? I originally had some thoughts/questions in this issue, but ideally, we get some input from the community on ClickHouse best practices. |
For our use cases it's ok:
|
Do you think this should be configurable, where the default behavior is the current behavior, but users can configure the ClickHouse |
I think it must be set for goose db version queries only, because nature of that queries is synchronously |
To confirm, we only need to do this for the goose down version, and the goose up version is writing data synchronously by default? EDIT: indeed that appears to be the case, the relevant docs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm.
Hmm, this means we should be able to update the ClickHouse test and remove the goose/tests/clickhouse/clickhouse_test.go Line 16 in 0aa73e3
I added a retry function to ensure the end state of the down-to 0 migration is correct, but with this change, we can remove that and expect the correct behavior because it's happening synchronously. |
3166d4b
to
343ee80
Compare
…n migration twice
343ee80
to
9a8415b
Compare
Thanks! |
I believe yes, because we are using MergeTree engine. For ReplicatedMergeTree or Distributed engines it should to use insert_quorum or insert_distributed_sync |
Got an error for
down-to
command, steps for reproduce:https://pressly.github.io/goose/blog/2022/improving-clickhouse/#getting-started
Expected:
Actual:
Fix: