Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added SharedReplacingMergeTree support #582

Open
wants to merge 7 commits into
base: 2.2.0
Choose a base branch
from

Conversation

vpol
Copy link

@vpol vpol commented May 9, 2024

Tables created by clickhouse.cloud have SharedReplacingMergeTree engine, which currently is not handled correctly.

@aadant aadant requested a review from subkanthi May 9, 2024 16:29
@subkanthi subkanthi added this to the 2.2.0 milestone May 9, 2024
@subkanthi
Copy link
Collaborator

Tables created by clickhouse.cloud have SharedReplacingMergeTree engine, which currently is not handled correctly.

Hi @vpol , thanks for your contribution, we have added it to 2.2.0 release, it will be great to have integration tests for this similar to the one for ReplicatedReplacingMergeTree - https://github.com/Altinity/clickhouse-sink-connector/blob/develop/sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ReplicatedRMTIT.java

@vpol
Copy link
Author

vpol commented May 9, 2024

Hi. Thank you for reply.

After some quick research I discovered that SharedMergeTree (and SharedReplacingMergeTree) is only available in clickhouse.cloud, not sure how I can write tests in this case.

From:

https://clickhouse.com/docs/en/cloud/reference/shared-merge-tree

SharedMergeTree Table Engine

  • Available exclusively in ClickHouse Cloud (and first party partner cloud services)

The SharedMergeTree table engine family is a cloud-native replacement of the ReplicatedMergeTree engines that is optimized to work on top of shared storage (e.g. Amazon S3, Google Cloud Storage, MinIO, Azure Blob Storage). There is a SharedMergeTree analog for every specific MergeTree engine type, i.e. ReplacingSharedMergeTree replaces ReplacingReplicatedMergeTree.

The SharedMergeTree table engine family powers ClickHouse Cloud. For an end-user, nothing needs to be changed to start using the SharedMergeTree engine family instead of the ReplicatedMergeTree based engines. It provides the following additional benefits:

From what I understand it works exactly the same way as ReplicatedReplacingMergeTree, and was not supported purely because it has a different name and is not present in enum's.

@subkanthi subkanthi changed the base branch from develop to 2.2.0 May 12, 2024 01:54
@aadant
Copy link
Collaborator

aadant commented May 21, 2024

@vpol out of curiosity, I thought ClickHouse cloud would automatically turn the MergeTree tables of any kind into their Shared version.

@vpol
Copy link
Author

vpol commented May 21, 2024

it does, but it looks like it's turning it into a different "proprietary" table engine, with different name. name that is not correctly handled by the current code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants