Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A lots new features coming #162

Open
imiskolee opened this issue Sep 25, 2019 · 3 comments
Open

A lots new features coming #162

imiskolee opened this issue Sep 25, 2019 · 3 comments

Comments

@imiskolee
Copy link

imiskolee commented Sep 25, 2019

hi,team:

I have finished some features on this,but thier just on on private branch, I want to hear more request and advice from anyone.

1. support checkpoint on table initial load.

1. require Mysql and have sorted primary key.

2. support update event.

1. just translate UPDATE event to INSERT, it's friendly for `ReplacingMergeTree`.
2. using `final` ON `SELECT`.

3. planing to support DELETE event.

1. just translate DELETE Event to INSERT(always using soft delete).
2. require dst clickhouse table has `deleted_at` field.
3. and also increment `version key`, currently just support `DateTime` field.

4. Batch Insert

as well as we know, clickhouse-mysql process each binlog one by one using single insert mode, it's very slow. the feature allow caching binlog event and batch insert.

@sunsingerus
Copy link
Collaborator

sunsingerus commented Sep 26, 2019

Hello Misko!

Thanks a lot! All these changes looks very interesting!
Can you, please, elaborate a little items 1 and 4, to clarify what exactly you propose.
Items 2 and 3 seems to be more/less clear.
Let's talk over all these changes and make plan on how to proceed.

@imiskolee
Copy link
Author

@sunsingerus I will starting create some PR the all thooses next a fews day.

@robertvolkmann
Copy link

@imiskolee Any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants