Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype lighweight mutation for mergetree #35090

Conversation

zhangjmruc
Copy link
Contributor

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
prototype of implementation of lightweight delete and update for merge tree engine.

Description:
This pull request does similar work as #32774.

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Mar 7, 2022
@nikitamikhaylov nikitamikhaylov added the can be tested Allows running workflows for external contributors label Mar 9, 2022
@antip00
Copy link

antip00 commented Mar 25, 2022

it does not work for me

create table au(id Int64, x Int64) engine=MergeTree order by id

insert into au select number % 10, number % 100 from numbers(10000)

alter table au delete where id>50

if you use alter delete with mutation_sync=1 or mutation_sync=2 then it breaks on alter delete

select count(*) from au

it is stuck for a few mins
then exeption
Exception on client:
Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF)

Connecting to localhost:9000 as user default.
Code: 210. DB::NetException: Connection reset by peer, while reading from socket (127.0.0.1:9000). (NETWORK_ERROR)

@zhangjmruc
Copy link
Contributor Author

zhangjmruc commented Apr 15, 2022

I recreated your issue. Will check further and fix it.
Fixed this issue during rebase.

alter table au delete where id>50

it does not work for me

create table au(id Int64, x Int64) engine=MergeTree order by id

insert into au select number % 10, number % 100 from numbers(10000)

alter table au delete where id>50

if you use alter delete with mutation_sync=1 or mutation_sync=2 then it breaks on alter delete

select count(*) from au

it is stuck for a few mins then exeption Exception on client: Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF)

Connecting to localhost:9000 as user default. Code: 210. DB::NetException: Connection reset by peer, while reading from socket (127.0.0.1:9000). (NETWORK_ERROR)

@zhangjmruc zhangjmruc marked this pull request as ready for review April 21, 2022 01:42
@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch from 78b34c8 to 0d8dc01 Compare April 21, 2022 05:40
@robot-ch-test-poll1 robot-ch-test-poll1 added the submodule changed At least one submodule changed in this PR. label Apr 21, 2022
@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch from d444b86 to b492f29 Compare April 22, 2022 13:24
@zhangjmruc
Copy link
Contributor Author

@nikitamikhaylov The failing check is not related to this request. How to proceed?

@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch 2 times, most recently from b733b61 to c5d4173 Compare April 28, 2022 08:10
@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch from c5d4173 to 671d5cc Compare May 4, 2022 13:26
@zhangjmruc zhangjmruc mentioned this pull request May 5, 2022
@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch 3 times, most recently from 9bb388f to 71da0f5 Compare May 10, 2022 01:17
@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch from 71da0f5 to bdb651d Compare May 26, 2022 03:03
@zhangjmruc
Copy link
Contributor Author

@davenger Are you reviewing this pull request? Please let me know if you have any comment. Thank you for your time

@zhangjmruc zhangjmruc force-pushed the prototype_lighweight_mutation_for_mergetree branch from df19560 to ebcafd5 Compare May 30, 2022 11:07
@zhangjmruc
Copy link
Contributor Author

Will split into lightweight delete and lightweight update.

@zhangjmruc zhangjmruc closed this Jun 6, 2022
@vkrot-innio
Copy link

@zhangjmruc , is there any ongoing progress / plans in lightweight deletes implementation? or is it abandoned?

@zhangjmruc
Copy link
Contributor Author

@vkrot-innio I'm working on the implementation of lightweight delete. Will use this new pull request #37893.
It's more complex for the select query when I tried to add a virtual column _delete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors pr-feature Pull request with new product feature submodule changed At least one submodule changed in this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants