Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

system.mutations is not responding after I ran an ALTER sql on a table A. Table A is also stuck. I just want to drop Table A. #45736

Open
daggerin3 opened this issue Jan 28, 2023 · 4 comments
Labels
question Question?

Comments

@daggerin3
Copy link

Make sure to check documentation https://clickhouse.com/docs/en/ first. If the question is concise and probably has a short answer, asking it in community Slack is probably the fastest way to find the answer. For more complicated questions, consider asking them on StackOverflow with "clickhouse" tag https://stackoverflow.com/questions/tagged/clickhouse

If you still prefer GitHub issues, remove all this text and ask your question here.
I ran an ALTER sql on Table A. Then table A is stuck, system.mutations is not responding. I also wait 3 hours. I tried below actions,

  1. restart Clickhouse service, reboot this Clickhouse server machine. reboot my client computer.
  2. checked log, and found this, "Current max source part size for mutation is 0 but part size 1513. Will not mutate part all_24_24_0. "
  3. The sql I used is like "alter table A update columnA=columnA/(select sum()....) where ...."
  4. No other data is inserting. No recoreds in system.merges or system.replication_queue.
  5. Table A has around 1000 records

I just want to drop this table A, then I can recreate it with the history records, but table A is not responding. Also system.mutations.

@daggerin3 daggerin3 added the question Question? label Jan 28, 2023
@DerekChia
Copy link
Contributor

I just want to drop this table A

Did you have trouble running DROP TABLE A?

@daggerin3
Copy link
Author

I just want to drop this table A

Did you have trouble running DROP TABLE A?

yes, no response from clickhouse when I ran "alter table A" or "select * from tableA limit 1", even no timeout. Table system.mutations also has no response when I tried "select * from system.mutations limit 1".

I increase the value in settings for background_pool_size(512), parts_to_delay_insert(2400), parts_to_throw_insert(2400). The error log "Current max source part size for mutation is 0 but part size " is gone. From table system.parts, I can see merge is running,

select count() parts_count, database,table,partition from system.parts
where active group by database,table,partition
order by parts_count desc
;

239,system,metric_log,202301
180,system,trace_log,202301
33,system,asynchronous_metric_log,202301
31,system,query_log,202301
31,system,query_thread_log,202301

But still, tableA and system.mutations is locked. Can't do anything now.
server disk/cpu works fine.

Just want to unlocked tableA(no need to restore history data, just drop/remove), and unlock system.mutations since all mutations are not working.

@daggerin3
Copy link
Author

fixed.

tableA and system.mutations are unlocked. I found the file containing my ALTER sql and deleted it.

/var/lib/clickhouse/data/"my schema name"/tableA/mutations_25.txt

@DerekChia
Copy link
Contributor

Looks like the mutation hung. Do check on your columnA that was altered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Question?
Projects
None yet
Development

No branches or pull requests

2 participants