Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CH replicas always fetch partition with ATTACH PARTITION #5312

Closed
den-crane opened this issue May 16, 2019 · 4 comments · Fixed by #7901
Closed

CH replicas always fetch partition with ATTACH PARTITION #5312

den-crane opened this issue May 16, 2019 · 4 comments · Fixed by #7901
Assignees
@den-crane
Labels
bug Confirmed user-visible misbehaviour in official release comp-documentation Documentation operations

Comments

@den-crane
Copy link
Contributor

den-crane commented May 16, 2019 •
edited

https://clickhouse.yandex/docs/en/query_language/alter/#alter_attach-partition
Documentation states: This query is replicated.
Each replica checks whether there is data in the detached directory.
If the data is in this directory, the query checks the integrity, verifies that it matches the
data on the server that initiated the query. If everything is correct, the query adds data
to the replica. If not, it downloads data from the query requestor replica, or from another replica where the data has already been added.

This is either a documentation bug or a server bug.

create table XXXX on cluster XXXX (A Int64)
ENGINE = ReplicatedMergeTree ('/clickhouse/{cluster}/tables/XXXX','{replica}')
order by tuple();
insert into XXXX values(1);

alter table XXXX detach partition tuple();
ls -1 /var/lib/clickhouse/data/dw/XXXX/detached/
all_0_0_0

alter table XXXX attach partition tuple();
ls -1 /var/lib/clickhouse/data/dw/XXXX/
all_2_2_0

REPLICA
ls -1 /var/lib/clickhouse/data/dw/XXXX/detached/
all_0_0_0

grep XXXX /var/log/clickhouse-server/clickhouse-server.log

<Debug> dw.XXXX (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002
<Debug> dw.XXXX (ReplicatedMergeTreeQueue): Pulled 1 entries to queue.
<Debug> dw.XXXX (StorageReplicatedMergeTree): Fetching part all_2_2_0 from /clickhouse/oCHDWcsp1/tables/XXXX/replicas/MASTER
<Trace> ReadWriteBufferFromHTTP: Sending request to http://chdw1-1.MASTER.mycmdb.net:9009/?endpoint=DataPartsExchange%3A%2Fclickhouse%2FoCHDWcsp1%2Ftables%2FXXXX%2Freplicas%2FMASTER&part=all_2_2_0&compress=false
<Trace> dw.XXXX (Data): Renaming temporary part tmp_fetch_all_2_2_0 to all_2_2_0.
<Debug> dw.XXXX (StorageReplicatedMergeTree): Fetched part all_2_2_0 from /clickhouse/oCHDWcsp1/tables/XXXX/replicas/MASTER
@den-crane den-crane added the bug Confirmed user-visible misbehaviour in official release label May 16, 2019
@den-crane
Copy link
Contributor Author

workaround https://t.me/clickhouse_ru/81231

@abraithwaite
Copy link

I believe we might be seeing this issue when restoring from backups.

Is there any plan to address it? Also, Google Translate is not working well for the workaround. Would someone mind translating it for us please?

@den-crane
Copy link
Contributor Author

den-crane commented Oct 28, 2019 •
edited

It's a documentation issue. ATTACH PARTITION works as designed.

I am not sure that workaround is working.

CREATE TABLE dw.test123 on cluster zzzz (x String) ENGINE = ReplicatedMergeTree ('/clickhouse/{cluster}/tables/test','{replica}') ORDER BY x;
insert into test123 values ('x');
alter table test123 detach partition tuple();
  1. create temporary table with the same structure (columns / order by) and attach data
CREATE TABLE dw.test123temp on cluster zzzz (x String) ENGINE = MergeTree  ORDER BY x;

server1
mv /var/lib/clickhouse/data/dw/test123/detached/all_0_0_0 /var/lib/clickhouse/data/dw/test123temp/detached/
alter table test123temp attach  partition tuple();
system stop merges test123temp;

server2
mv /var/lib/clickhouse/data/dw/test123/detached/all_0_0_0 /var/lib/clickhouse/data/dw/test123temp/detached/
alter table test123temp  attach  partition tuple();
system stop merges test123temp;
  1. execute ATTACH PARTITION ... FROM temporary table
server1
alter table test123 ATTACH PARTITION tuple() FROM test123temp

Though maybe it working. No fetch part.


2019.10.28 20:55:40.500690 [ 123 ] {2997a3cc-5273-48cc-a398-cc09659a662d} <Debug> dw.test123: Cloning 1 parts
2019.10.28 20:55:40.529710 [ 123 ] {2997a3cc-5273-48cc-a398-cc09659a662d} <Debug> dw.test123: Cloning part /var/lib/clickhouse/data/dw/test123temp/all_1_1_0/ to /var/lib/clickhouse/data/dw/test123/tmp_replace_from_all_3_3_0
2019.10.28 20:55:40.529974 [ 123 ] {2997a3cc-5273-48cc-a398-cc09659a662d} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Removed 0 entries from queue. Waiting for 0 entries that are currently executing.
2019.10.28 20:55:40.530001 [ 123 ] {2997a3cc-5273-48cc-a398-cc09659a662d} <Trace> dw.test123: Renaming temporary part tmp_replace_from_all_3_3_0 to all_3_3_0.
2019.10.28 20:55:40.539587 [ 33 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002
2019.10.28 20:55:40.552754 [ 33 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Pulled 1 entries to queue.
2019.10.28 20:55:40.552791 [ 17 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Removed 0 entries from queue. Waiting for 0 entries that are currently executing.


2019.10.28 20:55:40.538784 [ 39 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002
2019.10.28 20:55:40.549277 [ 39 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Pulled 1 entries to queue.
2019.10.28 20:55:40.549329 [ 11 ] {} <Debug> dw.test123 (ReplicatedMergeTreeQueue): Removed 0 entries from queue. Waiting for 0 entries that are currently executing.
2019.10.28 20:55:40.549369 [ 11 ] {} <Debug> dw.test123: Found 1 parts that could be cloned (of 1 required parts)
2019.10.28 20:55:40.549400 [ 11 ] {} <Debug> dw.test123: Cloning part /var/lib/clickhouse/data/dw/test123temp/all_1_1_0/ to /var/lib/clickhouse/data/dw/test123/tmp_replace_from_clone_all_3_3_0
2019.10.28 20:55:40.549636 [ 11 ] {} <Trace> dw.test123: Renaming temporary part tmp_replace_from_clone_all_3_3_0 to all_3_3_0.
2019.10.28 20:55:40.564486 [ 28 ] {} <Trace> dw.test123 (ReplicatedMergeTreeCleanupThread): Checking 1 blocks (1 are not cached) to clear old ones from ZooKeeper.

@abraithwaite
Copy link

Good to know! Thanks for sharing!

@den-crane den-crane self-assigned this Nov 18, 2019
den-crane added a commit to den-crane/ClickHouse that referenced this issue Nov 24, 2019
@den-crane
Update alter.md
0646448 
Fix for alter attach alter_attach-partition (fixes ClickHouse#5312)
@den-crane den-crane mentioned this issue Nov 24, 2019
Merged
den-crane added a commit to den-crane/ClickHouse that referenced this issue Nov 27, 2019
@den-crane
Update alter.md
c06496c 
Fix for alter attach alter_attach-partition (fixes ClickHouse#5312). English version.
@den-crane den-crane mentioned this issue Nov 29, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@den-crane den-crane

Labels
bug Confirmed user-visible misbehaviour in official release comp-documentation Documentation operations
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update alter.md den-crane/ClickHouse
3 participants
@filimonov @abraithwaite @den-crane