Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move parts between storage volumes according to configured TTL expressions #8140

Merged
merged 49 commits into from
Dec 11, 2019
Merged

Move parts between storage volumes according to configured TTL expressions #8140

merged 49 commits into from
Dec 11, 2019

Conversation

excitoon
Copy link
Contributor

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (up to few sentences, required except for Non-significant/Documentation categories):

Move parts between storage volumes according to configured TTL expressions.

@excitoon
Parser for extended TTL expressions.
7a37c4f
@excitoon
Fixed ClickHouse after changing syntax.
3e98460
@excitoon
Updated ttl test according to new TTL syntax.
5a12986
@excitoon
First attempt to store min/max for move ttl expressions.
33ded27
@excitoon
Minor fix.
893e0de
@excitoon
Attempt to add background moves by TTL expressions.
575de5a
@excitoon
Fixed 00933 ttl tests.
4cd75f9
@excitoon
Fixed ttl move logic in background move task and added ttl enforcemen…
76c4ac9 
…t on inserts.
@excitoon
Style fix.
dfe0edc
@excitoon
Refactored, added move enforcement on merges, fixed a bug with wrong …
58e824b 
…disk selection in `ReplicatedMergeTree`.
@excitoon
Removed 'DELETE' keyword from default TTL syntax description.
773bf7b
@excitoon
Removed TODO comments.
342ea5c
@excitoon
Minor fix of JSON handling in MergeTreeDataPartTTLInfo.
e519c04
@excitoon
Moved settings for BackgroundProcessingPool to configuration.
3e3e9ac
@excitoon
Correct merge of background move pool.
07c4fa3
@excitoon
Fixed Clang build.
88c7220
@excitoon
Style fixes.
ce066d3
@excitoon
Fixed Clang build one more time.
cc0c216
@excitoon
Renamed TTLDestinationType to PartDestinationType.
62016fe
@excitoon
Removed redundant template from updateTTL() method.
cb777f6
@excitoon
Added first version of tests for extended TTL expressions (still need…
617bf42 
… refactoring).
@excitoon
Fixed typos in test_ttl_move test.
cc70f71
@excitoon
Fixed more typos in test_ttl_move test.
e163ebb
@excitoon
Fixed MergeTreeData::hasTableTTL() method according to extended TTLs.
ce851fc
@excitoon
Fixes of tests for extended TTL syntax.
b3e2618
@excitoon
Added few FIXMEs to not forget.
0da7234
@excitoon
Fixed a bug with erasing columns with same name.
108e2b3
@excitoon
Fixed typos in a test.
b3427af
@excitoon
Refactored tests a little bit.
3250955
@excitoon
Fixed some typos in test_ttl_move test.
ae4b2b4
excitoon and others added 19 commits December 11, 2019 13:24
@excitoon
Placed move TTL rules to a vector.
7407f7c
@excitoon
Added move TTL rules to metadata of ReplicatedMergeTree.
14cf530
@excitoon
Fixed move TTL metadata.
df002f2
@excitoon
Fixed more typos in test_ttl_move.
2e853e3
@excitoon
Fixed test_ttl_move again.
164360e
@excitoon
Fixed move TTL expiration logic.
15fb64f
@excitoon
Finally fixed selection of destination of move TTL.
8ed3780
@excitoon
Removing expired rows shall not be triggered on move TTL expiration.
f72da4a
@excitoon
Don't move anything if part already belongs their destination.
618a39c
@excitoon
Fixed a typo in MergeTreeData::selectMoveDestination().
f83b28a
@excitoon
Finally fixed tests and logic for extended TTL syntax.
67aa268
@excitoon
Fixed Clang build.
e3ed311
@excitoon
Boosted test_ttl_move::test_moves_to_volume_work a little.
065b681
@excitoon
Review fixes.
7fd1668
@excitoon
Added comment to MergeTreePartsMover::selectPartsForMove().
e41deb5
@excitoon
Minor style fix.
f2aee64
@excitoon
Missed review fixes.
0ff0a52
@alesapin @excitoon
Just trigger CI
fd9483e
@alesapin @excitoon
Fix style
2940aef
alesapin
alesapin approved these changes Dec 11, 2019
View reviewed changes
Copy link
Member

@alesapin alesapin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@KochetovNicolai KochetovNicolai merged commit 34ce86a into ClickHouse:master Dec 11, 2019
@KochetovNicolai KochetovNicolai added the pr-feature Pull request with new product feature label Dec 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alesapin alesapin alesapin approved these changes

Assignees
No one assigned
Labels
pr-feature Pull request with new product feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@excitoon @alesapin @filimonov @KochetovNicolai