Skip to content

Conversation

@azat
Copy link
Member

@azat azat commented Sep 27, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Add ALTER TABLE REWRITE PARTS - rewrites the table parts from scratch, by using all new settings (since some, like use_const_adaptive_granularity, will be applied only for new parts).

@clickhouse-gh
Copy link

clickhouse-gh bot commented Sep 27, 2025

Workflow [PR], commit [95a9cd0]

Summary:

job_name test_name status info comment
Stateless tests (amd_binary, ParallelReplicas, s3 storage, parallel) failure
03631_huge_query_filter_analyze FAIL
Stateless tests (amd_debug, AsyncInsert, s3 storage, parallel) failure
02676_analyzer_limit_offset FAIL

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Sep 27, 2025
@azat azat force-pushed the alter-table-materialize branch from 309137a to 8bb4c1a Compare September 27, 2025 20:59
@alexey-milovidov
Copy link
Member

can be useful to apply some settings

This makes it confusing.

For MATERIALIZE INDEX, MATERIALIZE PROJECTION, and similar things, it does the calculation and persists the data if it didn't exist.

But from the description above, you made MATERIALIZE rewrite the table unconditionally.

@alexey-milovidov alexey-milovidov self-assigned this Sep 28, 2025
Copy link
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@azat
Copy link
Member Author

azat commented Sep 28, 2025

Yes, but this is only for MATERIALIZE, MATERIALIZE TTL and MATERIALIZE PROJECTION will still work as before.

Maybe the naming is not good, would appreciate any ideas

@azat
Copy link
Member Author

azat commented Sep 28, 2025

Maybe MATERIALIZE PARTS? (Had this idea at the beginning but decided to go more shorter option)

Or REWRITE, though this looks inconsistently with existing MATERIALIZE keyword to me.

@alexey-milovidov
Copy link
Member

Yes, the word should be different. It should contain the word "parts" to emphasize that the parts remain the same (not being merged) and have some other word than "materialize" (to emphasize that it rewrites parts even if everything is already persisted).

The feature is useful not only for changing index granularity or compress block size, but also - for rotating encryption keys.

@alexey-milovidov
Copy link
Member

Let it be ALTER TABLE REWRITE PARTS

@azat azat force-pushed the alter-table-materialize branch from 8bb4c1a to 26ed327 Compare September 28, 2025 16:25
@azat
Copy link
Member Author

azat commented Sep 28, 2025

Done

@azat azat changed the title Add ALTER TABLE MATERIALIZE Add ALTER TABLE REWRITE PARTS Sep 28, 2025
@azat
Copy link
Member Author

azat commented Sep 29, 2025

test_keeper_memory_soft_limit/test.py::test_soft_limit_create

@azat azat force-pushed the alter-table-materialize branch from 26ed327 to 2308066 Compare September 29, 2025 11:10
This can be useful to apply some settings (i.e.
enable `use_const_adaptive_granularity` once you realized that the size
of granulas array became to big for all parts) that requires rewriting
all parts.
@azat azat force-pushed the alter-table-materialize branch from 2308066 to 67cfed6 Compare September 29, 2025 13:23
@azat
Copy link
Member Author

azat commented Sep 29, 2025

Integration tests (amd_tsan, 3/6) failure

@azat
Copy link
Member Author

azat commented Oct 4, 2025

@alexey-milovidov PTAL

@azat azat enabled auto-merge October 4, 2025 11:17
@azat
Copy link
Member Author

azat commented Oct 4, 2025

CI:

@azat azat added this pull request to the merge queue Oct 4, 2025
Merged via the queue into ClickHouse:master with commit 5ccb2f5 Oct 4, 2025
120 of 123 checks passed
@azat azat deleted the alter-table-materialize branch October 4, 2025 16:34
@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Oct 4, 2025
@azat
Copy link
Member Author

azat commented Nov 2, 2025

@alexey-milovidov FYI I've added more tests here - #89382

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants