Skip to content

after change storage policy from cold_disk to s3, weird merge #42519

@zdyj3170101136

Description

@zdyj3170101136

version

22.9.2

queation

out old storage policy is hot_to_cold, it store data into disk.
new storage policy is hot_to_cold_s3 it store data into s3.
截屏2022-10-20 上午11 24 27

after change storage policy, there are a lot of weird system.merges:
it read data from cold disk, and then write back to it.
截屏2022-10-20 上午9 47 20

after kill this mutation, the weird merge is closed.
截屏2022-10-20 上午10 59 18

create table sql

old :

CREATE TABLE default.jaeger_spans_local
(
    `timestamp` DateTime('UTC') CODEC(Delta(4), ZSTD(1)),
)
ENGINE = MergeTree
PARTITION BY toDate(timestamp)
ORDER BY (-toUnixTimestamp(timestamp))
TTL timestamp + toIntervalDay(7), timestamp + toIntervalDay(1) TO DISK 'cold_disk'
SETTINGS index_granularity = 1024, storage_policy = 'hot_to_cold'

new:

CREATE TABLE default.jaeger_spans_local
(
    `timestamp` DateTime('UTC') CODEC(Delta(4), ZSTD(1)),
)
ENGINE = MergeTree
PARTITION BY toDate(timestamp)
ORDER BY (-toUnixTimestamp(timestamp))
TTL timestamp + toIntervalDay(7), timestamp + toIntervalDay(1) TO DISK 'cold_disk_s3'
SETTINGS index_granularity = 1024, storage_policy = 'hot_to_cold_s3'

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions