配置冷热存储分离疑惑 #45597
Unanswered
goeason-world
asked this question in
A - General / Q&A
配置冷热存储分离疑惑
#45597
Replies: 2 comments
-
|
搞明白了, |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
是的可以通过show tablets from table来查看是否已经存储到s3,也可以通过show partitions from table查看分区的冷热时间。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
我这里有一个测试样例
CREATE RESOURCE "tencent_cos"
PROPERTIES (
"type" = "s3",
"s3.endpoint" = "cos.ap-guangzhou.myqcloud.com",
"s3.region" = "ap-guangzhou",
"s3.bucket" = "xxxxxxxxxxx",
"s3.access_key" = "xxxxxxxxxxx",
"s3.secret_key" = "xxxxxxxxxxx",
"s3.use_aws_sdk_default_behavior" = "false",
"s3.use_path_style" = "false"
);
CREATE STORAGE POLICY cold_storage_policy
PROPERTIES(
"storage_resource" = "tencent_cos",
"cooldown_ttl" = "1d"
);
CREATE TABLE my_table_cos (
id INT,
date DATE,
value DOUBLE
)
DUPLICATE KEY(id, date)
PARTITION BY RANGE (date) (
PARTITION p202301 VALUES LESS THAN ('2023-02-01'),
PARTITION p202302 VALUES LESS THAN ('2023-03-01'),
PARTITION p202303 VALUES LESS THAN ('2023-04-01'),
PARTITION p202304 VALUES LESS THAN ('2023-05-01'),
PARTITION p202305 VALUES LESS THAN ('2023-06-01'),
PARTITION p202306 VALUES LESS THAN ('2023-07-01'),
PARTITION p202307 VALUES LESS THAN ('2023-08-01'),
PARTITION p202308 VALUES LESS THAN ('2023-09-01'),
PARTITION p202309 VALUES LESS THAN ('2023-10-01'),
PARTITION p202310 VALUES LESS THAN ('2023-11-01'),
PARTITION p202311 VALUES LESS THAN ('2023-12-01'),
PARTITION p202312 VALUES LESS THAN ('2024-01-01'),
PARTITION p202401 VALUES LESS THAN ('2024-02-01'),
PARTITION p202402 VALUES LESS THAN ('2024-03-01'),
PARTITION p202403 VALUES LESS THAN ('2024-04-01'),
PARTITION p202404 VALUES LESS THAN ('2024-05-01'),
PARTITION p202405 VALUES LESS THAN ('2024-06-01'),
PARTITION p202406 VALUES LESS THAN ('2024-07-01'),
PARTITION p202407 VALUES LESS THAN ('2024-08-01'),
PARTITION p202408 VALUES LESS THAN ('2024-09-01'),
PARTITION p202409 VALUES LESS THAN ('2024-10-01'),
PARTITION p202410 VALUES LESS THAN ('2024-11-01'),
PARTITION p202411 VALUES LESS THAN ('2024-12-01'),
PARTITION p202412 VALUES LESS THAN ('2025-01-01')
)
DISTRIBUTED BY HASH(id) BUCKETS 10
PROPERTIES (
"dynamic_partition.enable" = "true",
"storage_policy" = "cold_storage_policy",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.start" = "-48",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "10",
"replication_num" = "3"
);
我关闭服务后,手动将服务器的时间调整到3天后,即
date -s "+3 days"
启动服务,发现cos上已经有相关目录,之前为空,但依然无法判断,分区数据热的数据,怎么区分在本地卷还是在cos卷上
mysql> SELECT substring(date, 1, 7) AS partition_month, count() AS total_rows, sum(value) as total_value FROM my_table_cos GROUP BY partition_month ORDER BY partition_month;
+-----------------+------------+--------------------+
| partition_month | total_rows | total_value |
+-----------------+------------+--------------------+
| 2023-01 | 38217 | 1935533.0699999991 |
| 2023-02 | 38743 | 1955229.1399999985 |
| 2023-03 | 21467 | 1078427.160000001 |
| 2023-04 | 10542 | 529884.4199999995 |
| 2023-05 | 2522 | 128274.25999999997 |
| 2023-06 | 2513 | 123743.10000000003 |
| 2023-07 | 2524 | 126989.36000000002 |
| 2023-08 | 2448 | 122921.20999999998 |
| 2023-09 | 2505 | 128078.24999999997 |
| 2023-10 | 2535 | 127403.13000000003 |
| 2023-11 | 2502 | 126660.61000000004 |
| 2023-12 | 2482 | 121976.15000000005 |
| 2024-01 | 4604 | 233188.35 |
| 2024-02 | 2308 | 118334.12 |
| 2024-03 | 2298 | 116566.07000000002 |
| 2024-04 | 2300 | 114688.14 |
| 2024-05 | 2324 | 117612.52000000006 |
| 2024-06 | 2294 | 113606.74000000009 |
| 2024-07 | 2280 | 116766.52999999998 |
| 2024-08 | 2404 | 123040.32999999994 |
| 2024-09 | 2296 | 114863.36000000007 |
| 2024-10 | 2281 | 115320.52999999996 |
| 2024-11 | 2330 | 117626.92999999996 |
| 2024-12 | 2281 | 113436.41000000003 |
| 2025-01 | 5030 | 251946.04000000004 |
| 2025-02 | 2494 | 125585.30999999998 |
| 2025-03 | 2476 | 126977.76000000007 |
+-----------------+------------+--------------------+
请问有什么办法可以判断,数据存储在哪里吗?
通过
SHOW PROC '/dbs/zhazhaxiong/my_table_cos/partitions';
看不出什么端倪
Beta Was this translation helpful? Give feedback.
All reactions