AGGREGATE 模型随着ETL不断进行,查询结果不符合预期 #22112
Unanswered
linlin994395
asked this question in
A - General / Q&A
Replies: 1 comment 3 replies
-
|
which version? |
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment


Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
1. 建表语句


create table studio_xxx.xxx_dim_server
(
serverint COMMENT '服务器',pt_dtdate min NOT NULL COMMENT '开服日期') ENGINE=OLAP
AGGREGATE KEY(
server) COMMENT 'OLAP'DISTRIBUTED BY HASH(
server) BUCKETS 1PROPERTIES (
"replication_num" = "1"
);
2. 初始化插入语句
INSERT INTO studio_xxx.xxx_dim_server
select
replace(jsonb_extract(data, '$.server'), '"', ""), min(pt_dt) pt_dt
from studio_xxx.xxx_ods_total
where event_name in ('LogReg', 'LogLogin', 'LogPay')
group by replace(jsonb_extract(data, '$.server'), '"', "")
初始化插入后结果如下
3. ETL语句 每10分钟执行一次
INSERT INTO studio_xxx.xxx_dim_server
select
replace(jsonb_extract(data, '$.server'), '"', "") server,
min(pt_dt) pt_dt
from studio_xxx.xxx_ods_total
where pt_dt >= '{start_time}' and pt_dt <= '{end_time}'
and event_name in ('LogReg', 'LogLogin', 'LogPay')
group by replace(jsonb_extract(data, '$.server'), '"', "")
4. 过一段时间 select结果与预期不符(原有2服,3服数据丢失,1服数据pt_dt发生非预期改变)
分析:
Beta Was this translation helpful? Give feedback.
All reactions