通过jmeter压测doris耗时出现有规律的突刺

[Duingold]

配置:

fe:1台16核32G

be:3台32核64G

版本:1.2.4.1

表中数据量:50W+

压测工具:jmeter

现象:通过jmeter压测SQL语句,发现耗时图里面会出现规律的向上突刺,且给的并发越多,突刺间隔越短;不知道是什么原因造成的,有知道原因的同学吗?

[Duingold]

[yagagagaga]

Can you provide the sql and jmeter profile which can help doris be better?

[Duingold]

表A:

CREATE TABLE `test_a` (
  `A` int(11) NOT NULL COMMENT "",
  `B` bigint(20) NULL COMMENT "",
  `C` varchar(50) NULL COMMENT "",
  `D` bigint(20) NULL COMMENT "",
  `E` varchar(50) NULL COMMENT "",
  `F` bigint(20) NULL COMMENT "",
  `G` varchar(50) NULL COMMENT ""
) ENGINE=OLAP
UNIQUE KEY(`A`)
COMMENT ""
DISTRIBUTED BY HASH(`A`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false",
"storage_format" = "V2"
);

表B:

CREATE TABLE `test_b` (
  `A` int(11) NOT NULL COMMENT "",
  `BB` datetime NOT NULL COMMENT "",
  `CC` float NULL COMMENT "",
  `DD` float NULL COMMENT "",
  `EE` int(11) NULL COMMENT "",
  `FF` int(11) NULL COMMENT "",
  `GG` varchar(5) NULL COMMENT "",
  `HH` bigint(20) NULL COMMENT "",
  `B` bigint(20) NULL COMMENT "",
  `C` varchar(20) NULL COMMENT ""
  `D` bigint(20) NULL COMMENT "",
  `E` varchar(20) NULL COMMENT ""
) ENGINE=OLAP
UNIQUE KEY(`A`, `BB`)
COMMENT ""
DISTRIBUTED BY HASH(`BB`) BUCKETS 5
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false",
"storage_format" = "V2"
);

SQL:

SELECT
	t.B,
	t.F,
	t.G,
	t.A a,
	t1.BB bb1,
	t1.CC cc1,
	t1.DD dd,
	t1.EE ee1,
	t1.FF FF1,
	t1.GG gg1,
	t2.BB bb2,
	t2.CC cc2,
	t2.DD dd2,
	t2.EE ee2,
	t2.FF FF2,
	t2.GG gg2,
	t3.BB bb3,
	t3.CC cc3,
	t3.DD dd3,
	t3.EE ee3,
	t3.FF FF3,
	t3.GG gg3,
	t4.BB bb4,
	t4.CC cc4,
	t4.DD dd4,
	t4.EE ee4,
	t4.FF FF4,
	t4.GG gg4,
	t5.BB bb5,
	t5.CC cc5,
	t5.DD dd5,
	t5.EE ee5,
	t5.FF FF5,
	t5.GG gg5 
FROM
	test_a t
	LEFT JOIN (
SELECT
	* 
FROM
	( SELECT t.*, ROW_NUMBER ( ) OVER ( PARTITION BY A ORDER BY BB DESC ) RN FROM test_b t ) a 
WHERE
	rn = 1 
	AND A IS NOT NULL 
	) t1 ON t.A = t1.A
	LEFT JOIN (
SELECT
	* 
FROM
	( SELECT t.*, ROW_NUMBER ( ) OVER ( PARTITION BY A ORDER BY BB DESC ) RN FROM test_b t ) a 
WHERE
	rn = 2 
	AND A IS NOT NULL 
	) t2 ON t.A = t2.A
	LEFT JOIN (
SELECT
	* 
FROM
	( SELECT t.*, ROW_NUMBER ( ) OVER ( PARTITION BY A ORDER BY BB DESC ) RN FROM test_b t ) a 
WHERE
	rn = 3 
	AND A IS NOT NULL 
	) t3 ON t.A = t3.A
	LEFT JOIN (
SELECT
	* 
FROM
	( SELECT t.*, ROW_NUMBER ( ) OVER ( PARTITION BY A ORDER BY BB DESC ) RN FROM test_b t ) a 
WHERE
	rn = 4 
	AND A IS NOT NULL 
	) t4 ON t.A = t4.A
	LEFT JOIN (
SELECT
	* 
FROM
	( SELECT t.*, ROW_NUMBER ( ) OVER ( PARTITION BY A ORDER BY BB DESC ) RN FROM test_b t ) a 
WHERE
	rn = 5 
	AND A IS NOT NULL 
	) t5 ON t.A = t5.A 
WHERE
	t.D = '${data}' 
ORDER BY
	t.A 
	LIMIT 10

上面是其中的一个SQL,jmeter的配置如下:


压测的时候使用了两个相同的JDBC Request