[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch check #28431

hello-stephen · 2023-12-14T14:33:46Z

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions · 2023-12-14T14:34:43Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).

For more information:
  https://www.shellcheck.net/wiki/SC2034 -- DORIS_HOME appears unused. Verify...
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/run-tpcds-sf100.sh.orig
+++ regression-test/pipeline/performance/run-tpcds-sf100.sh
@@ -83,7 +83,7 @@
     if ${DEBUG:-false}; then
         SF="100"
     fi
-    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                               # no / at the end
+    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                                # no / at the end
     TPCDS_DATA_DIR_LINK="${teamcity_build_checkoutDir}"/tools/tpcds-tools/bin/tpcds-data # no / at the end
     db_name="tpcds_sf${SF}"
     sed -i "s|^export DB=.*$|export DB='${db_name}'|g" \
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions · 2023-12-14T14:54:02Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).

For more information:
  https://www.shellcheck.net/wiki/SC2034 -- DORIS_HOME appears unused. Verify...
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/run-tpcds-sf100.sh.orig
+++ regression-test/pipeline/performance/run-tpcds-sf100.sh
@@ -83,7 +83,7 @@
     if ${DEBUG:-false}; then
         SF="100"
     fi
-    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                               # no / at the end
+    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                                # no / at the end
     TPCDS_DATA_DIR_LINK="${teamcity_build_checkoutDir}"/tools/tpcds-tools/bin/tpcds-data # no / at the end
     db_name="tpcds_sf${SF}"
     sed -i "s|^export DB=.*$|export DB='${db_name}'|g" \
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions · 2023-12-15T07:57:37Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 35:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 37:
source ../../common/github-utils.sh
       ^--------------------------^ SC1091 (info): Not following: ../../common/github-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 39:
source ../../common/oss-utils.sh
       ^-----------------------^ SC1091 (info): Not following: ../../common/oss-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-tpcds-sf100.sh line 50:
DORIS_HOME="${teamcity_build_checkoutDir}/output"
^--------^ SC2034 (warning): DORIS_HOME appears unused. Verify use (or export if used externally).

For more information:
  https://www.shellcheck.net/wiki/SC2034 -- DORIS_HOME appears unused. Verify...
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/run-tpcds-sf100.sh.orig
+++ regression-test/pipeline/performance/run-tpcds-sf100.sh
@@ -83,7 +83,7 @@
     if ${DEBUG:-false}; then
         SF="100"
     fi
-    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                               # no / at the end
+    TPCDS_DATA_DIR="/data/tpcds/sf_${SF}"                                                # no / at the end
     TPCDS_DATA_DIR_LINK="${teamcity_build_checkoutDir}"/tools/tpcds-tools/bin/tpcds-data # no / at the end
     db_name="tpcds_sf${SF}"
     sed -i "s|^export DB=.*$|export DB='${db_name}'|g" \
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions · 2023-12-15T08:01:49Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)


In regression-test/pipeline/performance/run-clickbench.sh line 81:
        bash "${teamcity_build_checkoutDir}"/tools/clickbench-tools/bin/create-clickbench-tables.sh -s "${SF}"
                                                                                                        ^---^ SC2154 (warning): SF is referenced but not assigned.

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- SF is referenced but not assigned.
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/run-clickbench.sh.orig
+++ regression-test/pipeline/performance/run-clickbench.sh
@@ -69,14 +69,14 @@
     if ! restart_doris; then echo "ERROR: Restart doris failed" && exit 1; fi
 
     echo "#### 2. check if need to load data"
-    CLICKBENCH_DATA_DIR="/data/clickbench"                                                              # no / at the end
+    CLICKBENCH_DATA_DIR="/data/clickbench"                                               # no / at the end
     CLICKBENCH_DATA_DIR_LINK="${teamcity_build_checkoutDir}"/tools/clickbench-tools/data # no / at the end
     db_name="clickbench"
     if ! check_clickbench_table_rows "${db_name}"; then
         echo "INFO: need to load clickbench data"
         # prepare data
         mkdir -p "${CLICKBENCH_DATA_DIR}"
-        
+
         # create table and load data
         bash "${teamcity_build_checkoutDir}"/tools/clickbench-tools/bin/create-clickbench-tables.sh -s "${SF}"
         rm -rf "${CLICKBENCH_DATA_DIR_LINK}"
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions · 2023-12-15T14:39:13Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)

For more information:
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors

'shfmt ' found no issues.

github-actions · 2023-12-15T15:11:35Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In regression-test/pipeline/performance/clean.sh line 36:
source ../../common/doris-utils.sh
       ^-------------------------^ SC1091 (info): Not following: ../../common/doris-utils.sh: openBinaryFile: does not exist (No such file or directory)

For more information:
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../common/doris...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors

'shfmt ' found no issues.

dataroaring · 2023-12-19T14:01:18Z

ClickBench test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

ClickBench test result on commit 5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5, data reload: false

run clickbench query with default conf and session variables
query1,0.01,0.01,0.01
query2,0.01,0.01,0.01
query3,0.01,0.01,0.01
query4,0.01,0.01,0.01
query5,0.02,0.01,0.01
query6,0.02,0.01,0.01
query7,0.01,0.00,0.01
query8,0.02,0.01,0.01
query9,0.02,0.01,0.01
query10,0.02,0.02,0.01
query11,0.01,0.01,0.01
query12,0.01,0.01,0.00
query13,0.01,0.00,0.01
query14,0.01,0.00,0.01
query15,0.01,0.01,0.00
query16,0.01,0.00,0.01
query17,0.01,0.00,0.01
query18,0.01,0.01,0.01
query19,0.01,0.01,0.01
query20,0.01,0.01,0.00
query21,0.01,0.01,0.01
query22,0.01,0.01,0.02
query23,0.02,0.01,0.00
query24,0.02,0.01,0.02
query25,0.01,0.01,0.01
query26,0.01,0.01,0.01
query27,0.01,0.01,0.00
query28,0.02,0.01,0.01
query29,0.01,0.00,0.01
query30,0.03,0.03,0.04
query31,0.01,0.01,0.01
query32,0.02,0.02,0.01
query33,0.01,0.01,0.01
query34,0.01,0.01,0.00
query35,0.00,0.01,0.01
query36,0.00,0.01,0.01
query37,0.01,0.01,0.01
query38,0.01,0.01,0.00
query39,0.02,0.01,0.01
query40,0.02,0.01,0.01
query41,0.02,0.01,0.01
query42,0.00,0.00,0.01
query43,0.01,0.01,0.01
Total cold run time: 0.54 s
Total hot run time: 0.31 s

dataroaring · 2023-12-19T14:34:28Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

TPC-DS sf1 test result on commit 5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5, data reload: false

run tpcds-sf1 query with default conf and session variables
query78	188	198	177	177
query79	58	62	55	55
query80	167	175	171	171
query81	42	39	39	39
query82	24	24	25	24
query83	76	69	68	68
query84	37	37	36	36
query85	173	170	170	170
query86	38	37	38	37
query87	83	80	78	78
query88	181	193	180	180
query89	45	43	46	43
query90	41	39	40	39
query91	61	59	59	59
query92	26	27	26	26
query93	17	17	17	17
query94	70	71	69	69
query95	75	68	67	67
query96	35	32	31	31
query97	92	96	96	96
query98	40	39	41	39
query99	48	54	49	49
Total cold run time: 11081 ms
Total hot run time: 9824 ms

doris-robot · 2023-12-20T07:21:44Z

ClickBench test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

ClickBench test result on commit 5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5, data reload: false

run clickbench query with default conf and session variables
query1,0.03,0.01,0.01
query2,0.07,0.04,0.03
query3,0.23,0.03,0.03
query4,1.92,0.04,0.04
query5,0.62,0.59,0.64
query6,1.37,0.70,0.72
query7,0.02,0.00,0.00
query8,0.03,0.04,0.02
query9,0.63,0.57,0.56
query10,0.63,0.64,0.64
query11,0.15,0.09,0.09
query12,0.15,0.10,0.10
query13,0.69,0.67,0.67
query14,0.98,0.99,1.04
query15,0.95,0.95,0.96
query16,0.44,0.47,0.47
query17,1.24,1.26,1.25
query18,0.28,0.32,0.28
query19,2.31,2.27,2.24
query20,0.02,0.02,0.02
query21,15.14,1.38,1.44
query22,13.92,3.06,1.20
query23,17.28,1.42,1.31
query24,5.86,0.95,1.10
query25,2.28,0.13,0.14
query26,0.68,0.14,0.15
query27,0.14,0.14,0.11
query28,13.68,2.24,1.56
query29,12.74,3.56,3.57
query30,0.80,0.76,0.80
query31,2.88,0.47,0.47
query32,3.35,0.62,0.62
query33,3.59,3.58,3.59
query34,15.76,6.00,9.30
query35,15.70,5.90,15.84
query36,1.36,1.28,1.27
query37,0.15,0.06,0.06
query38,0.07,0.03,0.03
query39,0.04,0.02,0.02
query40,0.23,0.18,0.17
query41,0.09,0.02,0.02
query42,0.03,0.02,0.02
query43,0.04,0.02,0.02
Total cold run time: 138.57 s
Total hot run time: 38.96 s

doris-robot · 2023-12-20T07:49:02Z

ClickBench test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

ClickBench test result on commit 5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5, data reload: false

run clickbench query with tuned conf and tuned session variables
query1,0.03,0.02,0.02
query2,0.05,0.03,0.02
query3,0.23,0.04,0.04
query4,1.91,0.04,0.04
query5,0.59,0.61,0.60
query6,1.32,0.70,0.71
query7,0.02,0.01,0.01
query8,0.03,0.03,0.03
query9,0.61,0.58,0.57
query10,0.66,0.65,0.63
query11,0.15,0.09,0.10
query12,0.15,0.10,0.11
query13,0.67,0.67,0.66
query14,0.98,0.99,0.98
query15,0.98,0.95,0.94
query16,0.46,0.48,0.48
query17,1.23,1.22,1.23
query18,0.28,0.27,0.31
query19,2.33,2.28,2.18
query20,0.02,0.02,0.01
query21,15.13,1.31,1.43
query22,15.04,2.04,1.16
query23,17.05,1.41,1.28
query24,5.26,1.46,2.15
query25,2.30,0.12,0.13
query26,0.68,0.15,0.16
query27,0.14,0.12,0.12
query28,14.75,1.61,1.50
query29,12.75,3.62,3.60
query30,0.82,0.80,0.78
query31,2.96,0.49,0.48
query32,3.37,0.63,0.61
query33,3.53,3.50,3.54
query34,15.64,5.84,14.92
query35,15.86,5.72,8.90
query36,1.38,1.30,1.31
query37,0.15,0.06,0.06
query38,0.08,0.03,0.03
query39,0.04,0.02,0.02
query40,0.21,0.18,0.17
query41,0.08,0.02,0.02
query42,0.04,0.02,0.02
query43,0.03,0.02,0.03
Total cold run time: 139.99 s
Total hot run time: 38.86 s

github-actions · 2023-12-21T02:57:18Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' found no issues.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/compile.sh.orig
+++ regression-test/pipeline/performance/compile.sh
@@ -30,7 +30,7 @@
 EOF
 
 if ${DEBUG:-false}; then
-    
+
     pull_request_num="28431"
     commit_id="b052225cd0a180b4576319b5bd6331218dd0d3fe"
     target_branch="master"
--- regression-test/pipeline/performance/run-clickbench.sh.orig
+++ regression-test/pipeline/performance/run-clickbench.sh
@@ -41,7 +41,7 @@
 # upload_doris_log_to_oss
 source "${teamcity_build_checkoutDir}"/regression-test/pipeline/common/oss-utils.sh
 
-if ${DEBUG:-false}; then 
+if ${DEBUG:-false}; then
     pull_request_num="28431"
     commit_id="5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5"
 fi
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions · 2023-12-21T03:24:20Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' found no issues.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/compile.sh.orig
+++ regression-test/pipeline/performance/compile.sh
@@ -30,7 +30,7 @@
 EOF
 
 if ${DEBUG:-false}; then
-    
+
     pull_request_num="28431"
     commit_id="b052225cd0a180b4576319b5bd6331218dd0d3fe"
     target_branch="master"
--- regression-test/pipeline/performance/run-clickbench.sh.orig
+++ regression-test/pipeline/performance/run-clickbench.sh
@@ -41,7 +41,7 @@
 # upload_doris_log_to_oss
 source "${teamcity_build_checkoutDir}"/regression-test/pipeline/common/oss-utils.sh
 
-if ${DEBUG:-false}; then 
+if ${DEBUG:-false}; then
     pull_request_num="28431"
     commit_id="5f5c4c80564c76ff4267fc4ce6a5408498ed1ab5"
 fi
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

doris-robot · 2023-12-28T11:49:41Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

ClickBench test result on commit a00a48d163ea941da42193b1c51d324f8ddfe0cc, data reload: false

query1	0.03	0.01	0.01
query2	0.06	0.03	0.02
query3	0.25	0.05	0.04
query4	1.88	0.06	0.06
query5	0.51	0.50	0.49
query6	1.44	0.66	0.69
query7	0.02	0.01	0.00
query8	0.03	0.03	0.03
query9	0.56	0.49	0.48
query10	0.55	0.56	0.56
query11	0.15	0.09	0.10
query12	0.13	0.10	0.10
query13	0.60	0.59	0.60
query14	0.78	0.77	0.80
query15	0.85	0.80	0.79
query16	0.36	0.33	0.35
query17	0.97	0.89	0.92
query18	0.28	0.27	0.27
query19	1.73	1.69	1.72
query20	0.02	0.01	0.01
query21	15.34	0.59	0.53
query22	1.56	2.62	1.74
query23	17.20	0.98	0.82
query24	7.32	0.54	1.28
query25	1.47	0.14	0.15
query26	0.30	0.16	0.15
query27	0.16	0.14	0.15
query28	7.45	0.76	0.75
query29	12.79	3.23	3.21
query30	0.53	0.55	0.53
query31	2.79	0.36	0.37
query32	3.42	0.49	0.48
query33	3.22	3.18	3.20
query34	15.51	4.16	4.16
query35	4.14	4.16	4.13
query36	1.07	0.98	1.00
query37	0.07	0.05	0.05
query38	0.07	0.02	0.02
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.01
query42	0.04	0.01	0.01
query43	0.04	0.02	0.02
Total cold run time: 105.96 s
Total hot run time: 29.98 s

hello-stephen · 2023-12-28T12:18:08Z

run buildall

doris-robot · 2023-12-28T12:31:29Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit ca936673fe4cb73f19efcdfa1b31ef89ce0c536e, data reload: false

------ Round 1 ----------------------------------
q1	22171	4732	4638	4638
q2	3940	237	177	177
q3	14515	1222	1309	1222
q4	15135	856	963	856
q5	14816	3258	3270	3258
q6	266	139	168	139
q7	1087	547	520	520
q8	12946	2050	2149	2050
q9	9456	7220	7371	7220
q10	11136	3483	3614	3483
q11	478	246	220	220
q12	448	231	237	231
q13	28160	3657	3618	3618
q14	590	225	229	225
q15	613	559	570	559
q16	1182	428	428	428
q17	1574	830	559	559
q18	14680	7221	7843	7221
q19	1859	1599	1600	1599
q20	652	356	357	356
q21	3324	2767	2738	2738
q22	380	333	347	333
Total cold run time: 159408 ms
Total hot run time: 41650 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5013	4839	4731	4731
q2	343	209	180	180
q3	3395	3335	3309	3309
q4	2193	2188	2175	2175
q5	5608	5588	5593	5588
q6	234	115	116	115
q7	2305	1839	1847	1839
q8	3398	3409	3406	3406
q9	9122	9050	9092	9050
q10	3757	3868	3853	3853
q11	484	360	362	360
q12	768	596	585	585
q13	9421	3167	3181	3167
q14	285	241	260	241
q15	573	518	516	516
q16	495	454	433	433
q17	1884	1862	1841	1841
q18	8620	8025	8020	8020
q19	1644	1648	1637	1637
q20	2164	1909	1900	1900
q21	5889	5583	5589	5583
q22	539	456	446	446
Total cold run time: 68134 ms
Total hot run time: 58975 ms

doris-robot · 2023-12-28T12:42:56Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit ca936673fe4cb73f19efcdfa1b31ef89ce0c536e, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	930	351	335	335
query2	6515	1872	2063	1872
query3	6692	227	221	221
query4	26130	22284	22175	22175
query5	3960	515	523	515
query6	254	163	171	163
query7	4594	269	274	269
query8	220	207	216	207
query9	8284	2740	2714	2714
query10	460	269	250	250
query11	16091	15319	15424	15319
query12	130	72	73	72
query13	1605	350	359	350
query14	10951	7197	7195	7195
query15	221	184	196	184
query16	6255	273	289	273
query17	1453	488	478	478
query18	1805	263	259	259
query19	182	132	130	130
query20	82	77	78	77
query21	176	91	93	91
query22	5065	4775	4774	4774
query23	32032	31379	31038	31038
query24	11221	2833	2834	2833
query25	545	327	320	320
query26	866	141	139	139
query27	2708	283	283	283
query28	6055	2046	2046	2046
query29	830	392	382	382
query30	278	137	149	137
query31	975	767	772	767
query32	86	57	54	54
query33	686	266	261	261
query34	846	454	441	441
query35	866	726	764	726
query36	1365	1203	1223	1203
query37	102	66	69	66
query38	3412	3302	3237	3237
query39	1352	1296	1317	1296
query40	194	104	90	90
query41	37	39	39	39
query42	96	92	91	91
query43	530	503	499	499
query44	1099	769	781	769
query45	197	189	184	184
query46	1067	674	670	670
query47	1663	1656	1519	1519
query48	393	317	311	311
query49	1107	312	316	312
query50	739	383	392	383
query51	5356	5285	5297	5285
query52	102	81	85	81
query53	211	150	145	145
query54	1376	561	597	561
query55	104	84	88	84
query56	214	191	189	189
query57	983	970	956	956
query58	222	198	198	198
query59	2859	2764	2654	2654
query60	248	218	224	218
query61	80	78	73	73
query62	661	464	474	464
query63	166	144	146	144
query64	4214	1518	1582	1518
query65	3330	3255	3272	3255
query66	1151	329	334	329
query67	15565	15309	15277	15277
query68	12409	536	541	536
query69	589	265	278	265
query70	1576	1539	1468	1468
query71	563	218	224	218
query72	5212	3370	3318	3318
query73	1441	334	333	333
query74	6998	6455	6492	6455
query75	5580	2255	2287	2255
query76	5426	925	1167	925
query77	878	263	268	263
query78	9607	8493	8483	8483
query79	6075	520	512	512
query80	773	351	351	351
query81	535	209	207	207
query82	349	98	89	89
query83	155	136	143	136
query84	276	55	64	55
query85	1552	261	249	249
query86	459	399	431	399
query87	3575	3340	3380	3340
query88	3398	2443	2465	2443
query89	348	261	255	255
query90	1786	216	221	216
query91	185	122	130	122
query92	68	51	50	50
query93	2422	522	447	447
query94	668	193	195	193
query95	572	555	518	518
query96	661	340	339	339
query97	4232	4144	4198	4144
query98	208	193	186	186
query99	1142	818	818	818
Total cold run time: 289962 ms
Total hot run time: 179038 ms

doris-robot · 2023-12-28T12:59:35Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.04 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17183773933 Bytes

doris-robot · 2023-12-28T13:14:24Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit ca936673fe4cb73f19efcdfa1b31ef89ce0c536e, data reload: false

------ Round 1 ----------------------------------
q1	17598	4740	4695	4695
q2	2035	152	148	148
q3	10587	1103	1154	1103
q4	10220	854	925	854
q5	7815	3062	3062	3062
q6	244	125	128	125
q7	1025	550	489	489
q8	9218	1996	1999	1996
q9	6684	6648	6635	6635
q10	8357	3217	3173	3173
q11	428	217	215	215
q12	366	214	212	212
q13	18017	3425	3409	3409
q14	243	217	206	206
q15	556	523	512	512
q16	493	389	383	383
q17	954	717	612	612
q18	7475	6751	6753	6751
q19	1598	1540	1530	1530
q20	602	352	322	322
q21	3026	2600	2601	2600
q22	346	301	309	301
Total cold run time: 107887 ms
Total hot run time: 39333 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4643	4677	4657	4657
q2	271	186	180	180
q3	3341	3317	3309	3309
q4	2220	2202	2188	2188
q5	5608	5629	5636	5629
q6	231	116	119	116
q7	2348	1836	1855	1836
q8	3439	3427	3449	3427
q9	9190	9139	9158	9139
q10	3742	3833	3820	3820
q11	490	366	367	366
q12	779	600	622	600
q13	8989	3230	3222	3222
q14	282	261	266	261
q15	569	522	511	511
q16	496	436	459	436
q17	1907	1881	1872	1872
q18	8553	8097	8104	8097
q19	1652	1651	1643	1643
q20	2179	1922	1888	1888
q21	5943	5642	5631	5631
q22	532	474	451	451
Total cold run time: 67404 ms
Total hot run time: 59279 ms

doris-robot · 2023-12-28T13:25:46Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit ca936673fe4cb73f19efcdfa1b31ef89ce0c536e, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	923	353	339	339
query2	2741	2224	2153	2153
query3	6694	220	222	220
query4	27234	22304	22126	22126
query5	3193	581	516	516
query6	269	175	169	169
query7	4603	283	280	280
query8	219	203	199	199
query9	8209	2744	2743	2743
query10	427	238	238	238
query11	16061	15381	15319	15319
query12	131	81	74	74
query13	1627	355	338	338
query14	11209	7187	7213	7187
query15	250	181	194	181
query16	5531	283	279	279
query17	1816	472	473	472
query18	1940	271	258	258
query19	241	132	133	132
query20	79	72	89	72
query21	179	89	92	89
query22	4982	4809	4868	4809
query23	32258	31150	31166	31150
query24	12207	2854	2822	2822
query25	562	338	325	325
query26	1357	135	143	135
query27	2853	286	283	283
query28	6945	2042	2047	2042
query29	1186	389	376	376
query30	289	140	139	139
query31	983	756	786	756
query32	72	58	53	53
query33	701	276	258	258
query34	914	461	464	461
query35	848	758	684	684
query36	1258	1242	1168	1168
query37	182	67	68	67
query38	3408	3289	3343	3289
query39	1337	1299	1301	1299
query40	314	88	87	87
query41	37	35	36	35
query42	102	88	88	88
query43	538	537	504	504
query44	1129	755	770	755
query45	199	185	183	183
query46	1077	676	657	657
query47	1740	1508	1521	1508
query48	390	311	321	311
query49	1248	310	299	299
query50	745	336	347	336
query51	5383	5293	5205	5205
query52	99	81	85	81
query53	220	153	148	148
query54	1397	589	586	586
query55	102	84	84	84
query56	214	187	192	187
query57	1019	955	930	930
query58	209	189	194	189
query59	2832	2622	2611	2611
query60	239	217	228	217
query61	74	72	73	72
query62	632	475	460	460
query63	165	144	143	143
query64	4875	1485	1535	1485
query65	3343	3257	3240	3240
query66	1423	337	317	317
query67	15341	15604	15217	15217
query68	11335	535	576	535
query69	613	264	268	264
query70	1624	1597	1487	1487
query71	569	217	216	216
query72	4292	3302	3313	3302
query73	1389	328	329	328
query74	7022	6429	6417	6417
query75	5692	2289	2288	2288
query76	5189	1136	1175	1136
query77	885	264	237	237
query78	9490	8589	8478	8478
query79	1965	515	522	515
query80	1719	344	339	339
query81	534	208	206	206
query82	228	95	91	91
query83	376	139	135	135
query84	270	56	54	54
query85	1853	265	266	265
query86	422	385	405	385
query87	3586	3343	3378	3343
query88	3189	2455	2428	2428
query89	333	265	261	261
query90	2070	225	224	224
query91	173	133	132	132
query92	67	51	51	51
query93	1313	500	444	444
query94	826	187	189	187
query95	591	539	570	539
query96	646	345	338	338
query97	4290	4159	4122	4122
query98	201	195	180	180
query99	1139	871	811	811
Total cold run time: 284895 ms
Total hot run time: 179173 ms

doris-robot · 2023-12-28T13:33:47Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit ca936673fe4cb73f19efcdfa1b31ef89ce0c536e, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4981	4630	4654	4630
q2	359	175	155	155
q3	1475	1241	1187	1187
q4	1125	950	880	880
q5	3315	3182	3165	3165
q6	273	126	124	124
q7	1039	486	504	486
q8	2297	2262	2265	2262
q9	6729	6668	6712	6668
q10	3181	3284	3280	3280
q11	331	201	203	201
q12	345	205	205	205
q13	4163	3399	3438	3399
q14	239	209	211	209
q15	570	518	516	516
q16	437	383	378	378
q17	1046	704	531	531
q18	7122	6821	6883	6821
q19	1635	1640	1651	1640
q20	516	319	295	295
q21	3170	2760	2682	2682
q22	362	296	305	296
Total cold run time: 44710 ms
Total hot run time: 40010 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4596	4585	4599	4585
q2	275	166	181	166
q3	3392	3379	3353	3353
q4	2207	2205	2210	2205
q5	5798	5718	5701	5701
q6	237	117	119	117
q7	2376	1854	1846	1846
q8	3609	3617	3625	3617
q9	8982	8932	8946	8932
q10	3775	3887	3906	3887
q11	493	376	370	370
q12	770	593	611	593
q13	3922	3192	3147	3147
q14	289	259	256	256
q15	579	520	518	518
q16	498	463	459	459
q17	1971	1968	1965	1965
q18	8619	8173	8275	8173
q19	1736	1776	1767	1767
q20	2243	1929	1917	1917
q21	6082	5731	5739	5731
q22	558	476	469	469
Total cold run time: 63007 ms
Total hot run time: 59774 ms

github-actions · 2023-12-28T13:44:11Z

PR approved by anyone and no changes requested.

github-actions · 2023-12-28T14:09:31Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2023-12-28T14:18:08Z

run buildall

yiguolei

lgtm

github-actions · 2023-12-28T14:37:53Z

PR approved by at least one committer and no changes requested.

doris-robot · 2023-12-28T15:18:44Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 644e7650930893c887be428d38ce4093c67ad482, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5352	5234	5158	5158
q2	390	162	149	149
q3	1462	1133	1195	1133
q4	1089	871	824	824
q5	3116	3095	3047	3047
q6	233	138	138	138
q7	965	534	539	534
q8	2146	2171	2189	2171
q9	6879	6828	6820	6820
q10	3171	3145	3140	3140
q11	344	219	227	219
q12	384	242	237	237
q13	4397	3692	3643	3643
q14	266	215	218	215
q15	625	565	569	565
q16	461	404	411	404
q17	1046	598	502	502
q18	7095	6706	6802	6706
q19	1643	1516	1550	1516
q20	587	552	368	368
q21	2901	2550	2499	2499
q22	399	325	318	318
Total cold run time: 44951 ms
Total hot run time: 40306 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5146	5076	5084	5076
q2	334	249	260	249
q3	3386	3345	3295	3295
q4	2144	2055	2019	2019
q5	5945	5967	5946	5946
q6	234	135	130	130
q7	2383	1938	1919	1919
q8	3555	3647	3662	3647
q9	9036	9005	8969	8969
q10	3888	3907	3970	3907
q11	593	489	480	480
q12	811	647	692	647
q13	3858	3200	3179	3179
q14	306	263	293	263
q15	625	589	556	556
q16	557	509	523	509
q17	2028	1859	1815	1815
q18	8765	8411	8415	8411
q19	1752	1709	1735	1709
q20	2301	2011	2010	2010
q21	5717	5320	5320	5320
q22	567	541	489	489
Total cold run time: 63931 ms
Total hot run time: 60545 ms

doris-robot · 2023-12-28T15:25:00Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.34 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17183838999 Bytes

…check (apache#28431)

ci added 3 commits December 14, 2023 22:29

[ci](perf) add perf check of tpcds

f0beee0

2

c49824f

2

b052225

2

b9c3b73

Update deploy.sh

abb3067

2

24f8386

3

26da045

3

e331519

wm1581066 requested a review from englefly December 18, 2023 02:39

xiaokang added the dev/2.0.4 label Dec 18, 2023

hello-stephen marked this pull request as draft December 18, 2023 12:54

1219

980a679

ci added 4 commits December 20, 2023 22:39

1220

158168b

1220

334b7d7

1220

798d8fd

1221

253fd2e

1221

87a4349

1221

ae0e1c4

1228

ca93667

hello-stephen marked this pull request as ready for review December 28, 2023 12:17

SWJTU-ZhangLei approved these changes Dec 28, 2023

View reviewed changes

github-actions bot added the reviewed label Dec 28, 2023

Gabriel39 previously approved these changes Dec 28, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 28, 2023

1228

644e765

hello-stephen dismissed Gabriel39’s stale review via 644e765 December 28, 2023 14:17

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 28, 2023

yiguolei approved these changes Dec 28, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 28, 2023

yiguolei merged commit 61677d1 into apache:master Dec 29, 2023
26 of 27 checks passed

hello-stephen deleted the 1214-perf-ci branch December 29, 2023 01:27

hello-stephen added a commit to hello-stephen/doris that referenced this pull request Dec 29, 2023

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch …

b37220e

…check (apache#28431)

hello-stephen mentioned this pull request Dec 29, 2023

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch … #29295

Closed

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch …

e439d84

…check (apache#28431)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch check #28431

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch check #28431

hello-stephen commented Dec 14, 2023

github-actions bot commented Dec 14, 2023

github-actions bot commented Dec 14, 2023

github-actions bot commented Dec 15, 2023

github-actions bot commented Dec 15, 2023

github-actions bot commented Dec 15, 2023

github-actions bot commented Dec 15, 2023

dataroaring commented Dec 19, 2023

dataroaring commented Dec 19, 2023

doris-robot commented Dec 20, 2023

doris-robot commented Dec 20, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 21, 2023

doris-robot commented Dec 28, 2023

hello-stephen commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

github-actions bot commented Dec 28, 2023

github-actions bot commented Dec 28, 2023

hello-stephen commented Dec 28, 2023

yiguolei left a comment

github-actions bot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch check #28431

[ci](perf) 1. add perf check of tpcds, 2. adjust clickbench and tpch check #28431

Conversation

hello-stephen commented Dec 14, 2023

Proposed changes

Further comments

github-actions bot commented Dec 14, 2023

sh-checker report

github-actions bot commented Dec 14, 2023

sh-checker report

github-actions bot commented Dec 15, 2023

sh-checker report

github-actions bot commented Dec 15, 2023

sh-checker report

github-actions bot commented Dec 15, 2023

sh-checker report

github-actions bot commented Dec 15, 2023

sh-checker report

dataroaring commented Dec 19, 2023

dataroaring commented Dec 19, 2023

doris-robot commented Dec 20, 2023

doris-robot commented Dec 20, 2023

github-actions bot commented Dec 21, 2023

sh-checker report

github-actions bot commented Dec 21, 2023

sh-checker report

doris-robot commented Dec 28, 2023

hello-stephen commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

github-actions bot commented Dec 28, 2023

github-actions bot commented Dec 28, 2023

hello-stephen commented Dec 28, 2023

yiguolei left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

doris-robot commented Dec 28, 2023

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`