Skip to content

[Enhance](tools) change SSB perf tests to standardize behavior#30002

Merged
BiteTheDDDDt merged 1 commit intoapache:masterfrom
zclllyybb:shells
Jan 16, 2024
Merged

[Enhance](tools) change SSB perf tests to standardize behavior#30002
BiteTheDDDDt merged 1 commit intoapache:masterfrom
zclllyybb:shells

Conversation

@zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

before:
avg of 3 runs(1cold + 2hot)
now:
focus on fast hot run.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 136:
    res1=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res1=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 137:
    res2=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res2=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 138:
    res3=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res3=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 140:
    min_value=$(echo "$res1 $res2 $res3" | tr ' ' '\n' | sort -n | head -n 1)
                      ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                            ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                  ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    min_value=$(echo "${res1} ${res2} ${res3}" | tr ' ' '\n' | sort -n | head -n 1)


In tools/ssb-tools/bin/run-ssb-flat-queries.sh line 141:
    echo -e "q${i}:\t$res1\t$res2\t$res3\tfast:$min_value"
                     ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                            ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                               ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    echo -e "q${i}:\t${res1}\t${res2}\t${res3}\tfast:${min_value}"


In tools/ssb-tools/bin/run-ssb-queries.sh line 149:
    res1=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res1=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-queries.sh line 150:
    res2=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res2=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-queries.sh line 151:
    res3=$(mysql -vvv -h$FE_HOST -u$USER -P$FE_QUERY_PORT -D$DB -e "$(cat ${QUERIES_DIR}/q${i}.sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)
                        ^------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                        ^------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                           ^------------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                           ^------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                            ^-^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
                                                            ^-^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                                                          ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.
                                                                                          ^--^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.

Did you mean: 
    res3=$(mysql -vvv -h"${FE_HOST}" -u"${USER}" -P"${FE_QUERY_PORT}" -D"${DB}" -e "$(cat "${QUERIES_DIR}"/q"${i}".sql)" | perl -nle 'print $1 if /\((\d+\.\d+)+ sec\)/' || :)


In tools/ssb-tools/bin/run-ssb-queries.sh line 153:
    min_value=$(echo "$res1 $res2 $res3" | tr ' ' '\n' | sort -n | head -n 1)
                      ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                            ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                  ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    min_value=$(echo "${res1} ${res2} ${res3}" | tr ' ' '\n' | sort -n | head -n 1)


In tools/ssb-tools/bin/run-ssb-queries.sh line 154:
    echo -e "q${i}:\t$res1\t$res2\t$res3\tfast:$min_value"
                     ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                            ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                   ^---^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.
                                               ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    echo -e "q${i}:\t${res1}\t${res2}\t${res3}\tfast:${min_value}"

For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2248 -- Prefer double quoting even when v...
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors
'shfmt ' found no issues.

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

Copy link
Contributor

@hello-stephen hello-stephen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 16, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BiteTheDDDDt BiteTheDDDDt merged commit 04701b6 into apache:master Jan 16, 2024
@zclllyybb zclllyybb deleted the shells branch January 16, 2024 05:02
yiguolei pushed a commit that referenced this pull request Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants