Skip to content

[chore] Download apache orc source code from github if git does not work in build.sh.#18625

Merged
yiguolei merged 3 commits intoapache:masterfrom
kaka11chen:download_apache_orc_from_github
Apr 14, 2023
Merged

[chore] Download apache orc source code from github if git does not work in build.sh.#18625
yiguolei merged 3 commits intoapache:masterfrom
kaka11chen:download_apache_orc_from_github

Conversation

@kaka11chen
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Problem summary

Describe your changes.

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In build.sh line 256:
    if [ $exit_code -ne 0 ]; then
       ^------------------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
         ^--------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
         ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    if [[ "$exit_code" -ne 0 ]]; then


In build.sh line 257:
      mkdir -p ${DORIS_HOME}/be/src/apache-orc
               ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
      mkdir -p "${DORIS_HOME}"/be/src/apache-orc


In build.sh line 258:
      curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C ${DORIS_HOME}/be/src/apache-orc --strip-components=1
                                                                                                    ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
      curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C "${DORIS_HOME}"/be/src/apache-orc --strip-components=1

For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2248 -- Prefer double quoting even when v...
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- build.sh.orig
+++ build.sh
@@ -254,8 +254,8 @@
     exit_code=$?
     set -e
     if [ $exit_code -ne 0 ]; then
-      mkdir -p ${DORIS_HOME}/be/src/apache-orc
-      curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C ${DORIS_HOME}/be/src/apache-orc --strip-components=1
+        mkdir -p ${DORIS_HOME}/be/src/apache-orc
+        curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C ${DORIS_HOME}/be/src/apache-orc --strip-components=1
     fi
 fi
 
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


@kaka11chen kaka11chen force-pushed the download_apache_orc_from_github branch from 06a0704 to e7bcc80 Compare April 13, 2023 05:52
@kaka11chen kaka11chen marked this pull request as ready for review April 13, 2023 05:52
@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In build.sh line 256:
    if [ $exit_code -ne 0 ]; then
       ^------------------^ SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
         ^--------^ SC2248 (style): Prefer double quoting even when variables don't contain special characters.
         ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    if [[ "$exit_code" -ne 0 ]]; then


In build.sh line 257:
        mkdir -p ${DORIS_HOME}/be/src/apache-orc
                 ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
        mkdir -p "${DORIS_HOME}"/be/src/apache-orc


In build.sh line 258:
        curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C ${DORIS_HOME}/be/src/apache-orc --strip-components=1
                                                                                                      ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
        curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C "${DORIS_HOME}"/be/src/apache-orc --strip-components=1

For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2248 -- Prefer double quoting even when v...
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors
'shfmt ' found no issues.

@kaka11chen kaka11chen force-pushed the download_apache_orc_from_github branch from e7bcc80 to 3adf5ba Compare April 13, 2023 05:55
@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In build.sh line 256:
    if [[ $exit_code -ne 0 ]]; then
          ^--------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
    if [[ ${exit_code} -ne 0 ]]; then


In build.sh line 257:
        mkdir -p ${DORIS_HOME}/be/src/apache-orc
                 ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
        mkdir -p "${DORIS_HOME}"/be/src/apache-orc


In build.sh line 258:
        curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C ${DORIS_HOME}/be/src/apache-orc --strip-components=1
                                                                                                      ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
        curl -L https://github.com/apache/doris-thirdparty/archive/refs/heads/orc.tar.gz | tar -xz -C "${DORIS_HOME}"/be/src/apache-orc --strip-components=1

For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors
'shfmt ' found no issues.

@kaka11chen kaka11chen force-pushed the download_apache_orc_from_github branch from 3adf5ba to 72f758e Compare April 13, 2023 05:57
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 14, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
Copy link
Contributor

run buildall

@hello-stephen
Copy link
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.27 seconds
stream load tsv: 420 seconds loaded 74807831229 Bytes, about 169 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230414082313_clickbench_pr_129048.html

@yiguolei yiguolei merged commit 4284fc4 into apache:master Apr 14, 2023
gnehil pushed a commit to gnehil/doris that referenced this pull request Apr 21, 2023
…ork in build.sh. (apache#18625)

* [chore] Download apache orc source code from github if git does not work in build.sh.

* add cd "${DORIS_HOME}"

* Fix blank issue.
Reminiscent pushed a commit to Reminiscent/doris that referenced this pull request May 15, 2023
…ork in build.sh. (apache#18625)

* [chore] Download apache orc source code from github if git does not work in build.sh.

* add cd "${DORIS_HOME}"

* Fix blank issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants