Skip to content

Commit

Permalink
[SPARK-35463][BUILD] Skip checking checksum on a system without shasum
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Not every build system has `shasum`. This PR aims to skip checksum checks on a system without `shasum`.

### Why are the changes needed?

**PREPARE**
```
$ docker run -it --rm -v $PWD:/spark openjdk:11-slim /bin/bash
roota0e001a6e50f:/# cd /spark/
roota0e001a6e50f:/spark# apt-get update
roota0e001a6e50f:/spark# apt-get install curl
roota0e001a6e50f:/spark# build/mvn clean
```

**BEFORE (Failure due to `command not found`)**
```
roota0e001a6e50f:/spark# build/mvn clean
exec: curl --silent --show-error -L https://downloads.lightbend.com/scala/2.12.10/scala-2.12.10.tgz
exec: curl --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz?action=download
exec: curl --silent --show-error -L https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz.sha512
Veryfing checksum from /spark/build/apache-maven-3.6.3-bin.tar.gz.sha512
build/mvn: line 81: shasum: command not found
Bad checksum from https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz.sha512
```

**AFTER**
```
roota0e001a6e50f:/spark# build/mvn clean
exec: curl --silent --show-error -L https://downloads.lightbend.com/scala/2.12.10/scala-2.12.10.tgz
Skipping checksum because shasum is not installed.
exec: curl --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz?action=download
exec: curl --silent --show-error -L https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz.sha512
Skipping checksum because shasum is not installed.
Using `mvn` from path: /spark/build/apache-maven-3.6.3/bin/mvn
```

### Does this PR introduce _any_ user-facing change?

Yes, this will recover the build.

### How was this patch tested?

Manually with the above process.

Closes #32613 from dongjoon-hyun/SPARK-35463.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed May 20, 2021
1 parent 3757c18 commit 8e13b8c
Showing 1 changed file with 11 additions and 7 deletions.
18 changes: 11 additions & 7 deletions build/mvn
Expand Up @@ -74,14 +74,18 @@ install_app() {
exit 2
fi
# Checksum may not have been specified; don't check if doesn't exist
if [ -f "${local_checksum}" ]; then
echo " ${local_tarball}" >> ${local_checksum} # two spaces + file are important!
# Assuming SHA512 here for now
echo "Veryfing checksum from ${local_checksum}" 1>&2
if ! shasum -a 512 -c "${local_checksum}" > /dev/null ; then
echo "Bad checksum from ${remote_checksum}"
exit 2
if [ "$(command -v shasum)" ]; then
if [ -f "${local_checksum}" ]; then
echo " ${local_tarball}" >> ${local_checksum} # two spaces + file are important!
# Assuming SHA512 here for now
echo "Veryfing checksum from ${local_checksum}" 1>&2
if ! shasum -a 512 -c "${local_checksum}" > /dev/null ; then
echo "Bad checksum from ${remote_checksum}"
exit 2
fi
fi
else
echo "Skipping checksum because shasum is not installed."
fi

cd "${_DIR}" && tar -xzf "${local_tarball}"
Expand Down

0 comments on commit 8e13b8c

Please sign in to comment.