Skip to content

Commit

Permalink
[SPARK-29574][K8S][FOLLOWUP] Fix bash comparison error in Docker entr…
Browse files Browse the repository at this point in the history
…ypoint.sh

A small change to fix an error in Docker `entrypoint.sh`

When spark running on Kubernetes, I got the following logs:
```log
+ '[' -n ']'
+ '[' -z ']'
++ /bin/hadoop classpath
/opt/entrypoint.sh: line 62: /bin/hadoop: No such file or directory
+ export SPARK_DIST_CLASSPATH=
+ SPARK_DIST_CLASSPATH=
```
This is because you are missing some quotes on bash comparisons.

No

CI

Closes #28075 from dungdm93/patch-1.

Authored-by: Đặng Minh Dũng <dungdm93@live.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 1d0fc9a)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
dungdm93 authored and dongjoon-hyun committed Mar 31, 2020
1 parent bd2b6aa commit 5a96ee7
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ fi

# If HADOOP_HOME is set and SPARK_DIST_CLASSPATH is not set, set it here so Hadoop jars are available to the executor.
# It does not set SPARK_DIST_CLASSPATH if already set, to avoid overriding customizations of this value from elsewhere e.g. Docker/K8s.
if [ -n ${HADOOP_HOME} ] && [ -z ${SPARK_DIST_CLASSPATH} ]; then
export SPARK_DIST_CLASSPATH=$($HADOOP_HOME/bin/hadoop classpath)
if [ -n "${HADOOP_HOME}" ] && [ -z "${SPARK_DIST_CLASSPATH}" ]; then
export SPARK_DIST_CLASSPATH="$($HADOOP_HOME/bin/hadoop classpath)"
fi

if ! [ -z ${HADOOP_CONF_DIR+x} ]; then
Expand Down

0 comments on commit 5a96ee7

Please sign in to comment.