Skip to content

Commit eedbbcc

Browse files
committed
Include the sbin/spark-config.sh in spark-executor
This is needed because broadcast values are broken on pyspark on Mesos, it tries to import pyspark but can't.
1 parent 3292e2a commit eedbbcc

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

sbin/spark-executor

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,10 @@
1919

2020
FWDIR="$(cd `dirname $0`/..; pwd)"
2121

22+
sbin=`dirname "$0"`
23+
sbin=`cd "$sbin"; pwd`
24+
25+
. "$sbin/spark-config.sh"
26+
2227
echo "Running spark-executor with framework dir = $FWDIR"
2328
exec $FWDIR/bin/spark-class org.apache.spark.executor.MesosExecutorBackend

0 commit comments

Comments
 (0)