Skip to content

Commit

Permalink
[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be part of different roles and used for multi-tenancy.

To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance number when starting the dispatcher daemon.

## How was the this patch tested?

Manual testing

Author: Timothy Chen <tnachen@gmail.com>

Closes #11281 from tnachen/multiple_cluster_dispatchers.
  • Loading branch information
tnachen authored and rxin committed Feb 20, 2016
1 parent 6ce7c48 commit a4a081d
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
5 changes: 4 additions & 1 deletion sbin/start-mesos-dispatcher.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
SPARK_MESOS_DISPATCHER_HOST=`hostname`
fi

if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi

"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher $SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
7 changes: 6 additions & 1 deletion sbin/stop-mesos-dispatcher.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,10 @@ fi

. "${SPARK_HOME}/sbin/spark-config.sh"

"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher 1
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi

"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher \
$SPARK_MESOS_DISPATCHER_NUM

0 comments on commit a4a081d

Please sign in to comment.