Skip to content

Commit

Permalink
[SPARK-4832][Deploy]some other processes might take the daemon pid
Browse files Browse the repository at this point in the history
Some other processes might use the pid saved in pid file. In that case we should ignore it and launch daemons.

JIRA is down for maintenance. I will file one once it return.

Author: WangTaoTheTonic <barneystinson@aliyun.com>
Author: WangTaoTheTonic <wangtao111@huawei.com>

Closes #3683 from WangTaoTheTonic/otherproc and squashes the following commits:

daa86a1 [WangTaoTheTonic] some bash style fix
8befee7 [WangTaoTheTonic] handle the mistake scenario
cf4ecc6 [WangTaoTheTonic] remove redundant condition
f36cfb4 [WangTaoTheTonic] some other processes might take the pid

(cherry picked from commit 1768bd5)
Signed-off-by: Sean Owen <sowen@cloudera.com>
  • Loading branch information
WangTaoTheTonic authored and srowen committed Feb 13, 2015
1 parent a8f560c commit 1255e83
Showing 1 changed file with 11 additions and 9 deletions.
20 changes: 11 additions & 9 deletions sbin/spark-daemon.sh
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,9 @@ case $option in
mkdir -p "$SPARK_PID_DIR"

if [ -f $pid ]; then
if kill -0 `cat $pid` > /dev/null 2>&1; then
echo $command running as process `cat $pid`. Stop it first.
TARGET_ID="$(cat "$pid")"
if [[ $(ps -p "$TARGET_ID" -o args=) =~ $command ]]; then
echo "$command running as process $TARGET_ID. Stop it first."
exit 1
fi
fi
Expand All @@ -141,7 +142,7 @@ case $option in
fi

spark_rotate_log "$log"
echo starting $command, logging to $log
echo "starting $command, logging to $log"
if [ $option == spark-submit ]; then
source "$SPARK_HOME"/bin/utils.sh
gatherSparkSubmitOpts "$@"
Expand All @@ -154,7 +155,7 @@ case $option in
echo $newpid > $pid
sleep 2
# Check if the process has died; in that case we'll tail the log so the user can see
if ! kill -0 $newpid >/dev/null 2>&1; then
if [[ ! $(ps -p "$newpid" -o args=) =~ $command ]]; then
echo "failed to launch $command:"
tail -2 "$log" | sed 's/^/ /'
echo "full log in $log"
Expand All @@ -164,14 +165,15 @@ case $option in
(stop)

if [ -f $pid ]; then
if kill -0 `cat $pid` > /dev/null 2>&1; then
echo stopping $command
kill `cat $pid`
TARGET_ID="$(cat "$pid")"
if [[ $(ps -p "$TARGET_ID" -o args=) =~ $command ]]; then
echo "stopping $command"
kill "$TARGET_ID"
else
echo no $command to stop
echo "no $command to stop"
fi
else
echo no $command to stop
echo "no $command to stop"
fi
;;

Expand Down

0 comments on commit 1255e83

Please sign in to comment.