Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not start spark jobserver that was deployed #589

Closed
ak103 opened this issue Sep 13, 2016 · 2 comments
Closed

Not start spark jobserver that was deployed #589

ak103 opened this issue Sep 13, 2016 · 2 comments

Comments

@ak103
Copy link

ak103 commented Sep 13, 2016

Hi,
Trying to start spark jobserver that was deployed.
But That does not. Could you tell me what I should do?

Using Version: master, Spark Version: 1.6.2

git clone https://github.com/spark-jobserver/spark-jobserver

cp /opt/spark-jobserver/config/local.sh.template /opt/spark-jobserver/config/local.sh

local.sh
INSTALL_DIR=/opt/spark-jobserver
LOG_DIR=/var/log/job-server
PIDFILE=spark-jobserver.pid
JOBSERVER_MEMORY=1G
SPARK_VERSION=1.6.2
MAX_DIRECT_MEMORY=512M
SPARK_HOME=/opt/spark
SPARK_CONF_DIR=$SPARK_HOME/conf
SPARK_EXECUTOR_URI=/opt/spark-1.6.2-bin-hadoop2.6.tgz
SCALA_VERSION=2.10.6 # or 2.11.6

cp /opt/spark-jobserver/config/local.sh /opt/spark-jobserver/bin/settings.sh

./bin/server_deploy.sh local

cp /opt/spark-jobserver/config/log4j-server.properties /opt/spark-jobserver/bin

./server_start.sh

Warning: Local jar /opt/spark-jobserver/bin/spark-job-server.jar does not exist, skipping.
java.lang.ClassNotFoundException: spark.jobserver.JobServer

cp /opt/spark-jobserver/job-server/target/scala-2.10/job-server_2.10-0.7.0-SNAPSHOT.jar /opt/spark-jobserver/bin

./server_start.sh

Exception in thread "main" java.lang.ClassNotFoundException: akka.cluster.ClusterActorRefProvider
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:67)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:66)
        at scala.util.Try$.apply(Try.scala:161)
        at akka.actor.ReflectiveDynamicAccess.getClassFor(DynamicAccess.scala:66)
        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)
        at akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:585)
        at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:578)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
        at spark.jobserver.JobServer$.spark$jobserver$JobServer$$makeSupervisorSystem$1(JobServer.scala:154)
        at spark.jobserver.JobServer$$anonfun$main$1.apply(JobServer.scala:156)
@mbuccini
Copy link

mbuccini commented Sep 13, 2016

Run server_package.sh local, instead of server_deploy.sh. This will create a file /tmp/job-server.tgz or similar. You will then need to unpack it and execute it from there.

In the end, you should have a directory containing the following:

-rw-r--r--   1 user  staff    12K Sep  9 16:48 gc.out
-rw-r--r--   1 user  staff    23M Sep  9 16:20 job-server.tar.gz
-rwxr-xr-x   1 user  staff   1.6K Sep  9 16:20 kill-process-tree.sh
-rw-r--r--   1 user  staff   3.3K Sep  9 16:40 local.conf
-rw-r--r--   1 user  staff   710B Sep  9 16:20 log4j-server.properties
-rwxr-xr-x   1 user  staff   1.4K Sep  9 16:20 manager_start.sh
-rwxr-xr-x   1 user  staff   1.9K Sep  9 16:20 server_start.sh
-rwxr-xr-x   1 user  staff   607B Sep  9 16:20 server_stop.sh
-rw-r--r--   1 user  staff   1.8K Sep  9 16:20 setenv.sh
-rw-r--r--   1 user  staff   1.0K Sep  9 16:20 settings.sh
-rw-r--r--   1 user  staff    26M Sep  9 16:20 spark-job-server.jar

@ak103
Copy link
Author

ak103 commented Sep 15, 2016

Thank you for the quick reply.
This problem was resolved thanks to your support.

@ak103 ak103 closed this as completed Sep 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants