mkdir var/zk required, otherwise: Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain

[SamuelMarks]

Affected Version

0.14.2

Description

Ubuntu 18.04 following http://druid.io/docs/latest/tutorials/index.html

$ bin/supervise -c quickstart/tutorial/conf/tutorial-cluster.conf
[Sun Jun  2 00:52:18 2019] Running command[zk], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/zk.log]: bin/run-zk quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[coordinator], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/coordinator.log]: bin/run-druid coordinator quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[broker], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/broker.log]: bin/run-druid broker quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[router], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/router.log]: bin/run-druid router quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[historical], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/historical.log]: bin/run-druid historical quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[overlord], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/overlord.log]: bin/run-druid overlord quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Running command[middleManager], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/middleManager.log]: bin/run-druid middleManager quickstart/tutorial/conf
[Sun Jun  2 00:52:18 2019] Command[zk] exited (pid = 6876, exited = 1)
[Sun Jun  2 00:52:18 2019] Command[zk] failed, see logfile for more details: /home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/zk.log
[Sun Jun  2 00:52:21 2019] Running command[zk], logging to[/home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/zk.log]: bin/run-zk quickstart/tutorial/conf
[Sun Jun  2 00:52:21 2019] Command[zk] exited (pid = 7150, exited = 1)
[Sun Jun  2 00:52:21 2019] Command[zk] failed, see logfile for more details: /home/ubuntu/Downloads/apache-druid-0.14.2-incubating/var/sv/zk.log
^C[Sun Jun  2 00:52:22 2019] Sending signal[15] to command[middleManager] (timeout 10s).
[Sun Jun  2 00:52:22 2019] Command[middleManager] exited (pid = 6882, exited = 143)
[Sun Jun  2 00:52:22 2019] Sending signal[15] to command[overlord] (timeout 10s).
[Sun Jun  2 00:52:23 2019] Command[overlord] exited (pid = 6881, exited = 143)
[Sun Jun  2 00:52:23 2019] Sending signal[15] to command[coordinator] (timeout 10s).
[Sun Jun  2 00:52:23 2019] Sending signal[15] to command[broker] (timeout 10s).
[Sun Jun  2 00:52:23 2019] Sending signal[15] to command[router] (timeout 10s).
[Sun Jun  2 00:52:23 2019] Sending signal[15] to command[historical] (timeout 10s).
[Sun Jun  2 00:52:23 2019] Command[router] exited (pid = 6879, exited = 143)
[Sun Jun  2 00:52:23 2019] Command[historical] exited (pid = 6880, exited = 143)
[Sun Jun  2 00:52:23 2019] Command[coordinator] exited (pid = 6877, exited = 143)
[Sun Jun  2 00:52:23 2019] Command[broker] exited (pid = 6878, exited = 143)
[Sun Jun  2 00:52:23 2019] Exiting.

~/Downloads/apache-druid-0.14.2-incubating$ cat ~/Downloads/apache-druid-0.14.2-incubating/var/sv/zk.log
Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain

Looking through all the configs, and I found that zk points to a folder:

$ cat quickstart/tutorial/conf/zk/zoo.cfg
#
# Server
#

tickTime=2000
dataDir=var/zk
clientPort=2182
initLimit=5
syncLimit=2

#
# Autopurge
#

autopurge.snapRetainCount=5
autopurge.purgeInterval=1

So I tried mkdir ~/Downloads/apache-druid-0.14.2-incubating/var/zk, which worked.

[adziri]

Hi @SamuelMarks
I faced the same issue and tried what you did but it doesn't fix it

[erindru]

This also affects 0.15.0-incubating, and creating the directory also doesnt fix it

I wonder if the bundled Zookeeper is wrong, eg https://stackoverflow.com/a/56631033

[abhishekrb19]

@adziri @erindru I had the same issue. The problem seems to be that the jars are not found in the class path.

From the script run-zk,

exec java `cat "$CONFDIR"/jvm.config | xargs` \
  -cp "$WHEREAMI/../zk/lib/*:$WHEREAMI/../zk/*:$CONFDIR" \
  org.apache.zookeeper.server.quorum.QuorumPeerMain \
  "$CONFDIR"/zoo.cfg

This would expand to something like depending on your environment:
incubator-druid/examples/bin/../zk/lib/, so the libraries are expected to be in incubator-druid/examples/zk.

But the documentation here is incorrect:

The startup scripts for the tutorial will expect the contents of the Zookeeper tarball to be located at zk under the apache-druid-0.15.1-incubating package root.

Moving zk to examples/ fixes the issue for me.

I think we could either:

• update the documentation to reflect the correct directory for zk
• change the run scripts to use the package root directory

[markokole]

Did you guys follow the step where you download zookeeper and move the folder to DRUID_HOME/zk before you start the Druid service?

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.