Minor change in the "Start up Druid services" Section

[hpandeycodeit]

In the current "Start up Druid services" section, when starting the services in order ( as mentioned), we get this error shen starting the 1st two services. :

io.druid.java.util.common.IOE: No known server
	at io.druid.discovery.DruidLeaderClient.getCurrentKnownLeader(DruidLeaderClient.java:276) ~[druid-server-0.12.1.jar:0.12.1]
	at io.druid.discovery.DruidLeaderClient.makeRequest(DruidLeaderClient.java:128) ~[druid-server-0.12.1.jar:0.12.1]
	at io.druid.query.lookup.LookupReferencesManager.fetchLookupsForTier(LookupReferencesManager.java:569) ~[druid-server-0.12.1.jar:0.12.1]
	at io.druid.query.lookup.LookupReferencesManager.tryGetLookupListFromCoordinator(LookupReferencesManager.java:420) ~[druid-server-0.12.1.jar:0.12.1]
	at io.druid.query.lookup.LookupReferencesManager.lambda$getLookupListFromCoordinator$4(LookupReferencesManager.java:398) ~[druid-server-0.12.1.jar:0.12.1]
	at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63) [java-util-0.12.1.jar:0.12.1]
	at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) [java-util-0.12.1.jar:0.12.1]
	at io.druid.query.lookup.LookupReferencesManager.getLookupListFromCoordinator(LookupReferencesManager.java:388) [druid-server-0.12.1.jar:0.12.1]

So changing the order and moving these two process to the end:

java cat conf-quickstart/druid/historical/jvm.config | xargs -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historical
java cat conf-quickstart/druid/broker/jvm.config | xargs -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker

[jihoonson]

Hi @hpandeycodeit, the exception should be fine. In Druid, each component should retry until it finds the current leader.

I think it's more important to replace those long java commands with our simple scripts. For example, we have a script, i.e., bin/broker.sh to start/stop brokers. So, we can replace java cat conf-quickstart/druid/broker/jvm.config | xargs -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker to use this script. Probably we need to improve those scripts to accept an extra parameter for configuration directory. Similar for other components. Are you interested in this?

[hpandeycodeit]

@jihoonson,
Yes, I am interested in this. Do we have scripts ready for all these java commands or for just one?
And can you explain why an " extra parameter for configuration directory" is needed?

[jihoonson]

@hpandeycodeit we have the scripts for all Druid components. Please check ${DRUID_HOME}/bin directory.

And can you explain why an " extra parameter for configuration directory" is needed?

Ah, nvm. I thought those scripts doesn't allow to change configuration directory, and so they should be able to allow since the quickstart uses its own configuration files.
But it was wrong. You can change the configuration directory path by setting the DRUID_CONF_DIR environment variable.

[hpandeycodeit]

@jihoonson ,
The scripts are there but doesn't look like it's working:

For eg:

$./broker.sh start
Started broker node (5282)
$./broker.sh status
STOPPED

So it's not starting. Anything I am missing here to start these? I just want to see how it works before updating the doc.

[jihoonson]

It should work. Would you please check your broker logs?

[jon-wei]

I think it's more important to replace those long java commands with our simple scripts. For example, we have a script, i.e., bin/broker.sh to start/stop brokers. So, we can replace java cat conf-quickstart/druid/broker/jvm.config | xargs -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker to use this script. Probably we need to improve those scripts to accept an extra parameter for configuration directory. Similar for other components. Are you interested in this?

Just want to mention that I've been working on some improvements to the Druid quickstart with some new tutorials; as part of that upcoming patch, I'll be contributing a new "supervise" tool for running Druid services. The PR should be available in the next couple of days.

[jihoonson]

@jon-wei that sounds nice!

[hpandeycodeit]

@jihoonson,

I am getting "Not Enough direct memory" error:

 1) Not enough direct memory.  Please adjust -XX:MaxDirectMemorySize, druid.processing.buffer.sizeBytes, druid.processing.numThreads, or druid.processing.numMergeBuffers: maxDirectMemory[4,294,967,296], memoryNeeded[5,368,709,120] =
 druid.processing.buffer.sizeBytes[536,870,912] * (druid.processing.numMergeBuffers[2] + druid.processing.numThreads[7] + 1)
  at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:110) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.Dru
idProcessingModule)
  at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:110) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.Dru
idProcessingModule)
  while locating io.druid.collections.NonBlockingPool<java.nio.ByteBuffer> annotated with @io.druid.guice.annotations.Global()
    for the 2nd parameter of io.druid.query.groupby.GroupByQueryEngine.<init>(GroupByQueryEngine.java:81)
  at io.druid.guice.QueryRunnerFactoryModule.configure(QueryRunnerFactoryModule.java:88) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.QueryRunnerFact
oryModule)
  while locating io.druid.query.groupby.GroupByQueryEngine
    for the 2nd parameter of io.druid.query.groupby.strategy.GroupByStrategyV1.<init>(GroupByStrategyV1.java:77)
  while locating io.druid.query.groupby.strategy.GroupByStrategyV1
    for the 2nd parameter of io.druid.query.groupby.strategy.GroupByStrategySelector.<init>(GroupByStrategySelector.java:43)
  while locating io.druid.query.groupby.strategy.GroupByStrategySelector
    for the 1st parameter of io.druid.query.groupby.GroupByQueryQueryToolChest.<init>(GroupByQueryQueryToolChest.java:104)
  at io.druid.guice.QueryToolChestModule.configure(QueryToolChestModule.java:101) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.QueryRunnerFactoryModu
le)

[jon-wei]

@hpandeycodeit For that error, you'd want to adjust -XX:MaxDirectMemorySize=4096m in the jvm.config file for the corresponding service

[asdf2014]

Hi, @hpandeycodeit . I also encountered this situation. There is formula for it in the Druid doc. In my experience with this problem, you may need to add -XX:MaxDirectMemorySize option to both of those conf/druid/historical/jvm.config and conf/druid/broker/jvm.config files 😅

[hpandeycodeit]

Hi @asdf2014,

Thanks, that helped with the memory issue but I am hitting this new issue when trying to start the services using any .sh file in the bin directory.

Caused by: java.net.UnknownHostException: zk.host.ip: unknown error

Zookeeper is running. Do I have to set this "zk.host.ip" somewhere?

[hpandeycodeit]

@jihoonson,

I have replaced the commands with the scripts to use to start the services. Since they all can be started in the same terminal window, I have modified the preceding text as well.

Thanks!