Mist throwing exception for drill queries

[lalit10368]

My jobs with any drill query are not working with mist and I keep getting the following exception. The same job works fine with spark-submit.

17/06/15 05:59:04 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.lang.NoClassDefFoundError: oadd/org/apache/log4j/Logger
at oadd.org.apache.zookeeper.Login.(Login.java:44)
at oadd.org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:226)
at oadd.org.apache.zookeeper.client.ZooKeeperSaslClient.(ZooKeeperSaslClient.java:131)
at oadd.org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:949)
at oadd.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
17/06/15 05:59:05 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

Here's a sample drill query job I wrote which is working fine with spark-submit,

def execute(): Map[String, Any] = {
     
    Class.forName("org.apache.drill.jdbc.Driver")
    val connection = DriverManager.getConnection("jdbc:drill:zk=localhost:5181/drill/demo_mapr_com-drillbits;schema=dfs", "root", "mapr")
    val statement = connection.createStatement()
    val query = "select * from  dfs.tmp.`employees` limit 10"
    val resultSet = statement.executeQuery(query)
    var list : List[String] = List()
    
    while(resultSet.next()){  
      println(resultSet.getString(1));
      list = list ++ List(resultSet.getString(1))
    }
    Map("result" -> list)
  }

Also find attached the response I get from the mist API.

response.txt

[dos65]

It seems like your jar doesn't contains classes for oadd/org/apache/log4j/* inside.
Are you building fat-jar or just build your job via sbt package?

[lalit10368]

I am building a fat-jar via gradle.

[dos65]

Could you check resulting jar? Does it contains oadd/org/apache/log4j/Logger ?

[lalit10368]

It's not present in the jar.

I know that its coming from drill-jdbc-all-1.8.0.jar, the respective pom says :

<relocation>
<pattern>org.apache.log4j.</pattern>
<shadedPattern>oadd.org.apache.log4j.</shadedPattern>
</relocation>

I am wondering why the same jar doesn't cause any issue with bare spark-submit.

[lalit10368]

Any ideas on how to make it work for Mist? as the same is working with spark-submit.

[dos65]

You can try to include drill-jdbc-all-1.8.0.jar into job jar.

I don't know why it's working on bare spark. What version are you using? I don't see direct drill-jdbc dependency in spark 2.1.0

[lalit10368]

I am using spark-1.6.1, I tied to include
My spark configuration looks something like this :

mist.context-foo.spark-conf={
    spark.executor.extraClassPath = "/user/root/drill-jdbc-all-1.8.0.jar:/user/root/zookeeper-3.4.5.jar"
    spark.driver-class-path = "`hbase classpath`"
    spark.driver.extraClassPath = "/user/root/jars/sample-drill-5.3.2-all.jar"
    spark.jars = "/user/root/zookeeper-3.4.5.jar:/user/root/drill-jdbc-all-1.8.0.jar"
}

[singh-abhijeet]

I am wondering why the same jar doesn't cause any issue with bare spark-submit.

@lalit10368 Spark internally uses Log4j for its logging operations, thus the job runs when you use spark-submit.

As suggested by @dos65 , try including Log4j dependencies in the Uber JAR and submit the Drill-Mist job.

[dos65]

@singh-abhijeet it shouldn't help for that case. drill-jdbc-all depends on shaded log4j.

[lalit10368]

@dos65 , its fixed. something to do with zookeeper, changed the connection url to:

jdbc:drill:drillbit=192.168.56.103:31010;schema=dfs

[dos65]

Glad that you solved that problem!