TAJO-1480: Kafka Consumer for kafka strage.

[hys9958]

Hi guys,
I have implemented for this issue and unit tests.
Please review my patch.

Thanks.

[blrunner]

It looks like a wrong version.

[hys9958]

Oops! My mistake.
Hi, @blrunner
I have modified it.(0.11.0-SNAPSHOT)

[blrunner]

If users already ran above port for running zookeeper, this might be failed to run. Currently, Tajo provides hbase storage manager and it already implemented unit test cases with zookeeper. HBaseTestClusterUtil will be helpful for you to use zookeeper.

[blrunner]

I found many dependencies as follows.

[INFO] ------------------------------------------------------------------------
[INFO] Building Tajo Kafka Storage 0.11.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom (3 KB at 2.2 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom (2 KB at 5.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom (2 KB at 6.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloaded: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom (4 KB at 10.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloaded: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom (2 KB at 6.3 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloaded: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom (11 KB at 33.0 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom (3 KB at 9.7 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom (3 KB at 6.8 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom (2 KB at 4.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom (10 KB at 21.3 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airbase/20/airbase-20.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airbase/20/airbase-20.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom (59 KB at 61.5 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom (764 B at 2.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom (9 KB at 26.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom (25 KB at 52.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom (6 KB at 19.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom (3 KB at 9.7 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloaded: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar (52 KB at 57.5 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloaded: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar (25 KB at 81.4 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar
Downloaded: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar (162 KB at 109.2 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar (26 KB at 47.8 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar (317 KB at 163.2 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloaded: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar (73 KB at 129.0 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar (61 KB at 166.2 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar (64 KB at 120.3 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar (242 KB at 236.5 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar (3900 KB at 372.1 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar (6960 KB at 537.3 KB/sec)

I would like it more simple if you exclude unnecessary dependencies.

[hys9958]

@blrunner,
Thanks for your nice comment. I 'll fix it and update soon.

[blrunner]

Tajo follows Sun's code convention style. You need update class description as follows.

/**
 * SimpleConsumerManager is kafka...
*/

Please refer this site.
https://cwiki.apache.org/confluence/display/TAJO/Code+Style