Update Stream Reactor to master (f9c68d4) because too many fixes have…

… come in since the last release. Add 'PREFER_HBASE' variable which makes HBase connector work in expense of elasticsearch and twitter connector. Remove Druid connector since it is incomplete work and may cause problems to other connectors.

Signed-off-by: Marios Andreopoulos <opensource@andmarios.com>

Dockerfile

@@ -24,9 +24,10 @@ RUN wget https://packages.confluent.io/archive/3.0/confluent-3.0.1-2.11.tar.gz -
     && rm -rf /opt/confluent-3.0.1-2.11.tar.gz
 
 # Add Stream Reactor and Elastic Search (for elastic connector)
-ARG STREAM_REACTOR_URL=https://archive.landoop.com/third-party/stream-reactor/stream-reactor-20160915-v0.2.2-09da116.tar.gz
+ARG STREAM_REACTOR_URL=https://archive.landoop.com/third-party/stream-reactor/stream-reactor-v0.2.2-28-gf9c68d4.tar.gz
 RUN wget "${STREAM_REACTOR_URL}" -O stream-reactor.tar.gz \
     && tar -xzf stream-reactor.tar.gz --no-same-owner --strip-components=1 -C /opt/confluent-3.0.1/share/java \
+    && rm -rf /opt/confluent-3.0.1/share/java/kafka-connect-druid \
     && rm /stream-reactor.tar.gz \
     && wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.1/elasticsearch-2.4.1.tar.gz \
     && tar xf /elasticsearch-2.4.1.tar.gz --no-same-owner \

README.md

@@ -133,6 +133,17 @@ requisite:
 This is useful if you already have a cluster with Confluent's distribution
 install and want a fancy UI.
 
+### HBase Connector
+
+Due to some issues with dependencies, the ElasticSearch connector and the HBase
+connector cannot coexist. Whilst both are available, HBase won't work. We do provide
+the `PREFER_HBASE` environment variable which will remove ElasticSearch (and the
+Twitter connector) to let HBase work:
+
+    docker run --rm -it --net=host \
+               -e PREFER_HBASE=true \
+               landoop/fast-data-dev
+
 ## FAQ
 
 - Landoop's Fast Data Web UI tools and integration test requires a few seconds

setup-and-run.sh

@@ -74,6 +74,13 @@ sed -e 's/3030/'"$WEB_PORT"'/' -e 's/2181/'"$ZK_PORT"'/' -e 's/9092/'"$BROKER_PO
        /usr/share/landoop/kafka-tests.yml \
        /usr/local/bin/logs-to-kafka.sh
 
+# Remove ElasticSearch if needed
+PREFER_HBASE="${PREFER_HBASE:-false}"
+if echo $PREFER_HBASE | egrep -sq "true|TRUE|y|Y|yes|YES|1"; then
+    rm -rf /extra-connect-jars/* /opt/confluent-*/share/java/kafka-connect-elastic*
+    echo -e "\e[92mFixing HBase connector: Removing ElasticSearch and Twitter connector.\e[39m"
+fi
+
 # Set AV_HOST if needed
 if [[ ! -z "${ADV_HOST}" ]]; then
     echo -e "\e[92mSetting advertised host to \e[96m${ADV_HOST}\e[34m\e[92m.\e[34m"

web/env.js

@@ -4,11 +4,11 @@ var runningServices = [
       "description" : "1 broker, including kafka connect distributed, schema registry, kafka rest, etc"
     },
     {
-      "name" : "Datamountaineer Stream Reactor v0.2.2",
+      "name" : "Datamountaineer Stream Reactor v0.2.2-28-gf9c68d4",
       "description" : "Source & Sink connectors supporting KCQL. ElasticSearch, Cassandra, Redis etc"
     },
     {
-      "name" : "Landoop Fast Data platform v0.7",
+      "name" : "Landoop Fast Data platform v0.7.1 / v0.7.3",
       "description" : "Manage schemas, view history, browse topics & configure your cluster from your browser."
     }
 ];