WIP

docs/elasticsearch_connector.rst

@@ -1,7 +1,7 @@
 .. _elasticsearch-overview:
 
-Elasticsearch  Connector
-========================
+Elasticsearch Connector
+=======================
 The Elasticsearch connector allows moving data from Kafka to Elasticsearch. It writes data from
 a topic in Kafka to an `index <https://www.elastic.co/guide/en/elasticsearch/reference/current/_basic_concepts.html#_index>`_
 in Elasticsearch and all data for a topic have the same
@@ -26,16 +26,57 @@ connector provides a feature to infer mapping from the schemas of Kafka messages
 
 .. _elasticsearch-quickstart:
 
-Quick start
+Quick Start
 -----------
 This quick start uses the Elasticsearch connector to export data produced by the Avro console
 producer to Elasticsearch.
 
-.. include:: includes/prerequisites.rst
-    :start-line: 2
-    :end-line: 8
+**Prerequisites:**
 
+- :ref:`Confluent Platform <installation>` is installed. This quick start assumes that you are using the Confluent CLI, but standalone installations are also supported. For more information, see :ref:`installation_archive`.
+- Elasticsearch 6.1 or later is installed and running.
 
+----------------------------
+Start the Confluent Services
+----------------------------
+
+To use Elasticsearch you will need Schema Registry, |zk|, Kafka, and Kafka Connect.
+
+#.  Start Schema Registry using the Confluent CLI. This also starts its dependencies, |zk| and Kafka.
+
+    .. sourcecode:: bash
+
+        $ confluent start schema-registry
+
+    Your output should resemble:
+
+    .. sourcecode:: bash
+
+        Starting zookeeper
+        zookeeper is [UP]
+        Starting kafka
+        kafka is [UP]
+        Starting schema-registry
+        schema-registry is [UP]
+
+
+#.  Start Kafka Connect as a standalone service. Do not use the Confluent CLI.
+
+    .. tip:: This is a long-running process and should be run in its own terminal window.
+
+    .. sourcecode:: bash
+
+    <path-to-confluent>/bin/connect-standalone etc/schema-registry/connect-avro-standalone.properties \
+    etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties
+
+    Your output should resemble:
+
+    .. sourcecode:: bash
+
+        [2018-01-17 10:37:08,097] INFO Kafka Connect standalone worker initializing ... (org.apache.kafka.connect.cli.ConnectStandalone:65)
+        [2018-01-17 10:37:08,107] INFO WorkerInfo values:
+            jvm.args = -Xmx256M, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/Users/joel/projects/confluent-4.0.0/bin/../logs, -Dlog4j.configuration=file:./bin/../etc/kafka/connect-log4j.properties
+         ...
 
 ----------------------------
 Add a Record to the Consumer
@@ -45,8 +86,8 @@ Start the Avro console producer to import a few records to Kafka:
 
 .. sourcecode:: bash
 
-  $ ./bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test-elasticsearch-sink \
-  --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'
+    <path-to-confluent>/bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test-elasticsearch-sink \
+    --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'
 
 Then in the console producer, enter:
 
@@ -118,19 +159,17 @@ Load the predefined Elasticsearch connector.
             etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties
 
 
-#.  Optional: Verify that the connector started successfully by checking the Connect worker's log:
+#.  After the connector finishes ingesting data to Elasticsearch, check that the data is available in Elasticsearch:
 
     .. sourcecode:: bash
 
-      $ confluent log connect
+      $ curl -XGET 'http://localhost:9200/test-elasticsearch-sink/_search?pretty'
+
 
-    Towards the end of the log you should see that the connector starts, logs a few messages, and then exports
-    data from Kafka to Elasticsearch.
-    Once the connector finishes ingesting data to Elasticsearch, check that the data is available in Elasticsearch:
+    Your output should resemble:
 
     .. sourcecode:: bash
 
-      $ curl -XGET 'http://localhost:9200/test-elasticsearch-sink/_search?pretty'
       {
        "took" : 2,
        "timed_out" : false,
@@ -154,38 +193,6 @@ Load the predefined Elasticsearch connector.
        }
       }
 
-#.  Stop the Connect worker as well as all the rest of the Confluent services by running:
-
-    .. sourcecode:: bash
-
-          $ confluent stop
-          Stopping connect
-          connect is [DOWN]
-          Stopping kafka-rest
-          kafka-rest is [DOWN]
-          Stopping schema-registry
-          schema-registry is [DOWN]
-          Stopping kafka
-          kafka is [DOWN]
-          Stopping zookeeper
-          zookeeper is [DOWN]
-
-    or stop all the services and additionally wipe out any data generated during this quick start by running:
-
-    .. sourcecode:: bash
-
-          $ confluent destroy
-          Stopping connect
-          connect is [DOWN]
-          Stopping kafka-rest
-          kafka-rest is [DOWN]
-          Stopping schema-registry
-          schema-registry is [DOWN]
-          Stopping kafka
-          kafka is [DOWN]
-          Stopping zookeeper
-          zookeeper is [DOWN]
-          Deleting: /tmp/confluent.w1CpYsaI
 
 Features
 --------