Skip to content

Commit

Permalink
Added a note on the dummy access log data. Fixes #11
Browse files Browse the repository at this point in the history
  • Loading branch information
metadaddy committed Mar 23, 2016
1 parent e1950b9 commit e33796a
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions tutorial-1/log_shipping_to_elasticsearch_part1.md
Expand Up @@ -3,13 +3,14 @@


### Creating a Pipeline
* Open the DataCollector and create a new pipeline.
* Open the Data Collector and create a new pipeline.

* Note: *If you'd like, feel free to download a previously created [pipeline](pipelines/Directory_to_ElasticSearch_Tutorial_Part_1.json) that has been configured with the contents of this tutorial. In the DataCollector Main Screen, select Import Pipeline to begin*
* Note: *If you'd like, feel free to download a previously created [pipeline](pipelines/Directory_to_ElasticSearch_Tutorial_Part_1.json) that has been configured with the contents of this tutorial. In the Data Collector Main Screen, select Import Pipeline to begin*

<img style="width:80%;" src="img/import_pipeline.png">
<img style="width:80%;" src="img/import_pipeline.png">

#### Defining the source

* Drag the 'Directory' origin stage into your canvas.

* Go to the Configuration Settings below and Select the *Files* tab
Expand Down Expand Up @@ -118,6 +119,8 @@ The preview mode lets you interactively debug your stage configurations.

<img style="width:100%;" src="img/running_pipeline.png">

*Note - the sample access logs contain 'dummy' data with random IP addresses. You will see some errors logged in your pipeline for IP addresses that are not matched in the GeoIP2 database. StreamSets correctly detects and flags these errors for analysis.*

* You can fireup a [Kibana Dashboard](kibana/ApacheWebLog.json) to view the results of the import into ElasticSearch

<img style="width:100%;" src="img/part1_kibana_dashboard.png">
Expand Down

0 comments on commit e33796a

Please sign in to comment.