Server side

Server side

The whole framework is mainly based on Elastic Stack:

1. Filebeat
2. Logstash
3. ElasticSearch
4. Kibana

Logstash "listen" to each Fileabeat through a selected port (default: 5066). In order to process each data source, Logstash uses configuration files. Each data site will have its particular configuration according to the results of the mapping. The configuration files are automatically created in the server by the logstash-configurator-generator and placed in defined configuration folders where Logstash finds the configuration files. The ElasticSearch index is created in two steps:

1. Logstash process each filebeat data according to each configuration file and temporarily stores the data in a database (-we suggest using MySql database)
2. Logstash extract the data from the database and create or update the ElasticSearch index

It means that 2 instances of Logstash are runing in the server separately, one for processing and storing the beats in the database and another for creating the ElasticSearch index from the database.

Kibana is the visualization tool used by this framework to allow users explore the data from all the data sites. Once the index is created, it is visualized in Kibana. Kibana requires to be configured to visualized the index (or indexes creted by ElasticSearch).

For more information, please get familiar with the Elastic products.