running two icingabeats

[Klar]

Expected Behavior

Two icingabeats sending data to two logstash listeners, only one input gets inserted to elasticsearch.

Current Behavior

Without configuration the data will be written twice, with the elasticsearch fingerprint configuration (https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html) it should only write it once (fingerprint already exists if icingabeat #1 was faster)

The problem with the fingerprint is that icingabeat has a lot of different field so I can't just set a fingerprint for "message".

Possible Solution

Write if else for every icingabeat field if not empty and then set a fingerprint.
Not sure if this is the best way, there might be an easier solution from icinga direct?

Steps to Reproduce (for bugs)

1. two icingas with two running icingabeat services
2. both send data to logstash
3. logstash will write the data but it will be written twice (no check of if already written from icingabeat Eventstream comments and downtimes not send #1 || Explain what a beat is clearly in the repo #2)

Context

At the moment e run two icingabeats on two icingas but one is not running. If icingabeat on #1 is not working we will start #2 , a clean icingabeat cluster would be nice.

[dnsmichi]

Hm, those services know nothing from each other.

Some ideas from the #osmc hackathon (@bobapple & @urso)

• https://github.com/Icinga/icingabeat/blob/master/icingabeat.full.yml allows for the instance name and custom field addition. Maybe this could be used to dump the data and filter later on.
• Elasticsearch feature as an alternative does not support all the events Icingabeat does, neither is has Ha support
• Calculating a fingerprint for a message is hard - the timestamps might differ between the two cluster nodes. The output, state, etc. are not unique, they may occur like 5 seconds later.

I'd say for an immediate trial, go for custom fields or the name and experiment with that on de-duplication in Logstash.

[bobapple]

I don't think that there's something we can do in icingabeat to solve this. For workarounds, see @dnsmichi reply.

[manishkmbl0]

Hi ,

How to configure logstash script to take data from icingabeat. I have configure icingabeat.ym file to send data to logstash.