No Syslogs between 2:00 and 4:00 am?

[ben-sec]

Hello!

I successfully collect syslogs on my manager node on port 514 from a firewall and since a few days I notice, that I don't import them in ES between approx. 2:00 am and 4:00 am. I'm wondering if there is something going on in ES that prevents indexing, but I was not able to find any errors in Filebeat, Logstash or ES. According to the local logs of my firewall there are tons of events. Ideas?

Cheers, Ben

[weslambert]

There shouldn't be going on with ES or associated components, unless it has been restarted and is taking a long time for shards to become available. Otherwise, I would look something from the network side, or schedule a manual tcpdump to run at that time to see if anything is hitting the actual interface.

[ben-sec]

Looks like I have a timeout issue in Kibana. I'm currently trying to play around with the Kibana settings for elasticsearch.requestTimeout and elasticsearch.shardTimeout, so far without success. Is /opt/so/saltstack/local/salt/kibana/etc/kibana.yml the right place to put these settings?

[weslambert]

You should be able to adjust Kibana values in the pillar. Take a look at /opt/so/saltstack/default/salt/kibana/defaults.yml to get an idea of how the settings are configured.

[ben-sec]

I set elasticsearch.requestTimeout and elasticsearch.shardTimeout in /opt/so/saltstack/default/salt/kibana/defaults.yaml and now the timeout is gone. Thanks!