Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Possible issue with graylog Beats plugin when the compression level is set to 0 #10
From @mmayur2016 on September 14, 2016 18:38
We are using filebeats 1.2.3 version to send data to graylog. I was trying to test how would filebeats perform based on different compression_level configurations set for logstash output in the filebeat.yml configuration.
The expected behaviour should have been that the logs should have been pushed with no errors. I am not sure if there is any setting on the Beats input plugin side that needs to be configured for the logs to get received with no exception.
The current behavior is that we are seeing the following exception in the graylog server from the beats plugin side.
I am not sure what happens to these messages. If this is a configuration issue, I will close the issue.
Beats Input configuration:
Copied from original issue: Graylog2/graylog2-server#2831
From @mmayur2016 on September 14, 2016 18:53
I am using the graylog-plugin-beats-1.0.3.jar. Here is the filebeat configuration.
filebeat: registry_file: /tmp/.gl-filebeat-registry spool_size: 2048 prospectors: - paths: - /log/metrics*.log - /log/transactions*.log - /log/rawlogs*.log - /log/apache-tomcat/access_log*.log - /log/apache-tomcat/catalina.out input_type: log ignore_older: 1h max_bytes: 2097152 fields_under_root: false tail_files: true output: logstash: hosts: ["ip:5044", "ip:5044"] loadbalance: true worker: 2 max_retries: -1 bulk_max_size: 2048 compression_level: 0
Please let me know if you need more information. We will probably upgrade to latest version pretty soon.