Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible issue with graylog Beats plugin when the compression level is set to 0 #10

Closed
jalogisch opened this issue Sep 19, 2016 · 2 comments
Closed
Assignees
Labels

Comments

@jalogisch
Copy link
Member

@jalogisch jalogisch commented Sep 19, 2016

From @mmayur2016 on September 14, 2016 18:38

Expected Behavior

We are using filebeats 1.2.3 version to send data to graylog. I was trying to test how would filebeats perform based on different compression_level configurations set for logstash output in the filebeat.yml configuration.

The expected behaviour should have been that the logs should have been pushed with no errors. I am not sure if there is any setting on the Beats input plugin side that needs to be configured for the logs to get received with no exception.

Current Behavior

The current behavior is that we are seeing the following exception in the graylog server from the beats plugin side.

java.lang.IndexOutOfBoundsException: Readable byte limit exceeded: 295
    at org.jboss.netty.buffer.AbstractChannelBuffer.readByte(AbstractChannelBuffer.java:236) ~[graylog.jar:?]
    at org.graylog.plugins.beats.BeatsFrameDecoder.processBuffer(BeatsFrameDecoder.java:83) ~[?:?]
    at org.graylog.plugins.beats.BeatsFrameDecoder.decode(BeatsFrameDecoder.java:67) ~[?:?]
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) ~[graylog.jar:?]
    at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:482) ~[graylog.jar:?]
    at org.jboss.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:365) ~[graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:102) ~[graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.channelDisconnected(SimpleChannelHandler.java:199) [graylog.jar:?]
    at org.graylog2.plugin.inputs.util.ConnectionCounter.channelDisconnected(ConnectionCounter.java:49) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:120) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.channelDisconnected(SimpleChannelHandler.java:199) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:120) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:208) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:102) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:208) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:102) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [graylog.jar:?]
    at org.jboss.netty.channel.Channels.fireChannelDisconnected(Channels.java:396) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:360) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:81) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36) [graylog.jar:?]
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.closeRequested(SimpleChannelHandler.java:334) [graylog.jar:?]
    at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:260) [graylog.jar:?]

I am not sure what happens to these messages. If this is a configuration issue, I will close the issue.

Your Environment

Beats Input configuration:

bind_address: 0.0.0.0
override_source: <empty>
port: 5044
recv_buffer_size: 1048576
tcp_keepalive: false
tls_cert_file: <empty>
tls_client_auth: disabled
tls_client_auth_cert_file: <empty>
tls_enable: false
tls_key_file: <empty>
tls_key_password: ********
  • Graylog Version: 2.0.2
  • Elasticsearch Version:
  • MongoDB Version:
  • Operating System:
  • Browser version:

Copied from original issue: Graylog2/graylog2-server#2831

@jalogisch
Copy link
Member Author

@jalogisch jalogisch commented Sep 19, 2016

can you please share the configuration of your beat and give us the information which version of the beats plugin you are using.

additional is it possible for you to update to the latest version?

@jalogisch
Copy link
Member Author

@jalogisch jalogisch commented Sep 19, 2016

From @mmayur2016 on September 14, 2016 18:53

I am using the graylog-plugin-beats-1.0.3.jar. Here is the filebeat configuration.

filebeat:
  registry_file: /tmp/.gl-filebeat-registry

  spool_size: 2048
  prospectors:
    -
      paths:
        - /log/metrics*.log
        - /log/transactions*.log
        - /log/rawlogs*.log
        - /log/apache-tomcat/access_log*.log
        - /log/apache-tomcat/catalina.out
      input_type: log
      ignore_older: 1h
      max_bytes: 2097152
      fields_under_root: false

      tail_files: true

output:
  logstash:
    hosts: ["ip:5044", "ip:5044"]
    loadbalance: true
    worker: 2
    max_retries: -1
    bulk_max_size: 2048
    compression_level: 0

Please let me know if you need more information. We will probably upgrade to latest version pretty soon.

joschi pushed a commit that referenced this issue Oct 6, 2016
Jochen Schalanda
Fixes #10
@joschi joschi self-assigned this Oct 6, 2016
@joschi joschi added the in progress label Oct 6, 2016
@kroepke kroepke closed this in #12 Oct 6, 2016
kroepke added a commit that referenced this issue Oct 6, 2016
@ghost ghost assigned kroepke Oct 6, 2016
@ghost ghost removed the in progress label Oct 6, 2016
joschi added a commit that referenced this issue Oct 6, 2016
Fixes #10
(cherry picked from commit 68271be)
joschi added a commit that referenced this issue Oct 6, 2016
Fixes #10
(cherry picked from commit 68271be)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.