Getting fluent parse error in logstash

[raghuOrb]

I'm using fluentd v12.33 docker image with logstash version 6.2.2 docker image provided by elastic.co and fluent codec plugin version 3.1.5

[2018-03-19T14:01:26,385][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 1:Fixnum>, :data=>125}
[2018-03-19T14:01:26,386][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 1:Fixnum>, :data=>125}
[2018-03-19T14:01:26,387][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 1:Fixnum>, :data=>125}
[2018-03-19T14:01:26,387][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 1:Fixnum>, :data=>92}
[2018-03-19T14:01:26,388][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 0:Fixnum>, :data=>34}
[2018-03-19T14:01:26,389][ERROR][logstash.codecs.fluent   ] Fluent parse error, original data now in message field {:error=>#<NoMethodError: undefined method `merge' for 0:Fixnum>, :data=>32}

Am I using the right component version to use loomsystems plugin or was it qualified and tested with some other version?

My fluentd config is

<match **>
   @type loomsystems
   host "myhost.com"
   port "12345"
   use_ssl false
</match>

And logstash config is

input {
  tcp {
    codec => fluent
    port => 12345
  }
}

[doron327]

Hi @raghuOrb the data is being send as it is, you don't need fluent codec on the logstash input side.
this is what I'm using on the logstash receiving side:

tcp {

    port => 9999

    ssl_enable => true

    ssl_cert => "creds/cert.pem"

    ssl_key => "creds/cert.key"

    ssl_verify => false

}

[doron327]

I've updated the README file to include this example :)

[raghuOrb]

@doron327 I'm using insecure connection for testing purposes. Will move to SSL later. After removing the codec => fluent, it is working smoothly. However when I wanted to write the output clause in logstash.conf to a file, I tried line, json_lines, fluent, msgpack codecs. JSON_LINES worked best for me but info such as tag is missing.

output {
  stdout {
   codec => rubydebug
  }
}

output {
  file {
    path => "/usr/share/logstash/mylog/mylog-%{+YYYY-MM-dd}.log"
    codec => json_lines
  }
}

Both of the output had tag missing.
Output from rubydebug is

{
    "@timestamp" => 2018-03-20T12:56:17.102Z,
          "host" => "XX.XX.XX.XX",
          "port" => 57504,
      "@version" => "1",
       "message" => "{\"dummy log\"}"
}

Output from json_lines is

{"@timestamp":"2018-03-20T12:56:17.102Z","host":"XX.XX.XX.XX","port":57504,"@version":"1","message":"{\"dummy log\"}"}

[doron327]

Hi @raghuOrb, you should use the Logstash tcp output clause.
regarding the tags, what tag are you expecting? you can add any tag you like in fluent before sending the data through the loom output plugin.
is that help?

[raghuOrb]

I think documentation should have info about tag_key and include_tag_key in config parameters as well. I'm successfully using it with logstash but tag was not a part of the message by default which is not intuitive.