[netflow.event_time_msec] illegal_argument_exception Invalid format

[aleks-102]

Helo! I use opendistro 0.9.0 and logstash 6.7.1.
After start elastiflow in logstash-plain.log i often see this message.

[2019-05-18T17:29:35,712][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"elastiflow-3.4.2-2019.05.18", :_type=>"doc", :routing=>nil}, #LogStash::Event:0x7da036e9], :response=>{"index"=>{"_index"=>"elastiflow-3.4.2-2019.05.18", "_type"=>"doc", "_id"=>"dv3rymoBSwHmamFUPu9T", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [netflow.event_time_msec] of type [date] in document with id 'dv3rymoBSwHmamFUPu9T'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "12893817287834063931" is malformed at "3817287834063931""}}}}}

[aleks-102]

anybody can help me?

[robcowart]

That number 12893817287834063931 is way too large. Most likely it is due to the flow being incorrectly decoded. This most often happens when multiple kinds of devices are sending to the same Logstash instance using the same flowset ID, but a different combination of fields.

Refer to this issue... #205 (comment)

[aleks-102]

Thanks. one udp port for one device (cisco asa)?

[aleks-102]

I add redis, but didn’t help me.
After analysis list flow exporters i found, what same device have more older software version (8.2).
After i temporarily delete this device, errors ended

[robcowart]

This issue will be addressed once the following PRs are merged and released for the...

Logstash UDP Input: logstash-plugins/logstash-input-udp#46
Logstash Netflow Codec: logstash-plugins/logstash-codec-netflow#187

[robcowart]

Unfortunately the Elastic team declined to merge UDP input changes (see... logstash-plugins/logstash-input-udp#46). This leaves no other option than to continue to recommend the workaround of multiple instances of the ElastiFlow pipeline.