Support multiline logs

[zerkms]

So guys, keeping in mind logstash multiline plugin is not thread safe (and from what I understand - is not going to be for a lot of reasons) - any chance the logstash forwarder gets some option to join multiple lines and send a combined single line to the logstash?

The set of multiline plugin rules to distinguish the multiple lines of the same entry seems enough for almost every case.

May be an option to specify a join separator would be useful additionally though.

What do you think?

[kiranos]

+1 think it would be better to use this logic as close to the source as possible

[hartfordfive]

+1, would definitely be useful when it comes to elasticsearch error stack traces.

[odyssey4me]

+1 this would be very useful to distribute the multiline processing out to the log shipper instead of having to do this on the logstash indexer

[driskell]

If anyone is in dire need, this feature exists in Log Courier.
https://github.com/driskell/log-courier/blob/stable/docs/codecs/Multiline.md

[odyssey4me]

Thanks @driskell - also note that similar support appears to be available in the beaver log shipper: https://github.com/josegonzalez/python-beaver

[pandujar]

+1

[klynch]

+1

[tgagor]

+1

[debraj-manna]

+1

[Mrc0113]

+1

[cknapp]

+1

[prashanthrajagopal]

+1

[yancl]

+1

[swood]

Hello.

Is it already done? I still have the same problem with logstash-forwarder.

[vmcano]

+1

[ruflin]

This isn't implemented but is planned for Filebeat, the next version of logstash-forwarder. I'm closing this issue so all future comments and inputs can go directly into: https://github.com/elastic/filebeat/issues/89