Generate multiline parsers for fluent bit & support multiple parsers per filter

[dehaansa]

This config validated in test for multiline is copied from cassandra development, it works properly for cassandra system logs parsing.

The multiple parsers per filter has not yet been as thoroughly validated, but intended to be useable in situations like MySQL having a different error log format for version 8+.

[qingling128]

My main concern is that we are introducing a new Ops Agent user visible receiver parse_regex_complex. We do want to support multi format parsers. But it likely would also be applicable for json parsers and needs more time to design and integration test it.

Can we leave the multi parsers code at fluent bit subagent level for now in this PR since the goal is to facilitate specific use cases of some 3rd party applications?

[dehaansa]

My main concern is that we are introducing a new Ops Agent user visible receiver parse_regex_complex. We do want to support multi format parsers. But it likely would also be applicable for json parsers and needs more time to design and integration test it.

Can we leave the multi parsers code at fluent bit subagent level for now in this PR since the goal is to facilitate specific use cases of some 3rd party applications?

Does this also apply to the parse_regex_multiline receiver? It is currently user visible, but I can not expose it as well and keep it internally useable by fluent bit.

[qingling128]

Yes, it applies to parse_regex_multiline as well.

[dehaansa]

Code coverage reduction is due to the removal of user-facing parse_regex_complex & parse_regex_multiline, which prevents me from writing a confgenerator/testdata unit test for either of them. When we have log receivers using them, that code will get code coverage again. Alternatively, I could write an out of band unit test to cover the code.

[qingling128]

LGTM, modulo the outstanding comments get resolved.

[quentinmit]

LGTM modulo a resolution for the comment thread below