[Q&A] Fluentd to split a json record into multiple records

[aggarwalShivani]

What is a problem?

Hi,

Is it possible in fluentd to split a record into multiple records?
For ex. fluentd reads a json record from an in_http plugin

{
  "id": 123,
  "items": ["a1", "b1"]
}

The field "items" in the record is always a list.
The requirement is to read the record, split the items field and create separate records.

{ "id": 123, "item": "a1" }
{ "id": 123, "item": "b1" } 

This needs to be sent to the destination as separate records.

I see some fluentd plugins for splitting (mentioned here), but almost all of them have not been updated since 2-3y and have hardly 1-2 releases.

Any recommendations on how to achieve this use-case?
Can something be achieved using record-transformer plugin instead?

Describe the configuration of Fluentd

No response

Describe the logs of Fluentd

No response

Environment

- Fluentd version: 1.16.6
- Fluent Package version: 5.0.5
- Operating system: Rocky8/9

[mrudrego]

Hi Community Members,

We are also looking for a splitting one record to multiple records.
Any inputs on the above query would be very helpful.

Thanks in advance

[ashie]

@aggarwalShivani
It seems that fluent-plugin-filter-split you mentioned is the most preferred plugin for your usage, it just meets your needs.
It's maintained by current core active maintainers of Fluentd, so please let us know if you have any problem on this plugin.

[aggarwalShivani]

Thank you @ashie, will use this plugin and share feedback.