MetadataBuilder

[sjrl]

See the proposal: #5540 and see feature request for Haystack v1

---

LLMs clients output strings, but many components expect other object types, and LLMs may produce output in a parsable format that can be directly converted into objects. Output parsers transform these strings into objects of the user’s choosing.

MetadataBuilder. It takes the string replies and inserts them as metadata into the Documents that were originally passed to the LLM. I'm open to renaming this one, since the goal would be to output Documents with inserted metadata.

For example, a PromptNode could be used to summarize a longer doc and the user would like to have the result inserted as metadata for that Document. There it would allow us to easily add category tags, sentiment, summaries (...) to docs that can be utilized later at query time (e.g. to filter down the search space efficiently or utilize the metadata for online retrieval/generation steps)

[anakin87]

More information on the expected use cases and component I/O can be found here.

In general, it is probably best to focus on developing this component once looping and input lists are handleable by the Pipelines.
(Otherwise, we would be going to build a component that is effectively unusable in the Pipelines.)