Chat Generatol Protocol POC

[anakin87]

As discussed in deepset-ai/haystack-experimental#218, I propose introducing a ChatGenerator Protocol to qualify ChatGenerator components from a static type-checking perspective.

The current PR outlines a POC demonstrating how this could work and the benefits it provides.

Why

• Chat Generators are central components in Haystack. They are used directly and often incorporated into more complex components like LLMMetadataExtractor, Agent, and Summarizer. Even LLMEvaluator could fit, though it currently uses a Generator.

• Using them inside other components has been handled in various ways. For example, see the LLMMetadataExtractor implementation. This often results in several lazy imports, support for limited set of Chat Generators, inconsistent user experience.

• Introducing a shallow abstraction to qualify Chat Generators would improve flexibility, reduce code duplication and standardize the integration in other components.

Pros

• Introducing a Protocol for Chat Generator would give us static type checking (mypy/pyright), also providing better guidance for users.
• This could simplify existing components and introduce a consistent way of defining components based on Chat Generators.

No Runtime Checking

While the Protocol ensures static type safety, it does not provide strong runtime guarantees.

• If we mark the protocol as @runtime_checkable, runtime checks would only verify method existence—not their signatures (see mypy docs).
• For this reason, I propose avoiding @runtime_checkable entirely and handling runtime validation where needed.

Protocol vs Abstract Class

• An abstract class would offer stronger guarantees both statically and dynamically, but at the cost of flexibility.

• A Protocol, on the other hand, requires little to no changes to our codebase.

  • Chat Generators wouldn't need to inherit from a base class.
  • Most Chat Generators (including community ones) would already implement the protocol.

---

For more details on the implementation, look at comments in this PR.

[anakin87]

slightly related: after intense debugging, I found that the current haystack/__init__.py, which mixes eager and lazy imports, somehow overshadows the static type of component. This means that we would not able to check Protocol adherence of run methods decorated with @component.output_types.

For this reason, I decided to go back to eager imports in this module.

[sjrl]

Is this a bug when using all protocols based on components? If so it could be good to bring this change in a separate PR.

[anakin87]

Yes. I will do this in a different PR.

BTW, this PR is just a POC and not meant to be merged.

[anakin87]

In this module, you can find how we could simplify the LLMMetadataExtractor.
This is just an example, and we should respect our breaking changes policy, first introducing chat_generator (in 2.x.0 release) and then removing generator_api and generator_api_params (in 2.x+1.0 release).

[anakin87]

This is compatible with more rich run methods.
For example def run (self, messages: List[ChatMessage], param2="default", param3="another_default")

Creating a minimal Protocol ensures that our Chat Generators are already compatible.

[anakin87]

to be able to check the adherence of run method to the Protocol, we need to add type hints to the decorator and the decorator factory.

[anakin87]

just a simple file showing the Protocol in action

[sjrl]

@anakin87 this looks really cool and looks good to me!

[sjrl]

Also after rereading the thread on the original issue I believe this protocol is helpful for the static type checking as well as visually as a Haystack user I see that the type hint is ChatGenerator rather than just Component.

Although I still do think that we should add runtime checking where needed (as you originally mention @anakin87).

[coveralls]

Pull Request Test Coverage Report for Build 13855199401

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 41 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+0.08%) to 90.0%

---

Files with Coverage Reduction	New Missed Lines	%
core/component/component.py	1	99.38%
lazy_imports.py	3	78.57%
components/extractors/llm_metadata_extractor.py	37	68.38%

Totals
Change from base Build 13855177274:	0.08%
Covered Lines:	9702
Relevant Lines:	10780

---

💛 - Coveralls

[julian-risch]

PoC looks good to me! 👍 The examples in there are very helpful and I tried pipeline.dumps() with the updated LLMMetadataExtractor to see what it would look like in YAML.