fix: Remove streaming LLM tracking; they are all streaming now

[vblagoje]

Related Issues

After HF introduced token streaming in the 4.28.0 release, all their local LLMs and modes are now streaming-capable. All other PromptNode-supported models, including OpenAI, Cohere, and Anthropic, are already streaming-capable. With the introduction of recent HF inferencing API additions, all remote HF LLMs are also streaming.

There is no point in tracking streaming-capable models any longer. By default, streaming is on. If, for some reason, the Agent needs to be non-streaming, we provide the boolean parameter switch.

Proposed Changes:

• Add streaming boolean parameter to base Agent class, by default streaming is True
• Remove all tracking of streaming-capable models

How did you test it?

Manual tests

Notes for the reviewer

Let's manually test multiple models in both modes

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added tests that demonstrate the correct behavior of the change
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[julian-risch]

All streaming related changes look good to me. I have one change request regarding the added newline "\n". It means we can also change the example(s), if I am not mistaken. That's all. 🙂

[julian-risch]

This change is not related to the streaming but it means that we can slightly change the print(results) in the Agent examples, right? For example, here: https://github.com/deepset-ai/haystack/blob/main/examples/agent_multihop_qa.py#L112
Instead of

result = agent.run(query=question)
print(f"\n{result}")

we can do just print(result) in the end.

[julian-risch]

@vblagoje If I run the Agent example from this branch, the \n is still needed:

result = agent.run(query=question)
print(f"\n{result}")

Could you please briefly explain the effect of having the change end="\n" in this PR?

[vblagoje]

Sure thing. The effect is best observed when running non streaming agent. Simply switch Agent to be non streaming in init method and run multi hop examples with and without end="\n" fix. This fix has no effect on streaming agents, only on non-streaming!

[coveralls]

Coverage: 38.233% (-0.007%) from 38.239% when pulling cf0183f on all_models_are_streaming into 9d52998 on main.

[julian-risch]

Looks good to me! 👍 Just tried it in an example and a tutorial. Thanks for explaining the "\n".