Release v3.0.0 · deepset-ai/haystack

⭐️ Highlights

Haystack 3.0 is a major release for building production-grade agents with full control and flexibility. It ships a wave of new capabilities: a more capable Agent with hooks and first-class skills, built-in run introspection, first-class async for serving, a leaner core, and safer pipeline loading.

A few small, intentional breaking changes come with it but our Migration Guide and Upgrading to Haystack 3.0 make it easy to upgrade.

🤖 A more capable, adaptable `Agent`

The Agent gained a general-purpose hooks system and now owns tool execution end to end (the standalone ToolInvoker has been removed). Hooks at before_run, before_llm, before_tool, after_tool, on_exit, and after_run let you shape behavior, enforce guardrails, and add human-in-the-loop checkpoints without touching the agent's internals:

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.hooks import hook

@hook
def audit_tool_calls(state):
    pending = state.data["messages"][-1].tool_calls
    print(f"about to run: {[tc.tool_name for tc in pending]}")

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[...],
    hooks={"before_tool": [audit_tool_calls]},
)

Built on this foundation:

Skills as first-class citizens — SkillToolset discovers skills through progressive disclosure, so the model sees only names and one-line descriptions until it loads one, keeping context lean.
Dynamic tool selection at runtime — pass tools=... to run / run_async so one reusable Agent serves different teams, tenants, and tasks.
Native async tools — @tool routes async def callables to a Tool's new async_function.
Human-in-the-Loop is now a before_tool hook (ConfirmationHook). ⚠️
Tool result offloading (ToolResultOffloadHook) writes large results to a store and leaves a compact pointer in the conversation.

🔎 Built-in introspection and observability

Agents now expose step_count, token_usage, and tool_call_counts as built-in state. React to them to compact context when the window fills, cap runaway tool loops, or route to a cheaper approach past a budget threshold. Tracing now emits dedicated step-level spans (haystack.agent.step with nested .llm / .tool children) tagged with the tools actually used, so you can see exactly what your agent did. ⚠️

⚡ A core built for serving: first-class async

Pipeline and AsyncPipeline are now one class — no more switching between them. A single Pipeline exposes run, run_async, run_async_generator, and stream; serialized pipelines, SuperComponents, and PipelineTools load as Pipeline instances. ⚠️

from haystack import Pipeline

pipeline = Pipeline()
result = pipeline.run(data)                 # synchronous
result = await pipeline.run_async(data)     # asynchronous

Concurrent tool calls and token-by-token streaming come standard: Pipeline.stream() yields StreamingChunks as they're produced and exposes the final output on handle.result — the lower latency and streaming UX serving apps need. Components and Pipeline also get a symmetric lifecycle — warm_up to acquire, close to release — so long-running services don't leak connections, GPU memory, or file handles. External resources (and API keys) are now created at warm_up, not in __init__. ⚠️

🧱 A leaner, faster-moving framework ⚠️

Legacy Generators are gone, haystack-experimental is no longer a core dependency, and 30 components now live in independently released packages in haystack-core-integrations: Sentence Transformers, local & API Hugging Face, Whisper, spaCy/langdetect, Tika and Azure OCR, SerperDev/SearchApi, OpenAPI connectors, and the Datadog/OpenTelemetry tracers. As a result, there are fewer transitive dependencies to audit and we can ship independent of the core release cycle. Migration is an install plus an import change:

# pip install sentence-transformers-haystack
# Before: from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder

The full old → new import table is in MIGRATION.md.

💬 Chat Generators everywhere (legacy Generators removed) ⚠️

OpenAIGenerator, AzureOpenAIGenerator, HuggingFaceAPIGenerator, and HuggingFaceLocalGenerator have been removed in favor of their Chat Generator counterparts. To ease the switch, all Chat Generators now accept a plain str for messages:

# Before
from haystack.components.generators import OpenAIGenerator
text = OpenAIGenerator().run("What is NLP?")["replies"][0]        # str

# After
from haystack.components.generators.chat import OpenAIChatGenerator
reply = OpenAIChatGenerator().run("What is NLP?")["replies"][0]    # ChatMessage
text = reply.text                                                 # str; metadata in reply.meta

DALLEImageGenerator was also renamed to OpenAIImageGenerator. ⚠️

🔒 Safer pipeline loading ⚠️

Pipeline deserialization (Pipeline.load / loads / from_dict) is now gated by a trusted-module allowlist (haystack, haystack_integrations, haystack_experimental, builtins, typing, collections by default), and dangerous builtins like eval, exec, open, and getattr can no longer be resolved. Allowlist custom modules when needed:

Pipeline.load(fp, allowed_modules=["mypkg.*"])   # or HAYSTACK_DESERIALIZATION_ALLOWLIST / allow_deserialization_module(...)
Pipeline.load(fp, unsafe=True)                    # only if you fully trust the source

🧪 Testing without the API bill

Newly added mock components mean no API keys, no network, no flaky CI for testing and prototyping. MockChatGenerator returns predefined responses, MockTextEmbedder and MockDocumentEmbedder return stable, hash-derived embeddings.

A few smaller breaking changes to know about

Required template variables by default — Agent, PromptBuilder, and ChatPromptBuilder now treat every Jinja2 variable as required (required_variables=None restores the old behavior); a new {% insert %} tag interleaves runtime messages. ⚠️
No more implicit State injection by name — tools must declare inputs_from_state explicitly to read a State value; same-named parameters no longer fill in automatically. ⚠️
Deterministic Document.id — computed from canonical, key-sorted JSON of meta, so documents with non-empty meta get different IDs than in 2.x. ⚠️
Scoped logging — configure_logging attaches to Haystack's own loggers; importing Haystack no longer reconfigures structlog process-wide. ⚠️
Explicit tracing — no longer auto-enabled; add an OpenTelemetryConnector / DatadogConnector or call tracing.enable_tracing(...). ⚠️

See the Upgrade Notes below for the full list.

⬆️ Upgrade Notes

AsyncPipeline has been removed. Its asynchronous capabilities are now part of the single Pipeline class, which provides both a synchronous run method and the asynchronous run_async, run_async_generator, and stream methods (mirroring how components expose run and run_async).

To migrate, replace AsyncPipeline with Pipeline:
```
# Before
from haystack import AsyncPipeline
pipeline = AsyncPipeline()
result = await pipeline.run_async(data)

# After
from haystack import Pipeline
pipeline = Pipeline()
result = await pipeline.run_async(data)
```
Serialized pipelines, SuperComponents, and PipelineTools are loaded as Pipeline instances.

Pipeline.run runs synchronously and blocks until completion; in an async context use await pipeline.run_async(...). The former AsyncPipeline.run was a synchronous wrapper around the concurrent engine, so to preserve that behavior use asyncio.run(pipeline.run_async(...)). Note that Pipeline.run executes components sequentially and does not accept concurrency_limit.

Both synchronous and asynchronous runs are traced under a single haystack.pipeline.run operation name, distinguished by a haystack.pipeline.execution_mode tag (sync or async). Previously, asynchronous runs used the haystack.async_pipeline.run operation name.
continue_run is now a reserved key in Agent.state_schema (set by an on_exit hook to keep the Agent running). Passing it via the state_schema argument now raises ValueError. Rename any conflicting state key (e.g. my_continue_run) to migrate.
Agent no longer validates exit_conditions against tool names at initialization. Exit conditions are now evaluated at runtime, so a condition naming a tool that is loaded later — e.g. a tool passed at runtime or one discovered by SearchableToolset — works as expected. Consequently, an unknown exit-condition name no longer raises a ValueError at construction.
Tool support is now validated against the tools resolved for each run, not only the tools provided at initialization. Passing tools at runtime via run(tools=...) to a chat generator that does not support tools now raises a TypeError instead of silently ignoring them.
Agent now treats every Jinja2 template variable in user_prompt and system_prompt as required by default. The default of the required_variables init parameter has changed from None (all variables optional) to "*" (all variables required). This aligns Agent with LLM, PromptBuilder, and ChatPromptBuilder.

To check if you're affected: look for Agent(...) calls that pass a user_prompt or system_prompt containing template variables without passing required_variables. If those Agents relied on missing variables being silently rendered as empty strings, the behavior depends on how the missing variable was reaching the Agent:
- Calling agent.run() directly without providing a required variable now raises ValueError.
- In a pipeline, if a required variable is neither provided as a user input nor connected from another component, Pipeline.run raises ValueError("Missing mandatory input '<var>' for component '<name>'.").
- In a pipeline, if a required variable is connected to a sender that never produces output on a given run (for example, a loop-fed input on the first iteration or a conditional branch that isn't taken), the pipeline cannot make progress and raises PipelineComponentsBlockedError.
Migration options:
- To require only a subset of variables, pass required_variables=["var1", "var2"].
- To preserve the previous "all optional" behavior, explicitly pass required_variables=None.
- To require everything (the new default), no change is needed.
Components in Haystack that use external resources now consistently create those resources during warm-up rather than in __init__. As a result, some components that previously raised errors during initialization (for example, because of a missing API key) now raise them during warm_up instead. Similar changes will be rolled out to components in Haystack Core Integrations in the future.
step_count, token_usage, and tool_call_counts are now reserved keys in Agent.state_schema. Passing any of them via the state_schema argument now raises ValueError. Rename the conflicting state key (e.g. my_token_usage) to migrate.
Confirmation strategies now receive the model-requested tool arguments in tool_params rather than the fully-prepared arguments. Values injected from State via a tool's inputs_from_state mapping (and the State object injected into State-typed parameters) are no longer included in what is presented for confirmation — that injection now happens only at tool execution time. If your ConfirmationUI or ConfirmationPolicy displayed or inspected those state-injected values, it will now see only the arguments the model provided.
The hash used to auto-generate Document.id is now computed from a canonical (key-sorted) JSON serialization of meta. Documents with empty meta are unaffected, but most other documents will get different IDs than they did before:
- documents with non-empty meta (the serialization changes from dict's repr to JSON);
- documents whose meta contains non-JSON-serializable values such as datetime or custom classes (these are now serialized via str(...) rather than repr(...), e.g. "2024-01-01 00:00:00" instead of "datetime.datetime(2024, 1, 1, 0, 0)").
If you rely on auto-generated IDs to match documents already persisted in a DocumentStore, you will need to re-ingest the affected documents (or pass the previous id explicitly when constructing the Document).
Pipeline deserialization (Pipeline.load / Pipeline.loads / Pipeline.from_dict) now refuses to import classes from modules outside a trusted-module allowlist. By default the allowlist contains haystack, haystack_integrations, haystack_experimental, builtins, typing, and collections. Pipelines that reference custom components or callables in other packages will fail to load with a DeserializationError until the additional modules are added to the allowlist.

To restore loading of such pipelines, extend the allowlist via one of:
- per-call kwarg: Pipeline.load(fp, allowed_modules=["mypkg.*"]);
- process-wide programmatic API: from haystack.core.serialization import allow_deserialization_module;
- environment variable: HAYSTACK_DESERIALIZATION_ALLOWLIST="mypkg.*,otherpkg.*";
- or, if the source of the serialized data is fully trusted, bypass the allowlist with Pipeline.load(fp, unsafe=True).
During pipeline deserialization, default_from_dict now validates the keys of init_parameters against the parent class's __init__ signature before recursing into any nested {"type": "...", "init_parameters": {...}} dictionary. A nested dict whose key is not an accepted parameter name on the parent class is rejected with a DeserializationError before the nested type is imported. Classes whose constructor takes **kwargs are exempt, since their accepted parameter set cannot be statically determined.

This may surface pre-existing YAML bugs — e.g. typos, leftovers from renamed or removed parameters, or stale snapshots from older Haystack versions. The fix is to update the YAML so each nested-component key matches a real __init__ parameter on the parent class.
PromptBuilder and ChatPromptBuilder now treat every Jinja2 template variable as required by default. Previously, variables were optional by default unless explicitly listed in required_variables.

To check if you're affected: look for PromptBuilder(...) or ChatPromptBuilder(...) calls in your code that do not pass required_variables. If those components relied on missing variables being silently rendered as empty strings, the behavior depends on how the missing variable was reaching the builder:
- Calling builder.run() directly without providing a required variable now raises ValueError.
- In a pipeline, if a required variable is neither provided as a user input nor connected from another component, Pipeline.run raises ValueError("Missing mandatory input '<var>' for component '<name>'.").
- In a pipeline, if a required variable is connected to a sender that never produces output on a given run (for example, a loop-fed input on the first iteration or a conditional branch that isn't taken), the pipeline cannot make progress and raises PipelineComponentsBlockedError.
Migration options:
- To require only a subset of variables, pass required_variables=["var1", "var2"].
- To preserve the previous "all optional" behavior, explicitly pass required_variables=None.
- To require everything (the new default), no change is needed.
Human-in-the-Loop tool confirmation on Agent is now expressed as a before_tool hook instead of a dedicated concept. The confirmation_strategies and confirmation_strategy_context parameters have been removed from Agent.__init__, Agent.run, and Agent.run_async. To migrate, wrap your confirmation strategies in the new ConfirmationHook, register it under the before_tool hook point, and pass any request-scoped resources through the generic hook_context run argument instead of confirmation_strategy_context. The Human-in-the-Loop module has also moved from haystack.human_in_the_loop to haystack.hooks.human_in_the_loop, so that it lives alongside the other built-in hooks (such as tool result offloading); update your imports to the new location:
```
from haystack.components.agents import Agent
from haystack.hooks.human_in_the_loop import (
    AlwaysAskPolicy,
    BlockingConfirmationStrategy,
    ConfirmationHook,
    RichConsoleUI,
)

confirmation_hook = ConfirmationHook(
    confirmation_strategies={
        "my_tool": BlockingConfirmationStrategy(
            confirmation_policy=AlwaysAskPolicy(), confirmation_ui=RichConsoleUI()
        )
    }
)
agent = Agent(chat_generator=..., tools=[...], hooks={"before_tool": [confirmation_hook]})
agent.run(messages=[...], hook_context={"websocket": ws})
```
The agent-specific breakpoint API has been removed. The AgentBreakpoint, ToolBreakpoint, and AgentSnapshot dataclasses are no longer exported from haystack.dataclasses, and the break_point, snapshot, and snapshot_callback parameters have been removed from Agent.run and Agent.run_async. Pipeline.run no longer accepts AgentBreakpoint for its break_point argument, and the agent_snapshot field has been removed from PipelineSnapshot. Pipeline-level breakpoints via Breakpoint and PipelineSnapshot continue to work as before, but pausing and resuming execution inside an Agent (at the chat generator or tool invoker) is no longer supported.
AzureOCRDocumentConverter has been moved out of Haystack into the azure-form-recognizer-haystack integration package. Install the new package with pip install azure-form-recognizer-haystack and update your imports.

Before:
```
from haystack.components.converters import AzureOCRDocumentConverter
```
After:
```
from haystack_integrations.components.converters.azure_form_recognizer import AzureOCRDocumentConverter
```

The DatadogTracer has been moved out of Haystack into the datadog-haystack integration package. It is no longer auto-enabled when ddtrace is installed; instead, use the new DatadogConnector component to enable Datadog tracing in your pipeline. Install the new package with pip install datadog-haystack.

Before:

import ddtrace
from haystack import tracing
from haystack.tracing.datadog import DatadogTracer

tracing.enable_tracing(DatadogTracer(ddtrace.tracer))

After:

from haystack import Pipeline
from haystack_integrations.components.connectors.datadog import DatadogConnector

pipe = Pipeline()
pipe.add_component("tracer", DatadogConnector())

Alternatively, you can still enable the tracer manually:

import ddtrace
from haystack import tracing
from haystack_integrations.tracing.datadog import DatadogTracer

tracing.enable_tracing(DatadogTracer(ddtrace.tracer))

OpenAIGenerator, AzureOpenAIGenerator, HuggingFaceAPIGenerator, and HuggingFaceLocalGenerator have been removed. Use the Chat Generators instead: OpenAIChatGenerator, AzureOpenAIChatGenerator, HuggingFaceAPIChatGenerator, and HuggingFaceLocalChatGenerator. As of v3.0, all ChatGenerators accept a plain str as input and normalise it to a user message.

Before:

from haystack.components.generators import OpenAIGenerator

gen = OpenAIGenerator()
text = gen.run("What is NLP?")["replies"][0]          # str

After:

from haystack.components.generators.chat import OpenAIChatGenerator

gen = OpenAIChatGenerator()
reply = gen.run("What is NLP?")["replies"][0]         # ChatMessage
text = reply.text                                     # str; metadata is in reply.meta

DALLEImageGenerator has been renamed to OpenAIImageGenerator and moved to haystack.components.generators.openai_image_generator. The rename reflects that OpenAI retired the DALL-E model family but the component works with newer image generation models.

Before:
```
from haystack.components.generators import DALLEImageGenerator

generator = DALLEImageGenerator()
```
After:
```
from haystack.components.generators import OpenAIImageGenerator

generator = OpenAIImageGenerator()
```
haystack-experimental is no longer a core dependency of Haystack and is no longer installed automatically with haystack-ai. If your code imports from haystack_experimental (directly or through an integration that relies on it), you must now install it explicitly, for example with pip install haystack-experimental. Installations that do not use haystack_experimental are unaffected.

HuggingFaceAPIChatGenerator, HuggingFaceAPITextEmbedder, HuggingFaceAPIDocumentEmbedder, and HuggingFaceTEIRanker have been moved out of Haystack into the huggingface-api-haystack integration package. Install the new package with pip install huggingface-api-haystack and update your imports.

Before:

from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
from haystack.components.embedders import HuggingFaceAPITextEmbedder, HuggingFaceAPIDocumentEmbedder
from haystack.components.rankers import HuggingFaceTEIRanker

After:

from haystack_integrations.components.generators.huggingface_api import HuggingFaceAPIChatGenerator
from haystack_integrations.components.embedders.huggingface_api import (
    HuggingFaceAPITextEmbedder,
    HuggingFaceAPIDocumentEmbedder,
)
from haystack_integrations.components.rankers.huggingface_api import HuggingFaceTEIRanker

DocumentLanguageClassifier and TextLanguageRouter have been moved out of Haystack into the langdetect-haystack integration package. Install the new package with pip install langdetect-haystack and update your imports.

Before:

from haystack.components.classifiers import DocumentLanguageClassifier
from haystack.components.routers import TextLanguageRouter

After:

from haystack_integrations.components.classifiers.langdetect import DocumentLanguageClassifier
from haystack_integrations.components.routers.langdetect import TextLanguageRouter

The NamedEntityExtractor component (along with the NamedEntityAnnotation and NamedEntityExtractorBackend classes) has been removed from Haystack and moved into dedicated integration packages, one per backend.

For the Hugging Face backend, install transformers-haystack and update your imports:

Before:

from haystack.components.extractors import NamedEntityExtractor

extractor = NamedEntityExtractor(backend="hugging_face", model="dslim/bert-base-NER")

After:

from haystack_integrations.components.extractors.transformers import TransformersNamedEntityExtractor

extractor = TransformersNamedEntityExtractor(model="dslim/bert-base-NER")

For the spaCy backend, install spacy-haystack and update your imports:

Before:

from haystack.components.extractors import NamedEntityExtractor

extractor = NamedEntityExtractor(backend="spacy", model="en_core_web_trf")

After:

from haystack_integrations.components.extractors.spacy import SpacyNamedEntityExtractor

extractor = SpacyNamedEntityExtractor(model="en_core_web_trf")

The OpenAPIConnector, OpenAPIServiceConnector and OpenAPIServiceToFunctions components have been removed from Haystack and moved into the dedicated openapi-haystack integration package.

Install the package and update your imports:

Before:
```
from haystack.components.connectors import OpenAPIConnector, OpenAPIServiceConnector
from haystack.components.converters import OpenAPIServiceToFunctions
```
After:
```
from haystack_integrations.components.connectors.openapi import OpenAPIConnector, OpenAPIServiceConnector
from haystack_integrations.components.converters.openapi import OpenAPIServiceToFunctions
```
These components are a legacy way to connect Haystack to external APIs. For most use cases, consider using the MCPTool instead, which is the modern, standardized way to give pipelines and agents access to external tools and services.

The OpenTelemetryTracer has been moved out of Haystack into the opentelemetry-haystack integration package, and the opentelemetry-sdk dependency is no longer installed with Haystack. OpenTelemetry tracing is no longer auto-enabled when opentelemetry-sdk is installed; instead, use the new OpenTelemetryConnector component to enable OpenTelemetry tracing in your pipeline. Install the new package with pip install opentelemetry-haystack.

Before:

from opentelemetry import trace
from haystack import tracing
from haystack.tracing import OpenTelemetryTracer

tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))

After:

from opentelemetry import trace
from haystack import tracing
from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer

tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))

Alternatively, add the OpenTelemetryConnector to your pipeline to enable tracing:

from haystack import Pipeline
from haystack_integrations.components.connectors.opentelemetry import OpenTelemetryConnector

pipe = Pipeline()
pipe.add_component("tracer", OpenTelemetryConnector())

Breaking change: haystack.tracing.auto_enable_tracing has been removed (and is no longer called on import haystack). Since Haystack no longer ships a built-in tracing backend, there is nothing to auto-enable. Enable tracing explicitly instead, for example by adding the OpenTelemetryConnector component to your pipeline, or by calling haystack.tracing.enable_tracing(...) with your tracer. The HAYSTACK_AUTO_TRACE_ENABLED environment variable no longer has any effect.
SearchApiWebSearch has been moved out of Haystack into the searchapi-haystack integration package. Install the new package with pip install searchapi-haystack and update your imports.

Before:
```
from haystack.components.websearch import SearchApiWebSearch
```
After:
```
from haystack_integrations.components.websearch.searchapi import SearchApiWebSearch
```

The Sentence Transformers components have been moved out of Haystack into the sentence-transformers-haystack integration package. This includes SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder, SentenceTransformersSparseTextEmbedder, SentenceTransformersSparseDocumentEmbedder, SentenceTransformersDocumentImageEmbedder, SentenceTransformersSimilarityRanker, and SentenceTransformersDiversityRanker. Install the new package with pip install sentence-transformers-haystack and update your imports.

Before:

from haystack.components.embedders import (
    SentenceTransformersTextEmbedder,
    SentenceTransformersDocumentEmbedder,
    SentenceTransformersSparseTextEmbedder,
    SentenceTransformersSparseDocumentEmbedder,
)
from haystack.components.embedders.image import SentenceTransformersDocumentImageEmbedder
from haystack.components.rankers import SentenceTransformersSimilarityRanker, SentenceTransformersDiversityRanker

After:

from haystack_integrations.components.embedders.sentence_transformers import (
    SentenceTransformersTextEmbedder,
    SentenceTransformersDocumentEmbedder,
    SentenceTransformersSparseTextEmbedder,
    SentenceTransformersSparseDocumentEmbedder,
    SentenceTransformersDocumentImageEmbedder,
)
from haystack_integrations.components.rankers.sentence_transformers import (
    SentenceTransformersSimilarityRanker,
    SentenceTransformersDiversityRanker,
)

Removed the unused serialize_class_instance and deserialize_class_instance functions from haystack.utils.base_serialization. These were internal helpers not used anywhere in Haystack. If you were importing them directly, inline their logic: use {"type": generate_qualified_class_name(type(obj)), "data": obj.to_dict()} to serialize, and obj_class.from_dict(data["data"]) (after resolving obj_class with import_class_by_name(data["type"])) to deserialize.
SerperDevWebSearch has been moved out of Haystack into the serperdev-haystack integration package. Install the new package with pip install serperdev-haystack and update your imports.

Before:
```
from haystack.components.websearch import SerperDevWebSearch
```
After:
```
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
```
TikaDocumentConverter has been moved out of Haystack into the tika-haystack integration package. Install the new package with pip install tika-haystack and update your imports.

Before:
```
from haystack.components.converters import TikaDocumentConverter
```
After:
```
from haystack_integrations.components.converters.tika import TikaDocumentConverter
```
Removed the ToolInvoker component. Code that imports ToolInvoker from haystack.components.tools or uses it as a standalone Pipeline component must be updated.

To execute tools in an agentic loop, pass the tools directly to haystack.components.agents.Agent. The Agent now owns tool execution, including tool/toolset warm-up, state injection, streaming callback passthrough, and sync/async tool invocation. The previous tool_invoker_kwargs Agent parameter has been removed; use tool_concurrency_limit and tool_streaming_callback_passthrough instead.
Tool results are now always serialized to a string using json.dumps (with ensure_ascii=False, falling back to str only when the result is not JSON-serializable). The ToolInvoker's convert_result_to_json_string parameter, which defaulted to False and selected str-based serialization, has been removed along with the component. As a result, non-string tool results now appear as JSON rather than Python repr output: for example, a tool returning {"message": "hi"} now produces the string {"message": "hi"} instead of {'message': 'hi'}. Update any code or tests that assert on the previous str-based format.

HuggingFaceLocalChatGenerator, ExtractiveReader, TransformersZeroShotDocumentClassifier, TransformersTextRouter, and TransformersZeroShotTextRouter have been moved out of Haystack into the transformers-haystack integration package. Install the new package with pip install transformers-haystack and update your imports.

Before:

from haystack.components.classifiers import TransformersZeroShotDocumentClassifier
from haystack.components.generators.chat import HuggingFaceLocalChatGenerator
from haystack.components.readers import ExtractiveReader
from haystack.components.routers import TransformersTextRouter, TransformersZeroShotTextRouter

After:

from haystack_integrations.components.classifiers.transformers import TransformersZeroShotDocumentClassifier
from haystack_integrations.components.generators.transformers import TransformersChatGenerator
from haystack_integrations.components.readers.transformers import TransformersExtractiveReader
from haystack_integrations.components.routers.transformers import (
    TransformersTextRouter,
    TransformersZeroShotTextRouter,
)

The legacy TransformersSimilarityRanker component has been removed. Use SentenceTransformersSimilarityRanker instead, which provides the same functionality along with async support.

Before:

from haystack.components.rankers import TransformersSimilarityRanker

ranker = TransformersSimilarityRanker(model="cross-encoder/ms-marco-MiniLM-L-6-v2")

After:

from haystack.components.rankers import SentenceTransformersSimilarityRanker

ranker = SentenceTransformersSimilarityRanker(model="cross-encoder/ms-marco-MiniLM-L-6-v2")

The user_prompt and system_prompt parameters have been removed from Agent.run / Agent.run_async and LLM.run / LLM.run_async. Both prompts must now be set at initialization time on the component; they can no longer be overridden per-run, including when the component is used inside a pipeline (Pipeline.run(data={"agent": {"user_prompt": ...}}) or Pipeline.run(data={"llm": {"user_prompt": ...}}) is no longer supported).

system_prompt and user_prompt both accept either a plain string template or an explicit Jinja2 message template. Plain string templates are rendered as a single message with the expected role (system for system_prompt, user for user_prompt). Explicit Jinja2 message templates must contain a single message block and render to exactly one message with the matching role.

Before:
```
agent = Agent(chat_generator=..., tools=[...], system_prompt="Default system prompt.")
agent.run(messages=[...], system_prompt="Per-run override.")
```
After:
```
# Construct a new Agent or LLM with the desired prompt.
agent = Agent(chat_generator=..., tools=[...], system_prompt="Default system prompt.")
agent.run(messages=[...])

# If you need to build prompts at runtime, construct an Agent without
# init prompts and pass the prompt messages through the messages input.
agent = Agent(chat_generator=..., tools=[...])
agent.run(messages=[ChatMessage.from_system("Runtime system prompt."), ChatMessage.from_user("...")])
```
LocalWhisperTranscriber and RemoteWhisperTranscriber have been moved out of Haystack into the whisper-haystack integration package. Install the new package with pip install whisper-haystack and update your imports.

Before:
```
from haystack.components.audio import LocalWhisperTranscriber
from haystack.components.audio import RemoteWhisperTranscriber
```
After:
```
from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber
from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
```
LocalWhisperTranscriber additionally requires the openai-whisper package (and ffmpeg), which you can install with pip install "openai-whisper>=20231106".
Tools now read a value from the Agent's State by name only when they declare an explicit inputs_from_state mapping. Previously, when a tool had no inputs_from_state, the Agent would implicitly inject a State value into any tool parameter whose name happened to match a State key, on any call where the model did not already supply that argument (LLM-provided arguments always took precedence). This implicit name-matching has been removed. The change applies to every tool type, since ComponentTool, PipelineTool, MCPTool, and others all derive from the base Tool class.

This is a breaking change. If you relied on a tool parameter being filled from a same-named State key, add an explicit mapping, for example inputs_from_state={"documents": "documents"} to feed the documents state key into the tool's documents parameter.

Auto-injection of the full State object into a parameter annotated as State is unchanged and continues to work without any mapping.
The raise_on_invalid_filter_syntax utility function has been removed. Its implementation was incorrect and never used across Haystack and Haystack Core Integrations.
haystack.logging.configure_logging now attaches its formatting handler to Haystack's own logger namespaces (haystack, haystack_integrations and haystack_experimental) instead of the root logger. This makes Haystack a better-behaved library when it runs next to other services in the same process: the log records of the host application and other libraries are no longer reformatted or duplicated by Haystack. To restore the previous behavior of formatting every log record in the process, call configure_logging(logger_name="").
Importing Haystack no longer configures structlog process-wide. The import-time setup now only installs Haystack's own scoped logging handler (so Haystack's logs are still formatted) and leaves the global structlog configuration untouched, so the host application's own native structlog loggers are no longer reformatted by merely importing Haystack. structlog is configured only when you call configure_logging(configure_structlog=True).
The Agent component now emits a different shape of tracing spans. Previously each iteration of the agent loop produced two child spans via Pipeline._run_component — one for the chat generator and one for the tool invoker — each carrying haystack.component.name and haystack.component.type tags. The agent now opens a single haystack.agent.step span per loop iteration with two nested children: haystack.agent.step.llm (the chat generator call) and haystack.agent.step.tool (the tool invocation, only emitted when tool calls happen). These new spans do NOT carry haystack.component.* tags; instead they expose new content tags haystack.agent.step.llm.input/.output and haystack.agent.step.tool.input/.output.

Custom tracer backends or SpanHandler implementations that dispatched on component_type == "ToolInvoker" or component_type.endswith("ChatGenerator") for Agent-internal spans will need to be updated to dispatch on the new operation names (haystack.agent.step.llm / haystack.agent.step.tool) instead. The Langfuse integration's DefaultSpanHandler is affected — see MIGRATION.md for a concrete example of a custom SpanHandler that maps the new operations to Langfuse chain / generation / tool observations.

🚀 New Features

Added an after_tool hook point to the Agent. Hooks registered under after_tool run after tools execute, once their result messages are in state.data["messages"], and before the exit check and the next LLM call. Use it to rewrite the freshly produced tool-result messages, for example to offload, redact, truncate, or summarize results before the next LLM call sees them. It does not run on the plain-text exit step, where no tools run. Register it like any other hook: hooks={"after_tool": [my_hook]}.

Added hooks to the Agent. Pass hooks as a dictionary mapping a hook point to a list of hooks the Agent runs at that point. Each hook receives the live State and influences the run by mutating it in place; hooks for a hook point run in list order, and the same hook can be registered under multiple hook points. Valid hook points are:

before_llm: Runs before each chat-generator call.
before_tool: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from state["messages"]. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless max_agent_steps has been reached.
on_exit: Runs when the Agent is about to stop on an exit condition. An on_exit hook can keep the Agent running by setting the continue_run control flag (state.set("continue_run", True)), usually alongside a message telling the model what to do next. on_exit hooks run when the Agent stops on an exit condition, but not when it stops because max_agent_steps is reached.

A hook is any object with a run(state) method (optionally run_async(state)); use the @hook decorator to build one from a function. This enables patterns such as building run-time system context, retrieving memories, and requiring a tool to be called before finishing.

The example below registers one hook at each placement to show what each can do:

from datetime import datetime, timezone
from typing import Annotated

from haystack.components.agents import Agent
from haystack.components.agents.state import State, replace_values
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks import hook
from haystack.tools import tool

@tool
def search(query: Annotated[str, "The search query"]) -> str:
    """Search the web."""
    # Placeholder: would call a real search API
    return "Fusion startups reported net-energy-gain milestones this year."

@hook
def build_context(state: State) -> None:
    # before_llm: build run-time system context once, before the first model call.
    if state.get("step_count") == 0:
        now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
        system = ChatMessage.from_system(f"You are a research assistant. The current time is {now}.")
        state.set("messages", [system, *state.data["messages"]], handler_override=replace_values)

@hook
def audit_tool_calls(state: State) -> None:
    # before_tool: see which tools the model is about to run.
    pending = state.data["messages"][-1].tool_calls
    print(f"about to run: {[tc.tool_name for tc in pending]}")

@hook
def require_search(state: State) -> None:
    # on_exit: keep going until the agent has actually searched.
    if state.get("tool_call_counts", {}).get("search", 0) == 0:
        state.set("messages", [ChatMessage.from_system("Search before answering.")])
        state.set("continue_run", True)

agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
    tools=[search],
    hooks={
        "before_llm": [build_context],
        "before_tool": [audit_tool_calls],
        "on_exit": [require_search],
    },
)

result = agent.run(messages=[ChatMessage.from_user("What are the latest developments in fusion energy?")])
print(result["last_message"].text)

Class-based hooks may also implement the optional lifecycle methods warm_up / warm_up_async and close / close_async. The Agent calls them from its own warm_up / warm_up_async and close / close_async, so a hook can defer opening clients or reading credentials until warm-up and release them on close. When a class-based hook should be serializable, store serializable constructor arguments on the hook and rebuild runtime clients from those values. For example, an on_exit hook can grade the Agent's answer with its own LLM and ask it to improve a weak answer before finishing, warming the judge's client during the Agent's warm-up:

from typing import Any
from haystack.core.serialization import default_from_dict, default_to_dict

class GradeFinalAnswer:
    """Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing."""

    def __init__(self, model: str = "gpt-5.4-nano"):
        self.model = model
        self._judge = OpenAIChatGenerator(model=self.model)

    def warm_up(self) -> None:
        # Warm up the judge's own client during the Agent's warm-up.
        self._judge.warm_up()

    def run(self, state: State) -> None:
        answer = state.data["messages"][-1].text or ""
        verdict = self._judge.run(
            messages=[ChatMessage.from_user(f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}")]
        )["replies"][0].text or ""
        if "FAIL" in verdict.upper():
            state.set("messages", [ChatMessage.from_user("Your answer was incomplete. Please improve it.")])
            state.set("continue_run", True)

    def to_dict(self) -> dict[str, Any]:
        return default_to_dict(self, model=self.model)

    @classmethod
    def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer":
        return default_from_dict(cls, data)

agent = Agent(chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"), hooks={"on_exit": [GradeFinalAnswer()]})
result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")])
print(result["last_message"].text)

Add native async tool invocation. Tool now accepts an optional async_function (a coroutine function) alongside the existing function field. When Agent.run_async invokes a tool, it now awaits the async_function of Tool if available.

@tool / create_tool_from_function automatically routes async def callables to async_function, so simply decorating an async def produces an async tool:
```
from typing import Annotated
from haystack.tools import tool

@tool
async def weather(city: Annotated[str, "The name of the city"]) -> str:
    """Get the weather for a city."""
    ...
```
ComponentTool automatically wires an async invoker for components that define run_async (__haystack_supports_async__ is True). PipelineTool enables the async path when wrapping an AsyncPipeline and falls back to the threaded sync path when wrapping a regular Pipeline.
Added before_run and after_run hook points to the Agent. Hooks registered under before_run run exactly once per run, after the state is initialized and before the first LLM call — use them to rewrite the initial messages or seed state without re-running on every step like before_llm does. Hooks registered under after_run run exactly once per run, after the step loop has ended and before the Agent builds its return value, regardless of whether the run stopped on an exit condition or because max_agent_steps was reached (unlike on_exit); mutations to the state are reflected in the returned messages, last_message, and state_schema outputs. Register them like any other hook: hooks={"before_run": [my_hook], "after_run": [my_other_hook]}.
Added a new {% insert %} tag to the Jinja2 string templates used by ChatPromptBuilder. The tag is a placeholder that evaluates an expression to a ChatMessage or a list of ChatMessage objects and expands it into the prompt, so messages provided at runtime can be interleaved with literal {% message %} blocks. For example, you can wrap the runtime messages with a system message above and a templated user message below, then pass the messages (and any template variables) at run time:
```
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

template = """
{% message role="system" %}You are a helpful assistant.{% endmessage %}
{% insert messages %}
{% message role="user" %}{{ query }}{% endmessage %}
"""

builder = ChatPromptBuilder(template=template)
result = builder.run(
    messages=[ChatMessage.from_user("Hi"), ChatMessage.from_assistant("Hello!")],
    query="What's the weather?",
)
# result["prompt"] -> [system, user "Hi", assistant "Hello!", user "What's the weather?"]
```
All content types (tool calls, tool call results, images, reasoning, name and meta) round trip without loss. A missing or empty value expands to nothing.

The expression can be a plain variable ({% insert messages %}), a slice or index ({% insert messages[-1:] %}, {% insert messages[-1] %}), or a combination of variables ({% insert previous + current %}). Multiple {% insert %} tags can be used in a single template, so the runtime messages can be split, reordered, or repeated across different positions.
Added MockChatGenerator, a Chat Generator that returns predefined responses without calling any API. It is a deterministic, zero-cost drop-in replacement for real Chat Generators in tests, smoke tests, and quick prototypes. It supports a fixed response, a list of responses cycled across calls (useful to drive Agent-like loops), a response_fn callable for input-dependent replies, and an echo mode (the default) that returns the last user message. It implements the full Chat Generator interface, including run, run_async, streaming callbacks, and serialization.
Added MockTextEmbedder and MockDocumentEmbedder, embedders that return deterministic embeddings without calling any API. They are zero-cost drop-in replacements for real embedders in tests, smoke tests, and quick prototypes. By default each embedding is derived from a stable hash of the (prepared) text, so the same text always yields the same unit-length embedding and different texts yield different embeddings, making the mocks usable in retrieval pipelines and reproducible across runs and processes. A fixed embedding vector or a custom embedding_fn callable can be provided instead. Both implement the full embedder interface, including run, run_async, and serialization.
Added a reasoning_mode flattened generation kwarg to OpenAIResponsesChatGenerator, which is merged into the reasoning dictionary sent to the OpenAI Responses API. This lets you set the reasoning mode (for example standard or pro, supported since GPT-5.6) without having to construct the nested reasoning dict yourself.
Added an include_reasoning_encrypted_content flattened generation kwarg to OpenAIResponsesChatGenerator. Setting it to True appends reasoning.encrypted_content to the include list sent to the OpenAI Responses API, so reasoning items can be reused across multi-turn conversations when the Responses API is used statelessly (for example when store is False or zero data retention is enabled).
Added a run_async method to LLMRanker, QueryExpander, LLMMessagesRouter and LLMDocumentContentExtractor so they can run natively in an AsyncPipeline. The method delegates to the chat generator's run_async method when available. If the chat generator only implements a synchronous run method, it is executed in a thread to avoid blocking the event loop. LLMDocumentContentExtractor processes documents concurrently with asyncio, bounded by max_workers.
Added a run_async method to RemoteWhisperTranscriber and OpenAIImageGenerator so they can run natively in an AsyncPipeline. Both components now hold an AsyncOpenAI client alongside the existing synchronous client and call the OpenAI API without blocking the event loop.
Added SkillToolset, a Toolset that lets an Agent discover and read "skills" through progressive disclosure, similar to how Claude Code and Codex expose skills. A skill is a directory containing a SKILL.md file with YAML frontmatter (name and description) and a markdown body of instructions, optionally bundling additional reference files. SkillToolset is backed by a SkillStore; use FileSystemSkillStore to load skills from a local directory and add the toolset to an Agent:
```
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.tools import SkillToolset
from haystack.skill_stores.file_system import FileSystemSkillStore

skills_toolset = SkillToolset(FileSystemSkillStore("skills/"))
agent = Agent(chat_generator=OpenAIChatGenerator(), tools=skills_toolset)
```
Skills are discovered when the toolset is warmed up (the Agent does this automatically before a run). The names and descriptions of all discovered skills are baked into the load_skill tool description, so the model can see which skills exist without any system prompt injection. The toolset exposes two tools: load_skill returns a skill's full instructions on demand, and read_skill_file reads a file bundled with a skill (with path-traversal protection). read_skill_file returns text files as strings, images as ImageContent, and PDFs as FileContent, so an Agent backed by a multimodal chat generator can read a skill's visual assets — for example a reference screenshot or a showcase PDF — directly.

Custom stores can back SkillToolset with any storage system (a database, a remote API, ...) by implementing the SkillStore protocol.
Added tool result offloading for the Agent via the new haystack.hooks.tool_result_offloading module. ToolResultOffloadHook is an after_tool hook that writes selected tool results to a store and replaces them in the conversation with a compact pointer (a reference plus a short preview). The next LLM call sees the reference instead of the full result, which helps manage the context window and is a step towards letting an Agent operate on offloaded results with follow-up tools (e.g. a bash tool that reads the referenced files).

Configure what gets offloaded, per tool, via offload_strategies:
- Map a tool name, a tuple of tool names, or the wildcard "*" to an OffloadPolicy. More specific keys win over "*"; a tool with no matching key is not offloaded.
- Built-in policies: AlwaysOffload, NeverOffload, and OffloadOverChars (offload when the result exceeds a character threshold).
- For custom conditions, implement the OffloadPolicy protocol (it provides default to_dict/from_dict, so a stateless policy only needs a should_offload method).
Choose where results are stored:
- FileSystemToolResultStore writes results to the local file system.
- Implement the ToolResultStore protocol to target other backends.
- The hook is stateless and can be shared across concurrent runs. In a multi-user server, pass an isolated store per run via the Agent's hook_context (hook_context={RESULT_STORE_CONTEXT_KEY: per_request_store}) so concurrent users write to separate locations and never read each other's results.
What is offloaded:
- Only successful, text results are offloaded. Error results (including before_tool human-in-the-loop rejections) are left in context.
- Non-text results (image or file content) are left in context; supporting only text is a deliberate choice for now. A warning is logged when a non-text result has a matching offload policy.
- Each result is offloaded at most once.
```
from typing import Annotated

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks.tool_result_offloading import (
    FileSystemToolResultStore,
    OffloadOverChars,
    ToolResultOffloadHook,
)
from haystack.tools import tool

@tool
def search(query: Annotated[str, "The search query"]) -> str:
    """Search the web and return the (potentially large) results."""
    return f"... large result for {query} ..."

offload_hook = ToolResultOffloadHook(
    store=FileSystemToolResultStore(root="tool_results"),
    offload_strategies={"*": OffloadOverChars(4000)},
)
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
    tools=[search],
    hooks={"after_tool": [offload_hook]},
)
agent.run(messages=[ChatMessage.from_user("Summarize today's tech news")])
```

Added AsyncPipeline.stream(), a convenient way to stream LLM chunks from an async application, such as an API endpoint. It runs the pipeline and returns a handle that can be iterated with async for to receive StreamingChunk objects from streaming-capable components as they are produced. After iteration ends, handle.result holds the final pipeline output (the same dict returned by run_async). By default every streaming-capable component is forwarded; pass streaming_components to stream only specific components. If the consumer abandons iteration, the underlying pipeline run is cancelled automatically; pass cancel_on_abandon=False to let it run to completion.

import asyncio

from haystack import AsyncPipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

pipe = AsyncPipeline()
pipe.add_component(
    "prompt_builder",
    ChatPromptBuilder(template=[ChatMessage.from_user("Tell me about {{topic}}")]),
)
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("prompt_builder.prompt", "llm.messages")

async def main():
    handle = pipe.stream(data={"prompt_builder": {"topic": "Italy"}})
    async for chunk in handle:
        print(chunk.content, end="", flush=True)
    return handle.result

result = asyncio.run(main())

Components and pipelines now have a consistent resource lifecycle for acquiring and releasing resources such as HTTP clients and connections. In addition to the existing warm_up, components can now implement warm_up_async to acquire asynchronous resources, and close/close_async to release them. Pipeline now implements the same lifecycle methods. In addition to the existing warm_up, warm_up_async() runs async warm-up on the serving event loop (falling back to the synchronous warm_up), while close()/close_async() release synchronous and asynchronous resources respectively. Closing is always explicit; pipelines are never closed automatically after a run.
Added a shared parameter to InMemoryDocumentStore. When True (the default, matching previous behavior), documents live in process-global storage shared across instances that use the same index. When False, documents are kept instance-local and freed when the instance is garbage collected. Use shared=False for stores that are created frequently (for example, one per request in a served application) to avoid unbounded growth of the process-global storage.
Pipeline.load / Pipeline.loads / Pipeline.from_dict now accept two new keyword arguments:
- allowed_modules: list[str] | None — additional module patterns that may be imported during this deserialization call. Patterns are matched as prefixes ("mypkg" matches mypkg and any submodule) or, if they contain */?/[, using fnmatch rules.
- unsafe: bool = False — when set to True, the deserialization allowlist is bypassed entirely. Only use this when you fully trust the source of the serialized data.
A new public helper, haystack.core.serialization.allow_deserialization_module(pattern), extends the process-wide allowlist. The HAYSTACK_DESERIALIZATION_ALLOWLIST environment variable (comma-separated patterns) is read at runtime on every deserialization call.
Added ConfirmationHook, a before_tool Agent hook that applies Human-in-the-Loop confirmation strategies to pending tool calls. It confirms, modifies, or rejects the tool calls the model requested before they run by rewriting the conversation in the Agent's State. Its confirmation_strategies mapping accepts a single tool name, a tuple of tool names, or the wildcard "*" that applies to any tool without a more specific entry, so you can set a default strategy for all tools and override individual ones.
Agent hooks may now declare an allowed_hook_points attribute listing the hook points they support. The Agent validates it at construction and raises a ValueError if such a hook is registered under an unsupported hook point. For example, ConfirmationHook sets allowed_hook_points = ("before_tool",), so registering it under before_llm or on_exit now fails fast instead of silently doing nothing. Hooks that do not declare the attribute can still be registered under any hook point.
haystack.logging.configure_logging gained three parameters: logger_name to choose which logger(s) the formatting handler is attached to, propagate to control whether Haystack's loggers propagate their records to ancestor loggers, and configure_structlog to control whether the call configures the process-global structlog (taking over any existing configuration). Set propagate=False to let Haystack fully own the output of its own logs and avoid duplicate log lines when the host application also configures the root logger.

⚡️ Enhancement Notes

Adds a status field to each result produced by ContextRelevanceEvaluator and FaithfulnessEvaluator. The field is evaluated for valid results and error for generation or parsing failures continued with raise_on_failure=False.
Components that delegate to sub-components in their run_async method (LLMMetadataExtractor, EmbeddingBasedDocumentSplitter, FallbackChatGenerator, TextEmbeddingRetriever, MultiRetriever, MultiQueryTextRetriever and MultiQueryEmbeddingRetriever) now use a shared utility to run the sub-component. If the sub-component only implements a synchronous run method, it is executed in a thread to avoid blocking the event loop. Notably, LLMMetadataExtractor and EmbeddingBasedDocumentSplitter previously fell back to their fully synchronous run method in this case, which blocked the event loop for the duration of the call.
Duplicate tool names are now detected at the start of each Agent step (before the chat generator call) instead of only at tool execution, so the error surfaces earlier and the LLM is never sent duplicate tool schemas.
Pipeline.run now accepts break_point and pipeline_snapshot at the same time. This enables a stepped debugging experience: run until a breakpoint, then resume from the resulting snapshot with a new breakpoint on the next component, pausing again for inspection at each step. The only rejected combination is a break_point targeting the same component and visit count as the snapshot's break point, since it would trigger again before the resumed component runs and the pipeline could not make any progress.
ChatMessage.from_dict now also accepts the format Pydantic produces when it auto-serializes ChatMessage as a plain dataclass (e.g. via model_dump on a Pydantic model with ChatMessage fields). In this format, content parts appear without their wrapping key (for example {"tool_name": "search", "arguments": {}} instead of {"tool_call": {"tool_name": "search", "arguments": {}}}) and are identified by their required field names. This makes it possible to round-trip ChatMessage objects that were implicitly serialized as part of larger Pydantic models.
Removed duplicated class-import logic between core/serialization.py and utils/type_serialization.py, including the deserialization-allowlist and builtin-type-safety checks, by sharing a single internal implementation. No change to public behavior or to the security checks themselves.
Agent now exposes three new outputs that are populated automatically during a run and made available alongside messages and last_message in the result dict:
- step_count (int): the number of steps the agent ran.
- token_usage (dict[str, Any]): aggregated token usage summed across every LLM call in the run
- tool_call_counts (dict[str, int]): number of times each tool was invoked, keyed by tool name.
These fields are added to Agent.state_schema automatically so that tools registered via inputs_from_state can read them mid-run. They are exposed only as Agent outputs so cannot be passed in as inputs to Agent.run / Agent.run_async.
LLM now exposes a token_usage output alongside messages and last_message. Because LLM never invokes tools and always runs exactly one step, step_count and tool_call_counts inherited from Agent are not exposed on LLM.
The haystack.tools package now imports its heavier modules lazily. Importing the package no longer eagerly pulls in ComponentTool, PipelineTool, and SearchableToolset (and their transitive dependencies such as Pipeline and InMemoryDocumentStore); these are resolved on first access instead. This reduces import time when those tools are not used. All public names remain importable from haystack.tools exactly as before.
The Agent now emits one haystack.agent.step.tool tracing span per tool call instead of a single span grouping every tool call in a step. Each span carries the tool's name and description as tags, plus the call arguments and result when content tracing is enabled. This matches the OpenTelemetry and Langfuse convention of a single tool call per span, so traces render one observation per tool invocation.
LoggingTracer now serializes the log records it emits when a span closes, so a span's operation name and its tags stay contiguous even when spans are opened and closed from parallel threads (for example, the Agent running tool calls concurrently).

Streaming-capable components (OpenAIChatGenerator, OpenAIResponsesGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator, Agent) now accept a synchronous streaming callback in asynchronous contexts (run_async). A warning is logged because the callback runs synchronously on the event loop and may block it, but the run proceeds. Async callbacks are still preferred for performance.

import asyncio

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage, StreamingChunk

def print_chunk(chunk: StreamingChunk) -> None:
    print(chunk.content, end="", flush=True)

async def main():
    llm = OpenAIChatGenerator()
    # previously this raised; now it logs a warning and streams as expected
    await llm.run_async(
        [ChatMessage.from_user("Tell me about Italy")],
        streaming_callback=print_chunk,
    )

asyncio.run(main())

Simplified the pipeline scheduler by removing the DEFER_LAST component priority. Components that previously got DEFER_LAST (those with unresolved lazy-variadic sockets) now share the DEFER priority and are tie-broken via topological order, matching the behavior of all other deferred components. This should not materially change how components are scheduled in a pipeline.
ToolCallResult can now carry FileContent parts, in addition to TextContent and ImageContent. A Tool can return a file (such as a PDF) as its result so a multimodal chat generator that supports file inputs (for example OpenAIResponsesChatGenerator) passes it straight to the model. Combine this with a tool's outputs_to_string={"raw_result": True} to keep the content parts intact instead of converting them to a string.
Toolset gained two methods: get_selectable_tools() returns every tool available for name-based selection (ignoring any active selection restriction, e.g. the one used by SearchableToolset), and spawn() returns an isolated, run-scoped copy of the Toolset. Subclasses with additional run-scoped state can override spawn().
SearchableToolset now exposes its full catalog of tools via SearchableToolset.get_selectable_tools().
Toolset.warm_up() is now idempotent: warming up a toolset more than once no longer re-warms its tools. Calling Toolset.add() after the toolset has been warmed up now warms up the newly added tool (or toolset) immediately, so it is ready to use without an extra warm_up() call.
Removed the use of Pipeline._run_component inside Agent.run and Agent.run_async. The chat generator and tool invoker are now invoked directly via their run / run_async methods, decoupling the Agent from the Pipeline internals.

🔒 Security Notes

Fixed a deserialization-allowlist bypass that could allow arbitrary code execution when loading an untrusted pipeline (Pipeline.load / Pipeline.loads / Pipeline.from_dict). The module allowlist was enforced against a string prefix of the serialized handle, while the callable/class resolvers walked attributes from the deepest importable module. Because an allowlisted package can expose another module as an attribute (e.g. a module-scope import os makes haystack.utils.auth.os the standard-library os module), a handle such as haystack.utils.auth.os.system passed the allowlist and resolved to os.system. The allowlist is now enforced against the module each object is actually resolved from — every module hop during attribute resolution is re-checked, and the resolved callable's/class's real module must be on the allowlist. Legitimate handles (including harmless builtins such as builtins.print) continue to resolve.
Loading a pipeline from YAML used to dynamically import any class referenced in the file, which made a crafted YAML capable of causing arbitrary classes to be imported and instantiated by Pipeline.from_dict. Deserialization is now gated by an allowlist of trusted module namespaces (see the upgrade note for how to extend it). In addition, default_from_dict now refuses to recurse into nested {"type": "..."} dictionaries keyed by parameter names that the parent class does not actually accept — blocking attempts to smuggle untrusted classes into unused parameter slots.
Dangerous builtins such as eval, exec, compile, __import__, open, and getattr can no longer be resolved during deserialization, even though builtins is on the default allowlist. Attempting to deserialize one as a callable, a type annotation, or a nested {"type": ...} class reference raises a DeserializationError. The block can be bypassed with unsafe=True when the source of the serialized data is fully trusted.

🐛 Bug Fixes

Agent now correctly executes tools supplied only at runtime via run(tools=...) / run_async(tools=...) when no tools were provided at initialization. Previously the run stopped after the first LLM call because the step guard checked the init-time tools instead of the tools resolved for the run.
Initializing an Agent no longer warms up a SearchableToolset. Previously construction could iterate the tools, and SearchableToolset warms itself up when iterated, so it was warmed up too early. Its warm-up now happens only via Agent.warm_up() or at run time.
The haystack.agent.run tracing span's haystack.agent.tools tag now reflects the tools actually used for the run. Previously it always reported the init-time tools, so it was inconsistent with the run when tools were overridden at runtime via run(tools=...) / run_async(tools=...).
Fixed a bug in Agent (and the underlying tool invocation) where parallel tool calls in a single step did not respect read/write dependencies on State. Tool arguments were prepared once, up front, from the initial state, so a tool reading a State key via inputs_from_state never observed a write made by another tool in the same step via outputs_to_state. For example, two calls to a retrieval tool that both reads and writes the documents key would each see the original documents instead of the second call seeing the first call's results.

Tool calls within a step are now scheduled into batches based on their inputs_from_state and outputs_to_state mappings. Calls that only read, or that touch disjoint keys, still run in parallel; a call that reads a key another call writes is run in a later batch, after that write has been merged into State, with its arguments re-prepared so it sees the updated value. Tools that receive the live State object are treated as reading and writing every key and run on their own. The order the LLM requested the calls in no longer affects correctness.
AnswerBuilder now treats a [0] document reference as out of range instead of silently attaching the last document. References are 1-based, so [0] produced a negative index that bypassed the out-of-range guard, marking an unrelated document as referenced with a wrong source_index. The same applied to expanded reference ranges starting at 0 (e.g. [0-1]).
Fixed a task leak in AsyncPipeline when running components concurrently. Previously, if one component raised an error while sibling components were still running, those in-flight tasks were neither awaited nor cancelled and kept running in the background until the event loop was torn down. They are now cancelled and drained before the original error is re-raised. The same cleanup now also applies when iteration of run_async_generator is stopped early (e.g. the consumer breaks out of the loop and closes the generator) or the run is cancelled: any tasks still in flight are cancelled instead of leaking. Note that cancellation only interrupts components that run natively async. Sync components are offloaded to a worker thread, which cannot be interrupted and runs to completion in the background; its outputs are discarded, so pipeline state stays consistent, but the component's side effects still complete.
ComponentTool no longer adds a top-level description key to the generated parameters JSON schema. The wrapped component's docstring was emitted inside the parameters schema, where LLM providers (OpenAI, Anthropic, Google) do not use it, and for tools wrapping a SuperComponent it produced a confusing auto-generated summary. The parameters schema now only carries per-parameter descriptions, matching plain Tool and function-based tools. The tool-level description is unchanged.
Fixed CSVToDocument in conversion_mode="row" crashing the entire batch when a CSV row has more fields than the header (ragged rows, e.g. an unquoted comma inside a value). Surplus values are now stored under an explicit extra_columns metadata key instead of the None key, which previously broke Document id generation and discarded results from all sources.
Declare the meta output of LLMEvaluator, ContextRelevanceEvaluator, and FaithfulnessEvaluator, and the individual_scores output of ContextRelevanceEvaluator. These outputs were already returned but not declared, so they could not be connected to other components in a Pipeline.
Fixed DocumentRecallEvaluator returning wrong scores when the configured document_comparison_field is missing or empty on some documents. Those documents were considered a match for each other, so unrelated documents could count as correctly retrieved. They are now ignored when computing the score.
Improved the validation and compatibility layer of Document:
- since Haystack 2.x, embedding is a list of floats, while it was a NumPy array in 1.x, so a NumPy embedding is converted to a list for backward compatibility. The conversion only happened when embedding was passed as a keyword argument: passed positionally, the array was silently left as is, potentially causing downstream issues.
- content type was validated only when passed as a keyword argument.
- content type validation was also skipped when the value was falsy, such as 0 or [], and failed later with an unrelated error.
Fix FilterPolicy.MERGE in InMemoryBM25Retriever and InMemoryEmbeddingRetriever so initialization filters are combined with runtime comparison filters instead of being silently overwritten.
Fixed a TypeError raised when filtering documents with the ordering operators (>, >=, <, <=) where both the filter value and the document meta value are datetime objects but one is timezone-aware and the other is naive. The naive/aware reconciliation is now applied to any pair of datetime operands, matching the behavior already used for ISO 8601 string dates.
Fixed GeneratedAnswer.from_dict and ExtractedAnswer.from_dict mutating their input dictionary in place. They replaced nested values (documents, document, offsets and meta["all_messages"]) directly inside the passed dictionary, so deserializing the same dictionary a second time received already-parsed objects and raised an AttributeError. Both methods now copy their input before deserializing and are side-effect free.
Fixed component_to_dict serialization validation not detecting non-string keys in nested dictionaries. The check only inspected the top-level dictionary, so a non-string key inside a nested init_parameters value passed validation and could later fail during marshalling; nested dictionaries are now validated too.
Fixed CountUniqueMetadataByFilterTest.test_count_unique_metadata_by_filter_with_multiple_filters in the document store testing mixins: it called count_documents_by_filter instead of count_unique_metadata_by_filter. The test now exercises the intended method with a compound filter, matching its async counterpart.
Fixed _deepcopy_with_exceptions crashing when a value is a namedtuple (or any typing.NamedTuple). The list/tuple/set branch rebuilt the container with type(obj)(<generator>), but a namedtuple's __new__ expects its fields as positional arguments rather than a single iterable, so copying a component input or parameter that contained a namedtuple raised a TypeError. Namedtuples are now rebuilt by unpacking their deep-copied fields.
Fixed default_from_dict (and therefore component_from_dict) mutating the caller's data dictionary in place when deserializing nested objects such as Secret or ComponentDevice. The serialized sub-dictionaries were being replaced by their deserialized instances inside the input dict; the input is now left unchanged, so the same data can be safely deserialized more than once.
Fixed deserialize_callable failing for functions decorated with @tool when the decorated function is asynchronous. Such tools store the callable in async_function with function set to None, which previously caused deserialization to raise a DeserializationError. The underlying coroutine function is now resolved correctly.
Fixed DocumentCleaner wiping the middle page of a 3-page document when remove_repeated_substrings=True. Header/footer detection compares the pages between the first and last, which is a single page for a 3-page document; its own longest ngram was then treated as a repeated header/footer and removed. Detection now requires at least two pages to compare, so unique content is preserved while genuine repeated headers/footers are still removed.
Fixed DocumentTypeRouter misrouting documents whose MIME type contains a regex metacharacter. A declared type such as the standard IANA image/svg+xml was compiled as a regex, so the + was treated as a quantifier and the document fell into unclassified instead of its own bucket. Declared MIME types are now matched by exact equality first, falling back to regex matching so patterns like audio/.* keep working.
EmbeddingBasedDocumentSplitter now sets split_idx_start in the metadata of every output chunk, consistent with DocumentSplitter and RecursiveDocumentSplitter. Previously the field was absent, causing a KeyError in any downstream code that reads character offsets from splitter output (e.g. highlighting, provenance tracking, overlap recovery).
Fixed FileToFileContent sharing a single extra dictionary across all produced FileContent objects when extra is passed as one dict (or left as None). Each FileContent now gets its own copy, so mutating one file's extra no longer leaks into the others.
Fixed a bug where configuring an Agent confirmation strategy could make a tool run with stale arguments. HITL confirmation prepared and baked each tool's final arguments up front (injecting inputs_from_state values from the state at the start of the tool calling step), which defeated the per-batch argument preparation in tool execution: a tool that reads a state key written by another tool in the same step would run with the pre-step value instead of the freshly produced one. HITL confirmation now operates on the model-requested arguments and leaves state injection to tool execution, so dependent tools again receive the correct values.
Fixed a KeyError raised by the human-in-the-loop confirmation strategies when a chat model hallucinated a tool call whose name is not in the agent's tools. Such tool calls now bypass confirmation and pass through unchanged, so the tool-calling code reports the unknown tool uniformly via ToolNotFoundException (respecting raise_on_failure) instead of crashing.
Fixed JSONConverter crashing on a malformed JSON source when it is configured with content_key and no jq_schema. The non-jq branch called json.loads without a guard, so a single invalid .json file raised an unhandled JSONDecodeError and aborted the whole run. Invalid JSON is now caught, logged, and skipped, matching the behavior of the jq_schema path.
Fixed MarkdownHeaderSplitter dropping content when keep_headers=False. The secondary split re-stripped a leading header with a non-anchored search, which matched the first # line anywhere in the body (an embedded lower-level header, or a # comment inside a fenced code block) and discarded everything before it. The match is now anchored to the start of the content, so only a genuine leading header is stripped.
Fix the max_runs_per_component limit on Pipeline and AsyncPipeline so that a component is now stopped when its number of runs reaches max_runs_per_component rather than when it strictly exceeds it. Previously, a component could run one more time than the configured limit before PipelineMaxComponentRuns was raised. This only affected pipelines with loops. The check is also skipped for BLOCKED components, since they cannot run anyway and should not trigger the error.
Fixed ChatMessage.from_openai_dict_format crashing on zero-argument tool calls. OpenAI-compatible servers (vLLM, llama.cpp, Ollama, LM Studio, ...) may send an empty string or omit the arguments field for a tool call that takes no arguments; this previously raised json.JSONDecodeError or KeyError. Such tool calls are now parsed into a ToolCall with empty arguments.
request_with_retry and async_request_with_retry now honor a custom timeout on every retry attempt. Previously the timeout was consumed on the first attempt and any subsequent retry silently fell back to the default of 10 seconds.
SearchableToolset now builds its internal BM25 InMemoryDocumentStore with shared=False, so its search index is freed together with the toolset instead of accumulating in InMemoryDocumentStore's process-global storage. Previously, repeatedly constructing a SearchableToolset (for example, building an Agent per request in a served application) leaked one storage partition per instance for the lifetime of the process.
Document.id is now deterministic regardless of the insertion order of keys in meta. Previously the hash was built from dict's repr, which reflects insertion order, so two documents with the same content and the same meta could get different IDs depending on how the meta dict was constructed. This silently broke DuplicatePolicy.SKIP / FAIL and any cache or dedup table keyed on the document ID whenever upstream code produced meta in different orders.
Fixed MetaFieldGroupingRanker crashing with an uncaught TypeError when a group contained sort_docs_by values of mutually non-comparable types (for example an int and a str). The sort is now wrapped in a try/except, so in that case the group's original insertion order is kept and a warning is logged, mirroring the behavior of MetaFieldRanker.
Remove zero-frequency vocabulary entries after deleting documents from InMemoryDocumentStore so previous document history no longer changes BM25Okapi scores for the active corpus.
haystack.logging.configure_logging no longer freezes the log level at import time. Previously the level was captured once when the function ran (at import haystack), so a log level set later by the application was ignored for structlog-native loggers. The effective level is now read on every log call.
Fixed the logger in haystack.utils.requests_utils being named after the module's file path instead of haystack.utils.requests_utils, which kept its records outside the haystack logger namespace.
haystack.logging.getLogger is now idempotent. Previously, calling it more than once for the same logger name wrapped the already-wrapped logger methods again, which caused the log message to be run through str.format once per call. As a result a field value containing {...} could be re-interpolated and pull in the value of another field. Each logger is now patched only once.
The patched findCaller used to determine the source of a log entry no longer prints to stdout and no longer masks errors with a misleading NameError, matching structlog's own findCaller implementation.
SearchableToolset now raises a ValueError during warm_up() when its catalog contains tools with duplicate names. Previously duplicates were silently collapsed, which could cause a search hit to resolve to the wrong tool.
SearchableToolset serialization now preserves the catalog shape: a catalog passed as a single Toolset round-trips back to a Toolset (previously it was always rebuilt as a list). Serialization now reuses serialize_tools_or_toolset / deserialize_tools_or_toolset_inplace.
Runtime tool-name selection via Agent.run(tools=["tool_a", "tool_b"]) now resolves correctly when a SearchableToolset is configured. Previously the SearchableToolset was flattened into a single-item list (just its search tool), which broke its search and lazy-loading behavior.
A Toolset is no longer mutated in place during an Agent run. Each run now operates on an isolated per-run copy of the configured Toolset (via the new Toolset.spawn() method). This makes concurrent runs that share the same Toolset instance safe: per-run state such as the active tool-name selection, and a SearchableToolset's discovered tools, can no longer leak or collide across runs.
Agent now warms up tools and toolsets passed at runtime via Agent.run(tools=...) / Agent.run_async(tools=...), not only the tools provided at initialization.
Combining two toolsets with + produces an internal _ToolsetWrapper that can now be serialized and deserialized correctly. to_dict()/from_dict() preserve each wrapped toolset (via its own to_dict()), and the wrapper's warm_up() is now idempotent.

💙 Big thank you to everyone who contributed to this release!

@Aarkin7, @Amnah199, @anakin87, @bilgeyucel, @bogdankostic, @camgrimsec, @charliesheh, @chuenchen309, @davidsbatista, @eeshsaxena, @greymoth-jp, @hoangsonww, @hxaxd, @immuhammadfurqan, @jialiuyang, @jinhaosong-source, @julian-risch, @milljer, @mustafakhokhar, @omborda2002, @Osamaali313, @Otis0408, @RajanChavada, @rautaditya2606, @SeCuReDmE-main-dev, @sjrl, @tstadel, @Vedant-Agarwal, @vidigoat

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

v3.0.0

Choose a tag to compare

Sorry, something went wrong.

Sorry, something went wrong.

Uh oh!

No results found

⭐️ Highlights

🤖 A more capable, adaptable `Agent`

🔎 Built-in introspection and observability

⚡ A core built for serving: first-class async

🧱 A leaner, faster-moving framework ⚠️

💬 Chat Generators everywhere (legacy Generators removed) ⚠️

🔒 Safer pipeline loading ⚠️

🧪 Testing without the API bill

A few smaller breaking changes to know about

⬆️ Upgrade Notes

🚀 New Features

⚡️ Enhancement Notes

🔒 Security Notes

🐛 Bug Fixes

💙 Big thank you to everyone who contributed to this release!

Contributors

Uh oh!

Uh oh!

v3.0.0

⭐️ Highlights

🤖 A more capable, adaptable Agent

🔎 Built-in introspection and observability

⚡ A core built for serving: first-class async

🧱 A leaner, faster-moving framework ⚠️

💬 Chat Generators everywhere (legacy Generators removed) ⚠️

🔒 Safer pipeline loading ⚠️

🧪 Testing without the API bill

A few smaller breaking changes to know about

⬆️ Upgrade Notes

🚀 New Features

⚡️ Enhancement Notes

🔒 Security Notes

🐛 Bug Fixes

💙 Big thank you to everyone who contributed to this release!

Contributors

Uh oh!

🤖 A more capable, adaptable `Agent`