docs: add Agent Hooks page and recast Human-in-the-Loop as a before_tool hook - #11878
Merged
Conversation
…ool hook Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
4 tasks
sjrl
reviewed
Jul 8, 2026
|
|
||
| - `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run. | ||
| - `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`). | ||
| - `hook_context`: A dict of request-scoped resources made available to [hooks](./hooks.mdx) via `state.get("hook_context")` — for example, a user ID or a WebSocket connection. |
Contributor
There was a problem hiding this comment.
I know this is still in draft but want to catch this now. The recommended pattern for getting hook_context is state.data["hook_context"] or state.data.get("hook_context"). The plain state.get method does a deep copy of the object before handing it over which will often fail for the types of items stored in this dict (e.g. a WebSocket)
State.get returns a deep copy, which often fails for the resources stored in hook_context (e.g. a WebSocket). Addresses review feedback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sjrl
reviewed
Jul 9, 2026
|
|
||
| - `before_llm`: Runs before each chat-generator call. | ||
| - `before_tool`: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from `state["messages"]`. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless `max_agent_steps` has been reached. | ||
| - `after_tool`: Runs after tools execute, once their result messages are in `state["messages"]`, before the exit-condition check and the next LLM call. Use it to rewrite the freshly produced tool-result messages — for example, to offload, redact, truncate, or summarize results. It does not run on the plain-text exit step, where no tools run. |
Contributor
There was a problem hiding this comment.
Lets use a programmatically correct statement for access so state.data["messages"] or state.get("messages")
sjrl
reviewed
Jul 9, 2026
| Haystack ships two ready-made hooks: | ||
|
|
||
| - `ConfirmationHook`: A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx). | ||
| - `ToolResultOffloadHook`: An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. Import it from `haystack.hooks.tool_result_offloading`. |
Contributor
There was a problem hiding this comment.
Not sure if needs to be mentioned here, but eventually when the dedicated page for tool result offloading is made we need to make it clear some sort of tool capable of reading the offloaded tool result is needed. E.g. a BashTool would be a good choice when using the default FileSystemToolResultStore
- Use programmatically correct state access: state.data["messages"] - Note that reading offloaded tool results requires a file-reading tool Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Update Agent serialization YAML examples to current to_dict() output (hooks, tool_concurrency_limit, tool_streaming_callback_passthrough) - Replace the removed tool_invoker_kwargs parameter with tool_concurrency_limit and tool_streaming_callback_passthrough in the Agent parameter list and the multi-agent parallelism docs - Document how ConfirmationHook forwards hook_context to each strategy's run() as confirmation_strategy_context (HITL Hayhooks section) - Clarify that after_tool also fires when a before_tool hook removed the pending tool calls - Note that run-metadata keys are reserved too and link the State page from the Hooks state-keys section - Add close() to the GradeFinalAnswer lifecycle example - Link the Hooks API reference from the Hooks page; add Hooks to the Agent page's Additional References - Update the LangGraph migration table HITL row to ConfirmationHook Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
julian-risch
marked this pull request as ready for review
July 9, 2026 19:15
julian-risch
requested review from
davidsbatista and
sjrl
and removed request for
a team and
davidsbatista
July 9, 2026 19:15
sjrl
reviewed
Jul 10, 2026
| The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui). | ||
|
|
||
| The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that uses `confirmation_strategy_context` to pass per-request resources - a Redis client and an async event queue - into the strategy at runtime: | ||
| The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that receives per-request resources - a Redis client and an async event queue - at runtime. Pass such resources via the generic `hook_context` run argument (`agent.run(messages=[...], hook_context={"redis": client})`). `ConfirmationHook` reads this dict from state with `state.data["hook_context"]` (not `state.get`, which returns a deep copy that fails for live resources like clients and queues - see [Hooks](./hooks.mdx)) and passes it to each strategy's `run()` as the `confirmation_strategy_context` keyword argument, which is how a custom strategy receives the Redis client and event queue: |
Contributor
There was a problem hiding this comment.
The repo we reference here will need to be updated to work with Haystack 3.0
Member
Author
There was a problem hiding this comment.
Created an issue https://github.com/deepset-ai/haystack-private/issues/456
Member
Author
There was a problem hiding this comment.
And opened a draft PR: deepset-ai/hitl-hayhooks-redis-openwebui#2
sjrl
approved these changes
Jul 10, 2026
sjrl
left a comment
sjrl
left a comment
Contributor
There was a problem hiding this comment.
One minor comment that should be addressed as a separate PR. Otherwise looks good!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
after_toolhook needed for tool result offloading #11843Proposed Changes:
after_toolhook needed for tool result offloading #11843 — new "Hooks" page under Agents: thehooksdict onAgent, the four hook points and their run-loop semantics, hook-point validation incl.allowed_hook_points, the reserved state keys (continue_run,tools,hook_context), the@hookdecorator (multi-hook example), class-based hooks withwarm_up/closelifecycle andto_dict/from_dictserialization (LLM-judgeon_exitexample), and the ready-madeConfirmationHookandToolResultOffloadHook.ConfirmationHook(tuple-key and"*"-wildcard strategy mapping) replaces the removedconfirmation_strategiesAgent parameter in all examples; the Hayhooks/Open WebUI section documents thehook_contextrun argument and howConfirmationHookforwards it to each strategy'srun()asconfirmation_strategy_context; notes that strategies see only the model-requested tool arguments. The Agent page swapsconfirmation_strategiesforhooksand addshook_contextto the runtime arguments.multi-agent-systems.mdxand LangGraph migration page: serialization YAML examples regenerated from actualAgent.to_dict()output (hooks,tool_concurrency_limit,tool_streaming_callback_passthrough), and the removedtool_invoker_kwargsreplaced everywhere.after_toolalso fires when abefore_toolhook strips the pending tool calls, run-metadata keys documented as reserved with a link to State,close()added to the lifecycle example, and Hooks API-reference links added.How did you test it?
Agentrun withMockChatGeneratorand one hook per hook point: firing counts per hook point,hook_context/toolsreadable from state,ValueErroron unknown hook points,allowed_hook_pointsfailing fast, tuple/wildcard strategy keys, andconfirmation_strategies/confirmation_strategy_contextgone from theAgentAPI. YAML examples regenerated from realAgent.to_dict()output.Notes for the reviewer
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.🤖 Generated with Claude Code