From b0dcd02be79825db6ec721f1645f907a053563a4 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 11:37:41 -0400 Subject: [PATCH 01/11] initial fixes --- src/oss/langchain/agents.mdx | 8 +- src/oss/langchain/context-engineering.mdx | 504 +++++++++++++++++----- src/oss/langchain/middleware.mdx | 148 ++++--- src/oss/langchain/rag.mdx | 3 +- src/oss/langchain/runtime.mdx | 157 +++---- src/oss/langchain/short-term-memory.mdx | 308 ++++++------- src/oss/langchain/tools.mdx | 285 ++++++------ 7 files changed, 881 insertions(+), 532 deletions(-) diff --git a/src/oss/langchain/agents.mdx b/src/oss/langchain/agents.mdx index 42696cc10..1bdedbd80 100644 --- a/src/oss/langchain/agents.mdx +++ b/src/oss/langchain/agents.mdx @@ -62,6 +62,8 @@ Learn more about the [graph API](/oss/langgraph/graph-api). ## Core components +Agents are built from three core components: models, tools, and system prompts. For advanced control over agent behavior, you can use [middleware](/oss/langchain/middleware) to customize how your agent processes information, manages state, and responds to users. + ### Model The [model](/oss/langchain/models) is the reasoning engine of your agent. It can be specified in multiple ways, supporting both static and dynamic model selection. @@ -204,7 +206,7 @@ const agent = createAgent({ ::: -For model configuration details, see [Models](/oss/langchain/models). +For model configuration details, see [Models](/oss/langchain/models). For dynamic model selection patterns, see [Dynamic model in middleware](/oss/langchain/middleware#dynamic-model). ### Tools @@ -679,7 +681,7 @@ To learn more about memory, see [Memory](/oss/concepts/memory). For information ### Before model hook -Pre-model hook is middleware that processes state before the model is called. Use cases include message trimming, summarization, and context injection. +Pre-model hook is middleware that processes state before the model is called. Use cases include message trimming, summarization, and context injection. For comprehensive middleware documentation, see [Middleware](/oss/langchain/middleware). ```mermaid %%{ @@ -789,7 +791,7 @@ const agent = createAgent({ ### After model hook -After model hook is middleware that processes the model's response before tool execution. Use cases include validation, guardrails, or other post-processing. +After model hook is middleware that processes the model's response before tool execution. Use cases include validation, guardrails, or other post-processing. For comprehensive middleware documentation, see [Middleware](/oss/langchain/middleware). ```mermaid %%{ diff --git a/src/oss/langchain/context-engineering.mdx b/src/oss/langchain/context-engineering.mdx index 87205d998..910767ddb 100644 --- a/src/oss/langchain/context-engineering.mdx +++ b/src/oss/langchain/context-engineering.mdx @@ -86,106 +86,410 @@ This is not modified by the agent, and typically isn't passed into the LLM, but Examples include: user ID, DB connections -## Functionality our agent needs to support to enable context engineering +## How to do context engineering with LangChain Now we understand the basic agent loop, the importance of the model you use, and the different types of context that exist. -What functionality does our agent need to support, and how does LangChain's agent support this? - -### Specify custom system prompt - -You can use [`prompt` parameter](/oss/langchain/agents#prompt) to pass in a function that returns a string to use as system prompt - -Use cases: -- Personalize the system prompt with information in session context, long term memory, or runtime context - -### Explicit control over "messages generation" prior to calling model - -You can use [`prompt` parameter](/oss/langchain/agents#prompt) to pass in a function that returns a list of messages - -Use cases: -- Reinforce instructions by dynamically adding an extra system message to the end of the messages sent in, without updating state - -### Access to runtime configuration in "messages generation"/custom system prompt - -You can use [`prompt` parameter](/oss/langchain/agents#prompt) to pass in a function that returns a list of messages or a custom system prompt. -You can access runtime configuration by calling `get_runtime` - -Use cases: -- Use `user_id` passed in to look up user profile, and put it in the system prompt - -### Access to session context in "messages generation"/custom system prompt - -You can use [`prompt` parameter](/oss/langchain/agents#prompt) to pass in a function that returns a list of messages or a custom system prompt. -Session context is passed in with the [`state` parameter](/oss/langchain/short-term-memory#prompt) - -Use cases: -- Use more structured information that the user passes in at runtime (preferences) in the system prompt - -### Access to long term memory in "messages generation"/custom system prompt - -You can use [`prompt` parameter](/oss/langchain/agents#prompt) to pass in a function that returns a list of messages or a custom system prompt. -You can access long term memory by calling `get_store` - -Use cases: -- Look up user preferences from long term memory and put them in the system prompt - -### Update session context before model invocation - -You can use [pre_model_hook](/oss/langchain/agents#pre-model-hook) to update state - -Use cases: -- Filter out messages if message list is getting long, save filtered list in state and only use that -- Create a summary of conversation every N messages, save that in state - -### Access to runtime configuration in tools - -You can use `get_runtime` to [access runtime configuration](/oss/langchain/tools#accessing-runtime-context-inside-a-tool) in tools - -Use cases: -- Use `user_id` to look up information inside a tool call - -### Access to session context in tools - -You can add an argument with InjectedState to tools to access [session context in tools](/oss/langchain/short-term-memory#read-short-term-memory-in-a-tool) - -Use cases: -- Pass messages in state to a sub agent - -### Access to long term memory in tools - -You can use `get_store` to [access long term memory in tools](/oss/langchain/long-term-memory#read-long-term-memory-in-tools) - -Use cases: -- Look up memories from long term memory store - -### Update session context in tools - -You can [return state updates](/oss/langchain/short-term-memory#write-short-term-memory-from-tools) with Command from tools - -Use cases: -- Use tools to update a "virtual file system" - -### Update long term memory in tools - -You can use `get_store` to access long term memory and then [update it inside tools](/oss/langchain/long-term-memory#write-long-term-memory-from-tools) - -Use cases: -- Use tools to update user preferences that are stored in long term memory - -### Update tools before model call - -You can pass in a [function to `model` parameter](/oss/langchain/agents#dynamic-model) that attaches custom tools - -Use cases: -- Force the agent to call a certain tool first -- Only give the agent access to certain tools after it calls other tools -- Remove access to tools (forcing the agent to respond) after N iterations - -### Update model to use before model call - -You can pass in a [function to `model` parameter](/oss/langchain/agents#dynamic-model) that returns a custom model - -Use cases: -- Use a model with a longer context window once message history gets long -- Use a smarter model if the original model gets stuck +Let's explore the concrete patterns LangChain provides for context engineering. + +### Managing instructions (system prompts) + +#### Static instructions + +For fixed instructions that don't change, use the `system_prompt` parameter: + +:::python +```python +from langchain.agents import create_agent + +agent = create_agent( + model="openai:gpt-4o", + tools=[...], + system_prompt="You are a customer support agent. Be helpful, concise, and professional." +) +``` +::: + +:::js +```typescript +import { createAgent } from "langchain"; + +const agent = createAgent({ + model: "openai:gpt-4o", + tools: [...], + systemPrompt: "You are a customer support agent. Be helpful, concise, and professional.", +}); +``` +::: + +#### Dynamic instructions + +For instructions that depend on context (user profile, preferences, session data), use the `@dynamic_prompt` middleware: + +:::python +```python +from dataclasses import dataclass +from langchain.agents import create_agent +from langchain.agents.middleware import dynamic_prompt, ModelRequest +from langgraph.config import get_store + +@dataclass +class Context: + user_id: str + +@dynamic_prompt +def personalized_prompt(request: ModelRequest) -> str: + # Access runtime context + user_id = request.runtime.context.user_id + + # Look up user preferences from long-term memory + store = get_store() + user_prefs = store.get(("users",), user_id) + + # Access session state + message_count = len(request.state["messages"]) + + base = "You are a helpful assistant." + + if user_prefs: + style = user_prefs.value.get("communication_style", "balanced") + base += f"\nUser prefers {style} responses." + + if message_count > 10: + base += "\nThis is a long conversation - be extra concise." + + return base + +agent = create_agent( + model="openai:gpt-4o", + tools=[...], + middleware=[personalized_prompt], + context_schema=Context +) + +# Use the agent with context +result = agent.invoke( + {"messages": [{"role": "user", "content": "Help me debug this code"}]}, + context=Context(user_id="user_123") +) +``` +::: + +:::js +```typescript +import { z } from "zod"; +import { createAgent, dynamicSystemPromptMiddleware } from "langchain"; + +const contextSchema = z.object({ + userId: z.string(), +}); + +const agent = createAgent({ + model: "openai:gpt-4o", + tools: [...], + contextSchema, + middleware: [ + dynamicSystemPromptMiddleware((state, runtime) => { + const userId = runtime.context.userId; + const messageCount = state.messages.length; + + let base = "You are a helpful assistant."; + + // Add context-specific instructions + if (messageCount > 10) { + base += "\nThis is a long conversation - be extra concise."; + } + + return base; + }), + ], +}); + +// Use the agent with context +const result = await agent.invoke( + { messages: [{ role: "user", content: "Help me debug this code" }] }, + { context: { userId: "user_123" } } +); +``` +::: + + +**When to use each:** +- **Static prompts**: Base instructions that never change +- **Dynamic prompts**: Personalization, A/B testing, context-dependent behavior + + +### Managing conversation context (messages) + +Long conversations can exceed context windows or degrade model performance. Use middleware to manage conversation history: + +#### Trimming messages + +:::python +```python +from langchain.agents import create_agent +from langchain.agents.middleware import before_model, AgentState +from langchain.messages import RemoveMessage +from langgraph.graph.message import REMOVE_ALL_MESSAGES +from langgraph.runtime import Runtime + +@before_model +def trim_messages(state: AgentState, runtime: Runtime) -> dict | None: + """Keep only the most recent messages to stay within context window.""" + messages = state["messages"] + + if len(messages) <= 10: + return None # No trimming needed + + # Keep system message + last 8 messages + return { + "messages": [ + RemoveMessage(id=REMOVE_ALL_MESSAGES), + messages[0], # System message + *messages[-8:] # Recent messages + ] + } + +agent = create_agent( + model="openai:gpt-4o", + tools=[...], + middleware=[trim_messages] +) +``` +::: + +:::js +```typescript +import { createMiddleware, RemoveMessage } from "langchain"; +import { REMOVE_ALL_MESSAGES } from "@langchain/langgraph"; + +const trimMessages = createMiddleware({ + name: "TrimMessages", + beforeModel: (state) => { + const messages = state.messages; + + if (messages.length <= 10) { + return; // No trimming needed + } + + // Keep system message + last 8 messages + return { + messages: [ + new RemoveMessage({ id: REMOVE_ALL_MESSAGES }), + messages[0], // System message + ...messages.slice(-8) // Recent messages + ] + }; + }, +}); + +const agent = createAgent({ + model: "openai:gpt-4o", + tools: [...], + middleware: [trimMessages], +}); +``` +::: + +For more sophisticated message management, use the built-in [SummarizationMiddleware](/oss/langchain/middleware#summarization) which automatically summarizes old messages when approaching token limits. + +See [Before model hook](/oss/langchain/agents#before-model-hook) for more examples. + +### Contextual tool execution + +Tools can access runtime context, session state, and long-term memory to make context-aware decisions: + +:::python +```python +from dataclasses import dataclass +from langchain.tools import tool +from langchain.agents import create_agent +from langgraph.runtime import get_runtime +from langgraph.config import get_store +from typing_extensions import Annotated +from langchain.tools import InjectedState + +@dataclass +class Context: + user_id: str + api_key: str + +@tool +def search_documents( + query: str, + state: Annotated[dict, InjectedState] +) -> str: + """Search through documents.""" + # Access runtime context for user-specific configuration + runtime = get_runtime(Context) + user_id = runtime.context.user_id + + # Access long-term memory for user preferences + store = get_store() + search_prefs = store.get(("preferences", user_id), "search") + + # Access session state + conversation_history = state["messages"] + + # Use all context to perform a better search + results = perform_search(query, user_id, search_prefs, conversation_history) + return f"Found {len(results)} results: {results}" + +agent = create_agent( + model="openai:gpt-4o", + tools=[search_documents], + context_schema=Context +) +``` +::: + +See [Tools](/oss/langchain/tools) for comprehensive examples of accessing state, context, and memory in tools. + +### Dynamic tool selection + +Control which tools the agent can access based on context, state, or user permissions: + +:::python +```python +from langchain.agents import create_agent +from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse +from typing import Callable + +@wrap_model_call +def permission_based_tools( + request: ModelRequest, + handler: Callable[[ModelRequest], ModelResponse] +) -> ModelResponse: + """Filter tools based on user permissions.""" + user_role = request.runtime.context.get("user_role", "viewer") + + if user_role == "admin": + # Admins get all tools + pass + elif user_role == "editor": + # Editors can't delete + request.tools = [t for t in request.tools if t.name != "delete_data"] + else: + # Viewers get read-only tools + request.tools = [t for t in request.tools if t.name.startswith("read_")] + + return handler(request) + +agent = create_agent( + model="openai:gpt-4o", + tools=[read_data, write_data, delete_data], + middleware=[permission_based_tools] +) +``` +::: + +:::js +```typescript +import { createMiddleware } from "langchain"; + +const permissionBasedTools = createMiddleware({ + name: "PermissionBasedTools", + wrapModelCall: (request, handler) => { + const userRole = request.runtime.context.userRole || "viewer"; + let filteredTools = request.tools; + + if (userRole === "admin") { + // Admins get all tools + } else if (userRole === "editor") { + // Editors can't delete + filteredTools = request.tools.filter(t => t.name !== "delete_data"); + } else { + // Viewers get read-only tools + filteredTools = request.tools.filter(t => t.name.startsWith("read_")); + } + + return handler({ ...request, tools: filteredTools }); + }, +}); +``` +::: + +See [Dynamically selecting tools](/oss/langchain/middleware#dynamically-selecting-tools) for more examples. + +### Dynamic model selection + +Switch models based on conversation complexity, context window needs, or cost optimization: + +:::python +```python +from langchain.agents import create_agent +from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse +from langchain.chat_models import init_chat_model +from typing import Callable + +@wrap_model_call +def adaptive_model( + request: ModelRequest, + handler: Callable[[ModelRequest], ModelResponse] +) -> ModelResponse: + """Use different models based on conversation length.""" + message_count = len(request.messages) + + if message_count > 20: + # Long conversation - use model with larger context window + request.model = init_chat_model("anthropic:claude-sonnet-4-5-20250929") + elif message_count > 10: + # Medium conversation - use mid-tier model + request.model = init_chat_model("openai:gpt-4o") + else: + # Short conversation - use efficient model + request.model = init_chat_model("openai:gpt-4o-mini") + + return handler(request) + +agent = create_agent( + model="openai:gpt-4o-mini", # Default model + tools=[...], + middleware=[adaptive_model] +) +``` +::: + +:::js +```typescript +import { createMiddleware, initChatModel } from "langchain"; + +const adaptiveModel = createMiddleware({ + name: "AdaptiveModel", + wrapModelCall: (request, handler) => { + const messageCount = request.messages.length; + let model; + + if (messageCount > 20) { + // Long conversation - use model with larger context window + model = initChatModel("anthropic:claude-sonnet-4-5-20250929"); + } else if (messageCount > 10) { + // Medium conversation - use mid-tier model + model = initChatModel("openai:gpt-4o"); + } else { + // Short conversation - use efficient model + model = initChatModel("openai:gpt-4o-mini"); + } + + return handler({ ...request, model }); + }, +}); +``` +::: + +See [Dynamic model](/oss/langchain/agents#dynamic-model) for more examples. + +## Best practices + +1. **Start simple** - Begin with static prompts and tools, add dynamics only when needed +2. **Test incrementally** - Add one context engineering feature at a time +3. **Monitor performance** - Track model calls, token usage, and latency +4. **Use built-in middleware** - Leverage [SummarizationMiddleware](/oss/langchain/middleware#summarization), [LLMToolSelectorMiddleware](/oss/langchain/middleware#llm-tool-selector), etc. +5. **Document your context strategy** - Make it clear what context is being passed and why + +## Related resources + +- [Middleware](/oss/langchain/middleware) - Complete middleware guide +- [Tools](/oss/langchain/tools) - Tool creation and context access +- [Memory](/oss/concepts/memory) - Short-term and long-term memory patterns +- [Agents](/oss/langchain/agents) - Core agent concepts diff --git a/src/oss/langchain/middleware.mdx b/src/oss/langchain/middleware.mdx index 49a51fb3b..dfd6d4397 100644 --- a/src/oss/langchain/middleware.mdx +++ b/src/oss/langchain/middleware.mdx @@ -864,6 +864,103 @@ const agent = createAgent({ Build custom middleware by implementing hooks that run at specific points in the agent execution flow. +You can create middleware in two ways: +1. **Decorator-based** - Quick and simple for single-hook middleware +2. **Class-based** - More powerful for complex middleware with multiple hooks + +:::python + +## Decorator-based middleware + +For simple middleware that only needs a single hook, decorators provide the quickest way to add functionality: + +```python +from langchain.agents.middleware import before_model, after_model, wrap_model_call +from langchain.agents.middleware import AgentState, ModelRequest, ModelResponse +from langgraph.runtime import Runtime +from typing import Any, Callable + +# Node-style: logging before model calls +@before_model +def log_before_model(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: + print(f"About to call model with {len(state['messages'])} messages") + return None + +# Node-style: validation after model calls +@after_model +def validate_output(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: + last_message = state["messages"][-1] + if "BLOCKED" in last_message.content: + return { + "messages": [AIMessage("I cannot respond to that request.")], + "jump_to": "end" + } + return None + +# Wrap-style: retry logic +@wrap_model_call +def retry_model( + request: ModelRequest, + handler: Callable[[ModelRequest], ModelResponse], +) -> ModelResponse: + for attempt in range(3): + try: + return handler(request) + except Exception as e: + if attempt == 2: + raise + print(f"Retry {attempt + 1}/3 after error: {e}") + +# Wrap-style: dynamic prompts +@dynamic_prompt +def personalized_prompt(request: ModelRequest) -> str: + user_id = request.runtime.context.get("user_id", "guest") + return f"You are a helpful assistant for user {user_id}. Be concise and friendly." + +# Use decorators in agent +agent = create_agent( + model="openai:gpt-4o", + middleware=[log_before_model, validate_output, retry_model, personalized_prompt], + tools=[...], +) +``` + +### Available decorators + +**Node-style** (run at specific execution points): +- `@before_agent` - Before agent starts (once per invocation) +- `@before_model` - Before each model call +- `@after_model` - After each model response +- `@after_agent` - After agent completes (once per invocation) + +**Wrap-style** (intercept and control execution): +- `@wrap_model_call` - Around each model call +- `@wrap_tool_call` - Around each tool call + +**Convenience decorators**: +- `@dynamic_prompt` - Generates dynamic system prompts (equivalent to `@wrap_model_call` that modifies the prompt) + +### When to use decorators + + + + - You need a single hook + - Logic is straightforward + - No custom state schema needed + - Quick prototyping + + + - Multiple hooks needed + - Custom state/context schema + - Complex configuration + - Reusable across projects + + + +::: + +## Class-based middleware + ### Two hook styles @@ -1279,57 +1376,6 @@ await agent.invoke( ``` ::: - -:::python - -### Decorator-based middleware - -For simple middleware that only needs a single hook, use decorator shortcuts: - -```python -from langchain.agents.middleware import before_model, after_model, wrap_model_call -from langchain.agents.middleware import AgentState, ModelRequest, ModelResponse -from langgraph.runtime import Runtime -from typing import Any, Callable - -# Node-style decorator -@before_model -def log_before_model(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: - print(f"About to call model with {len(state['messages'])} messages") - return None - -# Wrap-style decorator -@wrap_model_call -def retry_model( - request: ModelRequest, - handler: Callable[[ModelRequest], ModelResponse], -) -> ModelResponse: - for attempt in range(3): - try: - return handler(request) - except Exception: - if attempt == 2: - raise - return handler(request) # This line is unreachable but satisfies type checker - -# Use decorators in agent -agent = create_agent( - model="openai:gpt-4o", - middleware=[log_before_model, retry_model], - tools=[...], -) -``` - -Available decorators: -- `@before_agent` -- `@before_model` -- `@after_model` -- `@after_agent` -- `@wrap_model_call` -- `@wrap_tool_call` -- `@dynamic_prompt` - Convenience for dynamic system prompts -::: - ### Execution order When using multiple middleware, understanding execution order is important: diff --git a/src/oss/langchain/rag.mdx b/src/oss/langchain/rag.mdx index bb6b19fed..751b30f5e 100644 --- a/src/oss/langchain/rag.mdx +++ b/src/oss/langchain/rag.mdx @@ -892,6 +892,8 @@ class State(AgentState): class RetrieveDocumentsMiddleware(AgentMiddleware[State]): + state_schema = State + def before_model(self, state: AgentState) -> dict[str, Any] | None: last_message = state["messages"][-1] retrieved_docs = vector_store.similarity_search(last_message.text) @@ -913,7 +915,6 @@ agent = create_agent( llm, tools=[], middleware=[RetrieveDocumentsMiddleware()], - state_schema=State, ) ``` ::: diff --git a/src/oss/langchain/runtime.mdx b/src/oss/langchain/runtime.mdx index d3eb55cf2..969a22cfa 100644 --- a/src/oss/langchain/runtime.mdx +++ b/src/oss/langchain/runtime.mdx @@ -20,7 +20,7 @@ LangGraph exposes a @[Runtime] object with the following information: 2. **Store**: a @[BaseStore] instance used for [long-term memory](/oss/langchain/long-term-memory) 3. **Stream writer**: an object used for streaming information via the `"custom"` stream mode -You can access the runtime information within [tools](#inside-tools), [prompt](#inside-prompt), and [pre and post model hooks](#inside-pre-and-post-model-hooks). +You can access the runtime information within [tools](#inside-tools) and [middleware](#inside-middleware). ## Access @@ -145,34 +145,48 @@ const fetchUserEmailPreferences = tool( ``` ::: -### Inside prompt +### Inside middleware + +You can access runtime information in middleware to create dynamic prompts, modify messages, or control agent behavior based on user context. :::python -Use the @[get_runtime] function from `langgraph.runtime` to access the @[Runtime] object inside a prompt function. +Use `request.runtime` to access the @[Runtime] object inside middleware decorators. The runtime object is available in the `ModelRequest` parameter passed to middleware functions. ```python from dataclasses import dataclass from langchain.messages import AnyMessage -from langchain.agents import create_agent -from langgraph.runtime import get_runtime # [!code highlight] +from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import dynamic_prompt, ModelRequest, before_model, after_model +from langgraph.runtime import Runtime @dataclass class Context: user_name: str -from langchain.agents.middleware import dynamic_prompt, ModelRequest - +# Dynamic prompts @dynamic_prompt def dynamic_system_prompt(request: ModelRequest) -> str: - user_name = request.runtime.context["user_name"] + user_name = request.runtime.context.user_name # [!code highlight] system_prompt = f"You are a helpful assistant. Address the user as {user_name}." return system_prompt +# Before model hook +@before_model +def log_before_model(state: AgentState, runtime: Runtime[Context]) -> dict | None: # [!code highlight] + print(f"Processing request for user: {runtime.context.user_name}") # [!code highlight] + return None + +# After model hook +@after_model +def log_after_model(state: AgentState, runtime: Runtime[Context]) -> dict | None: # [!code highlight] + print(f"Completed request for user: {runtime.context.user_name}") # [!code highlight] + return None + agent = create_agent( model="openai:gpt-5-nano", tools=[...], - middleware=[dynamic_system_prompt], + middleware=[dynamic_system_prompt, log_before_model, log_after_model], # [!code highlight] context_schema=Context ) @@ -183,36 +197,52 @@ agent.invoke( ``` ::: :::js -Use the `runtime` parameter to access the @[Runtime] object inside a prompt function. +Use the `runtime` parameter to access the @[Runtime] object inside middleware. ```ts import { z } from "zod"; -import { createAgent, type AgentState, SystemMessage } from "langchain"; +import { createAgent, createMiddleware, type AgentState, SystemMessage } from "langchain"; import { type Runtime } from "@langchain/langgraph"; // [!code highlight] const contextSchema = z.object({ userName: z.string(), }); -const prompt = ( - state: AgentState, - runtime: Runtime> // [!code highlight] -) => { - const userName = runtime.context?.userName; // [!code highlight] - if (!userName) { - throw new Error("userName is required"); +// Dynamic prompt middleware +const dynamicPromptMiddleware = createMiddleware({ + name: "DynamicPrompt", + beforeModel: (state: AgentState, runtime: Runtime>) => { // [!code highlight] + const userName = runtime.context?.userName; // [!code highlight] + if (!userName) { + throw new Error("userName is required"); + } + + const systemMsg = `You are a helpful assistant. Address the user as ${userName}.`; + return { + messages: [new SystemMessage(systemMsg), ...state.messages] + }; } +}); - const systemMsg = `You are a helpful assistant. Address the user as ${userName}.`; // [!code highlight] - return [new SystemMessage(systemMsg), ...state.messages]; -}; +// Logging middleware +const loggingMiddleware = createMiddleware({ + name: "Logging", + beforeModel: (state: AgentState, runtime: Runtime>) => { // [!code highlight] + console.log(`Processing request for user: ${runtime.context?.userName}`); // [!code highlight] + return; + }, + afterModel: (state: AgentState, runtime: Runtime>) => { // [!code highlight] + console.log(`Completed request for user: ${runtime.context?.userName}`); // [!code highlight] + return; + } +}); const agent = createAgent({ model: "openai:gpt-4o", tools: [ /* ... */ ], - prompt, + middleware: [dynamicPromptMiddleware, loggingMiddleware], // [!code highlight] contextSchema, }); @@ -222,86 +252,3 @@ const result = await agent.invoke( ); ``` ::: - -### Inside pre and post model hooks - -:::python -To access the underlying graph runtime information in a pre or post model hook, you can: - -1. Use the @[get_runtime] function from `langgraph.runtime` to access the @[Runtime] object inside the hook -2. Inject the @[Runtime] directly via the hook signature - -This above options are purely preferential and not functionally different. - - - - ```python - from langgraph.runtime import get_runtime # [!code highlight] - - def pre_model_hook(state: State) -> State: - runtime = get_runtime(Context) # [!code highlight] - ... - ``` - - - ```python - from langgraph.runtime import Runtime # [!code highlight] - - def pre_model_hook(state: State, runtime: Runtime[Context]): # [!code highlight] - ... - ``` - - -::: - -:::js -Use the `runtime` parameter to access the @[Runtime] object inside a pre or post model hook. - -```ts -import { z } from "zod"; -import { type Runtime } from "@langchain/langgraph"; // [!code highlight] -import { createAgent, type AgentState } from "langchain"; - -const contextSchema = z.object({ - userName: z.string(), -}); - -const preModelHook = ( - state: AgentState, - runtime: Runtime> // [!code highlight] -) => { - const userName = runtime.context?.userName; // [!code highlight] - if (!userName) { - throw new Error("userName is required"); - } - - return { - // ... - }; -}; - -const postModelHook = ( - state: AgentState, - runtime: Runtime> // [!code highlight] -) => { - const userName = runtime.context?.userName; // [!code highlight] - if (!userName) { - throw new Error("userName is required"); - } - - return { - // ... - }; -}; - -const agent = createAgent({ - model: "openai:gpt-4o-mini", - tools: [ - /* ... */ - ], - contextSchema, - preModelHook, - postModelHook, -}); -``` -::: diff --git a/src/oss/langchain/short-term-memory.mdx b/src/oss/langchain/short-term-memory.mdx index 519d3991a..81e3394cb 100644 --- a/src/oss/langchain/short-term-memory.mdx +++ b/src/oss/langchain/short-term-memory.mdx @@ -105,41 +105,67 @@ const checkpointer = PostgresSaver.fromConnString(DB_URI); By default, agents use `AgentState` to manage short term memory, specifically the conversation history via a `messages` key. -Users can subclass `AgentState` to add additional fields to the state. - -This custom state can then be accessed via tools and dynamic prompt / model functions. +You can extend `AgentState` to add additional fields. Custom state schemas are defined in middleware using the `state_schema` attribute. :::python ```python from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import AgentMiddleware from langgraph.checkpoint.memory import InMemorySaver + class CustomAgentState(AgentState): # [!code highlight] user_id: str # [!code highlight] + preferences: dict # [!code highlight] + +class StateExtensionMiddleware(AgentMiddleware[CustomAgentState]): + state_schema = CustomAgentState # [!code highlight] agent = create_agent( "openai:gpt-5", [get_user_info], - state_schema=CustomAgentState, # [!code highlight] + middleware=[StateExtensionMiddleware()], # [!code highlight] checkpointer=InMemorySaver(), ) + +# Custom state can be passed in invoke +result = agent.invoke({ + "messages": [{"role": "user", "content": "Hello"}], + "user_id": "user_123", # [!code highlight] + "preferences": {"theme": "dark"} # [!code highlight] +}) ``` ::: :::js ```typescript import { z } from "zod"; -import { createAgent } from "langchain"; -import { MemorySaver } from "@langchain/langgraph"; -const stateSchema = z.object({ // [!code highlight] - messages: z.array(z.any()), // [!code highlight] -}); // [!code highlight] +import { createAgent, createMiddleware } from "langchain"; +import { MessagesZodState, MemorySaver } from "@langchain/langgraph"; + +const customStateSchema = z.object({ // [!code highlight] + messages: MessagesZodState.shape.messages, // [!code highlight] + userId: z.string(), // [!code highlight] + preferences: z.record(z.string(), z.any()), // [!code highlight] +}); // [!code highlight] + +const stateExtensionMiddleware = createMiddleware({ + name: "StateExtension", + stateSchema: customStateSchema, // [!code highlight] +}); const checkpointer = new MemorySaver(); const agent = createAgent({ model: "openai:gpt-5", tools: [], - stateSchema, // [!code highlight] + middleware: [stateExtensionMiddleware] as const, // [!code highlight] checkpointer, }); + +// Custom state can be passed in invoke +const result = await agent.invoke({ + messages: [{ role: "user", content: "Hello" }], + userId: "user_123", // [!code highlight] + preferences: { theme: "dark" }, // [!code highlight] +}); ``` ::: @@ -171,19 +197,20 @@ Most LLMs have a maximum supported context window (denominated in tokens). One way to decide when to truncate messages is to count the tokens in the message history and truncate whenever it approaches that limit. If you're using LangChain, you can use the trim messages utility and specify the number of tokens to keep from the list, as well as the `strategy` (e.g., keep the last `maxTokens`) to use for handling the boundary. :::python -To trim message history in an agent, use @[`pre_model_hook`][create_agent] with the [`trim_messages`](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html) function: +To trim message history in an agent, use the `@before_model` middleware decorator with the [`trim_messages`](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html) function: ```python from langchain_core.messages.utils import trim_messages, count_tokens_approximately from langchain.messages import BaseMessage from langgraph.checkpoint.memory import InMemorySaver -from langchain.agents import create_agent +from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import before_model +from langgraph.runtime import Runtime from langchain_core.runnables import RunnableConfig -def pre_model_hook(state) -> dict[str, list[BaseMessage]]: - """ - This function will be called prior to every llm call to prepare the messages for the llm. - """ +@before_model +def trim_message_history(state: AgentState, runtime: Runtime) -> dict[str, list[BaseMessage]] | None: + """Trim messages before every model call to stay within context limits.""" trimmed_messages = trim_messages( state["messages"], strategy="last", @@ -192,14 +219,17 @@ def pre_model_hook(state) -> dict[str, list[BaseMessage]]: start_on="human", end_on=("human", "tool"), ) - return {"llm_input_messages": trimmed_messages} + # Return None if no trimming needed + if len(trimmed_messages) == len(state["messages"]): + return None + return {"messages": trimmed_messages} checkpointer = InMemorySaver() agent = create_agent( "openai:gpt-5-nano", tools=[], - pre_model_hook=pre_model_hook, + middleware=[trim_message_history], checkpointer=checkpointer, ) @@ -322,23 +352,28 @@ const deleteMessages = (state) => { ```python from langchain.messages import RemoveMessage -from langchain.agents import create_agent +from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import after_model from langgraph.checkpoint.memory import InMemorySaver +from langgraph.runtime import Runtime from langchain_core.runnables import RunnableConfig -def delete_messages(state): +@after_model +def delete_old_messages(state: AgentState, runtime: Runtime) -> dict | None: + """Remove old messages to keep conversation manageable.""" messages = state["messages"] if len(messages) > 2: # remove the earliest two messages return {"messages": [RemoveMessage(id=m.id) for m in messages[:2]]} + return None agent = create_agent( "openai:gpt-5-nano", tools=[], - prompt="Please be concise and to the point.", - post_model_hook=delete_messages, + system_prompt="Please be concise and to the point.", + middleware=[delete_old_messages], checkpointer=InMemorySaver(), ) @@ -442,40 +477,27 @@ Because of this, some applications benefit from a more sophisticated approach of :::python -To summarize message history in an agent, use @[`pre_model_hook`][create_agent] with a prebuilt [`SummarizationNode`](https://langchain-ai.github.io/langmem/reference/short_term/#langmem.short_term.SummarizationNode) abstraction: - -{/* TODO: FIX THIS EXAMPLE, NOT ABLE TO RUN */} +To summarize message history in an agent, use the built-in [`SummarizationMiddleware`](/oss/langchain/middleware#summarization): ```python -from langmem.short_term import SummarizationNode, RunningSummary -from langchain_core.messages.utils import count_tokens_approximately -from langchain.agents import create_agent, AgentState +from langchain.agents import create_agent +from langchain.agents.middleware import SummarizationMiddleware from langgraph.checkpoint.memory import InMemorySaver -from langchain_openai import ChatOpenAI from langchain_core.runnables import RunnableConfig -model = ChatOpenAI(model="gpt-4o-mini") - -summarization_node = SummarizationNode( - token_counter=count_tokens_approximately, - model=model, - max_tokens=384, - max_summary_tokens=128, - output_messages_key="llm_input_messages", -) - -class State(AgentState): - # Added for the SummarizationNode to be able to keep track of the running summary information - context: dict[str, RunningSummary] - -checkpointer = InMemorySaver() # [!code highlight] +checkpointer = InMemorySaver() agent = create_agent( - model=model, + model="openai:gpt-4o", tools=[], - pre_model_hook=summarization_node, - state_schema=State, - checkpointer=checkpointer, # [!code highlight] + middleware=[ + SummarizationMiddleware( + model="openai:gpt-4o-mini", + max_tokens_before_summary=4000, # Trigger summarization at 4000 tokens + messages_to_keep=20, # Keep last 20 messages after summary + ) + ], + checkpointer=checkpointer, ) config: RunnableConfig = {"configurable": {"thread_id": "1"}} @@ -484,25 +506,56 @@ agent.invoke({"messages": "write a short poem about cats"}, config) agent.invoke({"messages": "now do the same but for dogs"}, config) final_response = agent.invoke({"messages": "what's my name?"}, config) -print(final_response.keys()) - final_response["messages"][-1].pretty_print() -print("\nSummary:", final_response["context"]["running_summary"].summary) +""" +================================== Ai Message ================================== + +Your name is Bob! +""" ``` + +See [SummarizationMiddleware](/oss/langchain/middleware#summarization) for more configuration options. ::: :::js -TODO +To summarize message history in an agent, use the built-in [`summarizationMiddleware`](/oss/langchain/middleware#summarization): + +```typescript +import { createAgent, summarizationMiddleware } from "langchain"; +import { MemorySaver } from "@langchain/langgraph"; + +const checkpointer = new MemorySaver(); + +const agent = createAgent({ + model: "openai:gpt-4o", + tools: [], + middleware: [ + summarizationMiddleware({ + model: "openai:gpt-4o-mini", + maxTokensBeforeSummary: 4000, + messagesToKeep: 20, + }), + ], + checkpointer, +}); + +const config = { configurable: { thread_id: "1" } }; +await agent.invoke({ messages: "hi, my name is bob" }, config); +await agent.invoke({ messages: "write a short poem about cats" }, config); +await agent.invoke({ messages: "now do the same but for dogs" }, config); +const finalResponse = await agent.invoke({ messages: "what's my name?" }, config); + +console.log(finalResponse.messages.at(-1)?.content); +// Your name is Bob! +``` + +See [summarizationMiddleware](/oss/langchain/middleware#summarization) for more configuration options. ::: ## Access -You can access the short-term memory of an agent in a few different ways: - -* [Tools](#tools) -* [Pre model hook](#pre-model-hook) -* [Post model hook](#post-model-hook) +You can access and modify the short-term memory (state) of an agent in several ways: -### Tools +### In tools #### Read short-term memory in a tool @@ -706,9 +759,9 @@ await agent.invoke( ``` ::: -### Prompt +### In middleware (dynamic prompts) -Access short term memory (state) in a dynamic prompt function by injecting the agent's state into the prompt function signature. +Access short term memory (state) in middleware to create dynamic prompts based on conversation history or custom state fields. :::python ```python @@ -802,7 +855,6 @@ const agent = createAgent({ `You are a helpful assistant. Address the user as ${config.context?.userName}.` ), ...state.messages, - ]; }, }); @@ -845,29 +897,29 @@ for (const message of result.messages) { * // ... * } * AIMessage { - * "content": "John Smith, here’s the latest: The weather in San Francisco is always sunny!\n\nIf you’d like more details (temperature, wind, humidity) or a forecast for the next few days, I can pull that up. What would you like?", + * "content": "John Smith, here's the latest: The weather in San Francisco is always sunny!\n\nIf you'd like more details (temperature, wind, humidity) or a forecast for the next few days, I can pull that up. What would you like?", * // ... * } */ ``` ::: -### Pre model hook +### In middleware (message processing) -Access short term memory (state) in a pre model hook by injecting the agent's state into the hook signature. +Access short term memory (state) in middleware to process messages before or after model calls. :::python ```python from langchain_core.messages.utils import trim_messages, count_tokens_approximately -from langchain.messages import BaseMessage +from langchain.messages import BaseMessage, RemoveMessage from langgraph.checkpoint.memory import InMemorySaver from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import before_model, after_model +from langgraph.runtime import Runtime - -def pre_model_hook(state: AgentState) -> dict[str, list[BaseMessage]]: - """ - This function will be called prior to every llm call to prepare the messages for the llm. - """ +@before_model +def trim_message_history(state: AgentState, runtime: Runtime) -> dict | None: + """Trim messages before model call.""" trimmed_messages = trim_messages( state["messages"], strategy="last", @@ -876,79 +928,23 @@ def pre_model_hook(state: AgentState) -> dict[str, list[BaseMessage]]: start_on="human", end_on=("human", "tool"), ) - return {"llm_input_messages": trimmed_messages} - -agent = create_agent( - model="openai:gpt-5-nano", - tools=[], - pre_model_hook=pre_model_hook, - checkpointer=InMemorySaver(), -) - -result = agent.invoke({"messages": "hi, my name is bob"}, {"configurable": {"thread_id": "1"}}) -print(result["messages"][-1].content) -``` -::: -:::js -```typescript -import { - createAgent, - type AgentState, - trimMessages, - type BaseMessage, -} from "langchain"; - -const preModelHook = async (state: AgentState) => { - return { - messages: await trimMessages(state.messages, { - maxTokens: 384, - strategy: "last", - startOn: "human", - endOn: ["human", "tool"], - tokenCounter: (msgs: BaseMessage[]) => msgs.length, - }), - }; -}; - -const agent = createAgent({ - model: "openai:gpt-5-nano", - tools: [], - preModelHook, -}); - -const result = await agent.invoke( - { - messages: [{ role: "user", content: "hi, my name is bob" }], - }, - { - context: { thread_id: "1" }, - } -); -console.log(result.messages.at(-1)?.content); -``` -::: - -### Post model hook - -Access short term memory (state) in a post model hook by injecting the agent's state into the hook signature. - -:::python -```python -from langchain.agents import create_agent, AgentState - -STOP_WORDS = ["password", "secret"] - -def validate_response(state: AgentState) -> dict[str, list[BaseMessage]]: - """Confirm the response doesn't have any content that is in the stop words list.""" + if len(trimmed_messages) == len(state["messages"]): + return None + return {"messages": trimmed_messages} + +@after_model +def validate_response(state: AgentState, runtime: Runtime) -> dict | None: + """Remove messages containing sensitive words.""" + STOP_WORDS = ["password", "secret"] last_message = state["messages"][-1] if any(word in last_message.content for word in STOP_WORDS): return {"messages": [RemoveMessage(id=last_message.id)]} - return {} + return None agent = create_agent( model="openai:gpt-5-nano", tools=[], - post_model_hook=validate_response, + middleware=[trim_message_history, validate_response], checkpointer=InMemorySaver(), ) ``` @@ -956,25 +952,41 @@ agent = create_agent( :::js ```typescript import { RemoveMessage } from "@langchain/core/messages"; -import { createAgent, type AgentState } from "langchain"; +import { createAgent, createMiddleware, trimMessages, type AgentState } from "langchain"; + +const trimMessageHistory = createMiddleware({ + name: "TrimMessages", + beforeModel: async (state) => { + const trimmed = await trimMessages(state.messages, { + maxTokens: 384, + strategy: "last", + startOn: "human", + endOn: ["human", "tool"], + tokenCounter: (msgs) => msgs.length, + }); + return { messages: trimmed }; + }, +}); -function validateResponse(state: AgentState) { +const validateResponse = createMiddleware({ + name: "ValidateResponse", + afterModel: (state) => { const lastMessage = state.messages.at(-1)?.content; - if ( - typeof lastMessage === "string" && - lastMessage.toLowerCase().includes("confidential") - ) { - return { + if (typeof lastMessage === "string" && lastMessage.toLowerCase().includes("confidential")) { + return { messages: [new RemoveMessage({ id: "all" }), ...state.messages], - }; + }; } - return {}; -} + return; + }, +}); const agent = createAgent({ - model: "openai:gpt-5", + model: "openai:gpt-5-nano", tools: [], - postModelHook: validateResponse, + middleware: [trimMessageHistory, validateResponse], }); ``` ::: + + diff --git a/src/oss/langchain/tools.mdx b/src/oss/langchain/tools.mdx index 1fbb0c5f7..6e4e9381f 100644 --- a/src/oss/langchain/tools.mdx +++ b/src/oss/langchain/tools.mdx @@ -10,133 +10,17 @@ Many AI applications interact with users via natural language. However, some use Tools are components that [agents](/oss/langchain/agents) call to perform actions. They extend model capabilities by letting them interact with the world through well-defined inputs and outputs. Tools encapsulate a callable function and its input schema. These can be passed to compatible [chat models](/oss/langchain/models), allowing the model to decide whether to invoke a tool and with what arguments. In these scenarios, tool calling enables models to generate requests that conform to a specified input schema. -## Create tools - -### Basic tool definition - -:::python -The simplest way to create a tool is with the `@tool` decorator. By default, the function's docstring becomes the tool's description that helps the model understand when to use it: - -```python wrap -from langchain.tools import tool - -@tool -def search_database(query: str, limit: int = 10) -> str: - """Search the customer database for records matching the query. - - Args: - query: Search terms to look for - limit: Maximum number of results to return - """ - return f"Found {limit} results for '{query}'" -``` - -Type hints are **required** as they define the tool's input schema. The docstring should be informative and concise to help the model understand the tool's purpose. -::: - -:::js -The simplest way to create a tool is by importing the `tool` function from the `langchain` package. You can use [zod](https://zod.dev/) to define the tool's input schema: - -```ts -import { z } from "zod" -import { tool } from "langchain" - -const searchDatabase = tool( - ({ query, limit }) => `Found ${limit} results for '${query}'`, - { - name: "search_database", - description: "Search the customer database for records matching the query.", - schema: z.object({ - query: z.string().describe("Search terms to look for"), - limit: z.number().describe("Maximum number of results to return"), - }), - } -); -``` -::: - -:::python -### Customize tool properties - -#### Custom tool name - -By default, the tool name comes from the function name. Override it when you need something more descriptive: - -```python wrap -@tool("web_search") # Custom name -def search(query: str) -> str: - """Search the web for information.""" - return f"Results for: {query}" - -print(search.name) # web_search -``` - -#### Custom tool description - -Override the auto-generated tool description for clearer model guidance: - -```python wrap -@tool("calculator", description="Performs arithmetic calculations. Use this for any math problems.") -def calc(expression: str) -> str: - """Evaluate mathematical expressions.""" - return str(eval(expression)) -``` - -### Advanced schema definition - -Define complex inputs with Pydantic models or JSON schemas: - - - ```python wrap Pydantic model - from pydantic import BaseModel, Field - from typing import Literal - - class WeatherInput(BaseModel): - """Input for weather queries.""" - location: str = Field(description="City name or coordinates") - units: Literal["celsius", "fahrenheit"] = Field( - default="celsius", - description="Temperature unit preference" - ) - include_forecast: bool = Field( - default=False, - description="Include 5-day forecast" - ) - - @tool(args_schema=WeatherInput) - def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str: - """Get current weather and optional forecast.""" - temp = 22 if units == "celsius" else 72 - result = f"Current weather in {location}: {temp} degrees {units[0].upper()}" - if include_forecast: - result += "\nNext 5 days: Sunny" - return result - ``` +## Accessing agent data in tools - ```python wrap JSON Schema - weather_schema = { - "type": "object", - "properties": { - "location": {"type": "string"}, - "units": {"type": "string"}, - "include_forecast": {"type": "boolean"} - }, - "required": ["location", "units", "include_forecast"] - } + +**Why this matters:** Tools are most powerful when they can access agent state, runtime context, and long-term memory. This enables tools to make context-aware decisions, personalize responses, and maintain information across conversations. + - @tool(args_schema=weather_schema) - def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str: - """Get current weather and optional forecast.""" - temp = 22 if units == "celsius" else 72 - result = f"Current weather in {location}: {temp} degrees {units[0].upper()}" - if include_forecast: - result += "\nNext 5 days: Sunny" - return result - ``` - -::: +Tools can access three types of data: -## State, context, and memory +- **State** - Mutable data that flows through execution (messages, counters, custom fields) +- **Runtime Context** - Immutable configuration (user IDs, API keys, session details) +- **Long-term Memory** - Persistent data across conversations (user preferences, history) :::python @@ -177,11 +61,37 @@ Define complex inputs with Pydantic models or JSON schemas: ) -> str: """Get a user preference value.""" return preferences.get(pref_name, "Not set") + + # Access runtime context alongside state + from langgraph.runtime import get_runtime + from dataclasses import dataclass + + @dataclass + class Context: + user_id: str + + @tool + def get_personalized_greeting( + state: Annotated[dict, InjectedState] + ) -> str: + """Generate a personalized greeting.""" + # Access runtime context + runtime = get_runtime(Context) + user_id = runtime.context.user_id + + # Access state + message_count = len(state["messages"]) + + return f"Hello user {user_id}! This is message #{message_count}" ``` State-injected arguments are hidden from the model. For the example above, the model only sees `pref_name` in the tool schema - `preferences` is *not* included in the request. + + + **State vs Runtime**: Use `InjectedState` for data that changes during execution (like messages or counters). Use `get_runtime()` for immutable configuration (like user IDs or API keys). + @@ -496,3 +406,130 @@ Define complex inputs with Pydantic models or JSON schemas: ::: + +## Create tools + +### Basic tool definition + +:::python +The simplest way to create a tool is with the `@tool` decorator. By default, the function's docstring becomes the tool's description that helps the model understand when to use it: + +```python wrap +from langchain.tools import tool + +@tool +def search_database(query: str, limit: int = 10) -> str: + """Search the customer database for records matching the query. + + Args: + query: Search terms to look for + limit: Maximum number of results to return + """ + return f"Found {limit} results for '{query}'" +``` + +Type hints are **required** as they define the tool's input schema. The docstring should be informative and concise to help the model understand the tool's purpose. +::: + +:::js +The simplest way to create a tool is by importing the `tool` function from the `langchain` package. You can use [zod](https://zod.dev/) to define the tool's input schema: + +```ts +import { z } from "zod" +import { tool } from "langchain" + +const searchDatabase = tool( + ({ query, limit }) => `Found ${limit} results for '${query}'`, + { + name: "search_database", + description: "Search the customer database for records matching the query.", + schema: z.object({ + query: z.string().describe("Search terms to look for"), + limit: z.number().describe("Maximum number of results to return"), + }), + } +); +``` +::: + +:::python +### Customize tool properties + +#### Custom tool name + +By default, the tool name comes from the function name. Override it when you need something more descriptive: + +```python wrap +@tool("web_search") # Custom name +def search(query: str) -> str: + """Search the web for information.""" + return f"Results for: {query}" + +print(search.name) # web_search +``` + +#### Custom tool description + +Override the auto-generated tool description for clearer model guidance: + +```python wrap +@tool("calculator", description="Performs arithmetic calculations. Use this for any math problems.") +def calc(expression: str) -> str: + """Evaluate mathematical expressions.""" + return str(eval(expression)) +``` + +### Advanced schema definition + +Define complex inputs with Pydantic models or JSON schemas: + + + ```python wrap Pydantic model + from pydantic import BaseModel, Field + from typing import Literal + + class WeatherInput(BaseModel): + """Input for weather queries.""" + location: str = Field(description="City name or coordinates") + units: Literal["celsius", "fahrenheit"] = Field( + default="celsius", + description="Temperature unit preference" + ) + include_forecast: bool = Field( + default=False, + description="Include 5-day forecast" + ) + + @tool(args_schema=WeatherInput) + def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str: + """Get current weather and optional forecast.""" + temp = 22 if units == "celsius" else 72 + result = f"Current weather in {location}: {temp} degrees {units[0].upper()}" + if include_forecast: + result += "\nNext 5 days: Sunny" + return result + ``` + + ```python wrap JSON Schema + weather_schema = { + "type": "object", + "properties": { + "location": {"type": "string"}, + "units": {"type": "string"}, + "include_forecast": {"type": "boolean"} + }, + "required": ["location", "units", "include_forecast"] + } + + @tool(args_schema=weather_schema) + def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str: + """Get current weather and optional forecast.""" + temp = 22 if units == "celsius" else 72 + result = f"Current weather in {location}: {temp} degrees {units[0].upper()}" + if include_forecast: + result += "\nNext 5 days: Sunny" + return result + ``` + +::: + From 8837d78932c347b92e0ffb80970acde7c7c1ea6f Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 11:56:54 -0400 Subject: [PATCH 02/11] improving more pages --- src/oss/langchain/agents.mdx | 235 +++------------------- src/oss/langchain/context-engineering.mdx | 2 +- 2 files changed, 26 insertions(+), 211 deletions(-) diff --git a/src/oss/langchain/agents.mdx b/src/oss/langchain/agents.mdx index 1bdedbd80..a2776061f 100644 --- a/src/oss/langchain/agents.mdx +++ b/src/oss/langchain/agents.mdx @@ -521,7 +521,7 @@ const result = await agent.invoke( For more details on message types and formatting, see [Messages](/oss/langchain/messages). For comprehensive middleware documentation, see [Middleware](/oss/langchain/middleware). -## Advanced configuration +## Advanced concepts ### Structured output @@ -679,215 +679,6 @@ const CustomAgentState = createAgent({ To learn more about memory, see [Memory](/oss/concepts/memory). For information on implementing long-term memory that persists across sessions, see [Long-term memory](/oss/langchain/long-term-memory). -### Before model hook - -Pre-model hook is middleware that processes state before the model is called. Use cases include message trimming, summarization, and context injection. For comprehensive middleware documentation, see [Middleware](/oss/langchain/middleware). - -```mermaid -%%{ - init: { - "fontFamily": "monospace", - "flowchart": { - "curve": "basis" - }, - "themeVariables": {"edgeLabelBackground": "transparent"} - } -}%% -graph TD - S(["\_\_start\_\_"]) - PRE(before_model) - MODEL(model) - TOOLS(tools) - END(["\_\_end\_\_"]) - - S --> PRE - PRE --> MODEL - MODEL -.-> TOOLS - MODEL -.-> END - TOOLS --> PRE - - classDef blueHighlight fill:#0a1c25,stroke:#0a455f,color:#bae6fd; - class S blueHighlight; - class END blueHighlight; -``` - -:::python - -Use the `@before_model` decorator to create middleware that runs before the model is called: - -```python wrap -from langchain.messages import RemoveMessage -from langgraph.graph.message import REMOVE_ALL_MESSAGES -from langchain.agents import create_agent, AgentState -from langchain.agents.middleware import before_model -from langgraph.runtime import Runtime - -@before_model -def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: - """Keep only the last few messages to fit context window.""" - messages = state["messages"] - - if len(messages) <= 3: - return None # No changes needed - - first_msg = messages[0] - recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:] - new_messages = [first_msg] + recent_messages - - return { - "messages": [ - RemoveMessage(id=REMOVE_ALL_MESSAGES), - *new_messages - ] - } - -agent = create_agent( - model, - tools=tools, - middleware=[trim_messages] -) -``` - - -When returning `messages` from `before_model` middleware, you should **overwrite the `messages` key** by including `RemoveMessage(id=REMOVE_ALL_MESSAGES)` first, followed by your new messages. - - -::: -:::js -```ts wrap -import { createAgent, type AgentState } from "langchain"; -import { REMOVE_ALL_MESSAGES } from "@langchain/langgraph"; -import { RemoveMessage } from "@langchain/core/messages"; - -const trimMessages = (state: AgentState) => { - const messages = state.messages; - - if (messages.length <= 3) { - return { messages }; - } - - const firstMsg = messages[0]; - const recentMessages = messages.length % 2 === 0 - ? messages.slice(-3) - : messages.slice(-4); - - const newMessages = [firstMsg, ...recentMessages]; - - return { - messages: [ - new RemoveMessage({ id: REMOVE_ALL_MESSAGES }), - ...newMessages - ] - }; -}; - -const agent = createAgent({ - model: "openai:gpt-4o", - tools, - preModelHook: trimMessages, -}); -``` -::: - -### After model hook - -After model hook is middleware that processes the model's response before tool execution. Use cases include validation, guardrails, or other post-processing. For comprehensive middleware documentation, see [Middleware](/oss/langchain/middleware). - -```mermaid -%%{ - init: { - "fontFamily": "monospace", - "flowchart": { - "curve": "basis" - }, - "themeVariables": {"edgeLabelBackground": "transparent"} - } -}%% -graph TD - S(["\_\_start\_\_"]) - MODEL(model) - POST(after_model) - TOOLS(tools) - END(["\_\_end\_\_"]) - - S --> MODEL - MODEL --> POST - POST -.-> END - POST -.-> TOOLS - TOOLS --> MODEL - - classDef blueHighlight fill:#0a1c25,stroke:#0a455f,color:#bae6fd; - class S blueHighlight; - class END blueHighlight; - class POST greenHighlight; -``` - -:::python - -Use the `@after_model` decorator to create middleware that runs after the model is called: - -```python wrap -from typing import Any -from langchain.messages import AIMessage, RemoveMessage -from langgraph.graph.message import REMOVE_ALL_MESSAGES -from langchain.agents import create_agent, AgentState -from langchain.agents.middleware import after_model -from langgraph.runtime import Runtime - -@after_model -def validate_response(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: - """Check model response for policy violations.""" - messages = state["messages"] - last_message = messages[-1] - - if "confidential" in last_message.content.lower(): - return { - "messages": [ - RemoveMessage(id=REMOVE_ALL_MESSAGES), - *messages[:-1], - AIMessage(content="I cannot share confidential information.") - ] - } - - return None # No changes needed - -agent = create_agent( - model, - tools=tools, - middleware=[validate_response] -) -``` - -::: -:::js -```ts wrap -import { createAgent, type AgentState, AIMessage, RemoveMessage } from "langchain"; -import { REMOVE_ALL_MESSAGES } from "@langchain/langgraph"; - -const validateResponse = (state: AgentState) => { - const messages = state.messages; - const lastMessage = messages.at(-1)?.text; - - if (lastMessage?.toLowerCase().includes("confidential")) { - return { - messages: [ - new RemoveMessage({ id: REMOVE_ALL_MESSAGES }), - ...state.messages.slice(0, -1), - new AIMessage("I cannot share confidential information."), - ], - }; - } - return {}; -}; - -const agent = createAgent({ - model: "openai:gpt-4o", - tools, - postModelHook: validateResponse, -}); -``` -::: - ### Streaming We've seen how the agent can be called with `.invoke` to get a final response. If the agent executes multiple steps, this may take a while. To show intermediate progress, we can stream back messages as they occur. @@ -933,3 +724,27 @@ for await (const chunk of stream) { For more details on streaming, see [Streaming](/oss/langchain/streaming). + +### Middleware + +[Middleware](/oss/langchain/middleware) provides powerful extensibility for customizing agent behavior at different stages of execution. You can use middleware to: + +- Process state before the model is called (e.g., message trimming, context injection) +- Modify or validate the model's response (e.g., guardrails, content filtering) +- Handle tool execution errors with custom logic +- Implement dynamic model selection based on state or context +- Add custom logging, monitoring, or analytics + +Middleware integrates seamlessly into the agent's execution graph, allowing you to intercept and modify data flow at key points without changing the core agent logic. + +:::python + +For comprehensive middleware documentation including decorators like `@before_model`, `@after_model`, and `@wrap_tool_call`, see [Middleware](/oss/langchain/middleware). + +::: + +:::js + +For comprehensive middleware documentation including hooks like `beforeModel`, `afterModel`, and `wrapToolCall`, see [Middleware](/oss/langchain/middleware). + +::: diff --git a/src/oss/langchain/context-engineering.mdx b/src/oss/langchain/context-engineering.mdx index 910767ddb..7b082bb05 100644 --- a/src/oss/langchain/context-engineering.mdx +++ b/src/oss/langchain/context-engineering.mdx @@ -86,7 +86,7 @@ This is not modified by the agent, and typically isn't passed into the LLM, but Examples include: user ID, DB connections -## How to do context engineering with LangChain +## Context engineering with LangChain Now we understand the basic agent loop, the importance of the model you use, and the different types of context that exist. Let's explore the concrete patterns LangChain provides for context engineering. From b9eebdbd38f3aefcbb1816ce4ea9b6b91902edf6 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 12:10:21 -0400 Subject: [PATCH 03/11] nit --- src/oss/langchain/agents.mdx | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/oss/langchain/agents.mdx b/src/oss/langchain/agents.mdx index a2776061f..32d14582b 100644 --- a/src/oss/langchain/agents.mdx +++ b/src/oss/langchain/agents.mdx @@ -62,8 +62,6 @@ Learn more about the [graph API](/oss/langgraph/graph-api). ## Core components -Agents are built from three core components: models, tools, and system prompts. For advanced control over agent behavior, you can use [middleware](/oss/langchain/middleware) to customize how your agent processes information, manages state, and responds to users. - ### Model The [model](/oss/langchain/models) is the reasoning engine of your agent. It can be specified in multiple ways, supporting both static and dynamic model selection. From 113761d2c701a5b973e536961e8d8968cb01c5c8 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 12:47:17 -0400 Subject: [PATCH 04/11] rewrite section --- src/oss/langchain/middleware.mdx | 11 +- src/oss/langchain/tools.mdx | 769 +++++++++++++++---------------- 2 files changed, 376 insertions(+), 404 deletions(-) diff --git a/src/oss/langchain/middleware.mdx b/src/oss/langchain/middleware.mdx index dfd6d4397..1e23a7954 100644 --- a/src/oss/langchain/middleware.mdx +++ b/src/oss/langchain/middleware.mdx @@ -864,12 +864,12 @@ const agent = createAgent({ Build custom middleware by implementing hooks that run at specific points in the agent execution flow. +:::python + You can create middleware in two ways: 1. **Decorator-based** - Quick and simple for single-hook middleware 2. **Class-based** - More powerful for complex middleware with multiple hooks -:::python - ## Decorator-based middleware For simple middleware that only needs a single hook, decorators provide the quickest way to add functionality: @@ -945,15 +945,12 @@ agent = create_agent( - You need a single hook - - Logic is straightforward - - No custom state schema needed - - Quick prototyping + - No complex configuration - Multiple hooks needed - - Custom state/context schema - Complex configuration - - Reusable across projects + - Reusable across projects (config on init) diff --git a/src/oss/langchain/tools.mdx b/src/oss/langchain/tools.mdx index 6e4e9381f..e01a86576 100644 --- a/src/oss/langchain/tools.mdx +++ b/src/oss/langchain/tools.mdx @@ -10,403 +10,6 @@ Many AI applications interact with users via natural language. However, some use Tools are components that [agents](/oss/langchain/agents) call to perform actions. They extend model capabilities by letting them interact with the world through well-defined inputs and outputs. Tools encapsulate a callable function and its input schema. These can be passed to compatible [chat models](/oss/langchain/models), allowing the model to decide whether to invoke a tool and with what arguments. In these scenarios, tool calling enables models to generate requests that conform to a specified input schema. -## Accessing agent data in tools - - -**Why this matters:** Tools are most powerful when they can access agent state, runtime context, and long-term memory. This enables tools to make context-aware decisions, personalize responses, and maintain information across conversations. - - -Tools can access three types of data: - -- **State** - Mutable data that flows through execution (messages, counters, custom fields) -- **Runtime Context** - Immutable configuration (user IDs, API keys, session details) -- **Long-term Memory** - Persistent data across conversations (user preferences, history) - - -:::python - - - **`state`**: The agent maintains state throughout its execution - this includes messages, custom fields, and any data your tools need to track. State flows through the graph and can be accessed and modified by tools. - - - - **`InjectedState`**: An annotation that allows tools to access the current graph state without exposing it to the LLM. This lets tools read information like message history or custom state fields while keeping the tool's schema simple. - - - Tools can access the current graph state using the `InjectedState` annotation: - - ```python wrap - from typing_extensions import Annotated - from langchain.tools import InjectedState - - # Access the current conversation state - @tool - def summarize_conversation( - state: Annotated[dict, InjectedState] - ) -> str: - """Summarize the conversation so far.""" - messages = state["messages"] - - human_msgs = sum(1 for m in messages if m.__class__.__name__ == "HumanMessage") - ai_msgs = sum(1 for m in messages if m.__class__.__name__ == "AIMessage") - tool_msgs = sum(1 for m in messages if m.__class__.__name__ == "ToolMessage") - - return f"Conversation has {human_msgs} user messages, {ai_msgs} AI responses, and {tool_msgs} tool results" - - # Access custom state fields - @tool - def get_user_preference( - pref_name: str, - preferences: Annotated[dict, InjectedState("user_preferences")] # InjectedState parameters are not visible to the model - ) -> str: - """Get a user preference value.""" - return preferences.get(pref_name, "Not set") - - # Access runtime context alongside state - from langgraph.runtime import get_runtime - from dataclasses import dataclass - - @dataclass - class Context: - user_id: str - - @tool - def get_personalized_greeting( - state: Annotated[dict, InjectedState] - ) -> str: - """Generate a personalized greeting.""" - # Access runtime context - runtime = get_runtime(Context) - user_id = runtime.context.user_id - - # Access state - message_count = len(state["messages"]) - - return f"Hello user {user_id}! This is message #{message_count}" - ``` - - - State-injected arguments are hidden from the model. For the example above, the model only sees `pref_name` in the tool schema - `preferences` is *not* included in the request. - - - - **State vs Runtime**: Use `InjectedState` for data that changes during execution (like messages or counters). Use `get_runtime()` for immutable configuration (like user IDs or API keys). - - - - - - **`Command`**: A special return type that tools can use to update the agent's state or control the graph's execution flow. Instead of just returning data, tools can return `Command`s to modify state or direct the agent to specific nodes. - - - Use a tool that returns a `Command` to update the agent state: - - ```python wrap - from langgraph.types import Command - from langchain.messages import RemoveMessage - from langgraph.graph.message import REMOVE_ALL_MESSAGES - from langchain.tools import tool, InjectedToolCallId - from typing_extensions import Annotated - - # Update the conversation history by removing all messages - @tool - def clear_conversation() -> Command: - """Clear the conversation history.""" - - return Command( - update={ - "messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)], - } - ) - - # Update the user_name in the agent state - @tool - def update_user_name( - new_name: str, - tool_call_id: Annotated[dict, InjectedToolCallId] - ) -> Command: - """Update the user's name.""" - return Command(update={"user_name": new_name}) - ``` - -::: - - - - **`runtime`**: The execution environment of your agent, containing immutable configuration and contextual data that persists throughout the agent's execution (e.g., user IDs, session details, or application-specific configuration). - - - :::python - Tools can access an agent's runtime context through `get_runtime`: - - ```python wrap - from dataclasses import dataclass - from langchain_openai import ChatOpenAI - from langchain.agents import create_agent - from langchain.tools import tool - from langgraph.runtime import get_runtime - - USER_DATABASE = { - "user123": { - "name": "Alice Johnson", - "account_type": "Premium", - "balance": 5000, - "email": "alice@example.com" - }, - "user456": { - "name": "Bob Smith", - "account_type": "Standard", - "balance": 1200, - "email": "bob@example.com" - } - } - - @dataclass - class UserContext: - user_id: str - - @tool - def get_account_info() -> str: - """Get the current user's account information.""" - runtime = get_runtime(UserContext) - user_id = runtime.context.user_id - - if user_id in USER_DATABASE: - user = USER_DATABASE[user_id] - return f"Account holder: {user['name']}\nType: {user['account_type']}\nBalance: ${user['balance']}" - return "User not found" - - model = ChatOpenAI(model="gpt-4o") - agent = create_agent( - model, - tools=[get_account_info], - context_schema=UserContext, - system_prompt="You are a financial assistant." - ) - - result = agent.invoke( - {"messages": [{"role": "user", "content": "What's my current balance?"}]}, - context=UserContext(user_id="user123") - ) - ``` - ::: - :::js - Tools can access an agent's runtime context through the `config` parameter: - - ```ts wrap - import { z } from "zod" - import { ChatOpenAI } from "@langchain/openai" - import { createAgent } from "langchain" - - const getUserName = tool( - (_, config) => { - return config.context.user_name - }, - { - name: "get_user_name", - description: "Get the user's name.", - schema: z.object({}), - } - ); - - const contextSchema = z.object({ - user_name: z.string(), - }); - - const agent = createAgent({ - model: new ChatOpenAI({ model: "gpt-4o" }), - tools: [getUserName], - contextSchema, - }); - - const result = await agent.invoke( - { - messages: [{ role: "user", content: "What is my name?" }] - }, - { - context: { user_name: "John Smith" } - } - ); - ``` - ::: - - - - - **`store`**: LangChain's persistence layer. An agent's long-term memory store, e.g. user-specific or application-specific data stored across conversations. - - - :::python - Tools can access an agent's store through `get_store`: - - ```python wrap - from langgraph.config import get_store - - @tool - def get_user_info(user_id: str) -> str: - """Look up user info.""" - store = get_store() - user_info = store.get(("users",), user_id) - return str(user_info.value) if user_info else "Unknown user" - ``` - ::: - :::js - You can initialize an `InMemoryStore` to store long-term memory: - - ```ts wrap - import { z } from "zod"; - import { createAgent, InMemoryStore } from "langchain"; - import { ChatOpenAI } from "@langchain/openai"; - - const store = new InMemoryStore(); - - const getUserInfo = tool( - ({ user_id }) => { - return store.get(["users"], user_id) - }, - { - name: "get_user_info", - description: "Look up user info.", - schema: z.object({ - user_id: z.string(), - }), - } - ); - - const agent = createAgent({ - model: new ChatOpenAI({ model: "gpt-4o" }), - tools: [getUserInfo], - store, - }); - ``` - ::: - - - - To update long-term memory, you can use the `.put()` method of `InMemoryStore`. A complete example of persistent memory across sessions: - - :::python - - ```python wrap expandable - from typing import Any - from langgraph.config import get_store - from langgraph.store.memory import InMemoryStore - from langchain.agents import create_agent - from langchain.tools import tool - - @tool - def get_user_info(user_id: str) -> str: - """Look up user info.""" - store = get_store() - user_info = store.get(("users",), user_id) - return str(user_info.value) if user_info else "Unknown user" - - @tool - def save_user_info(user_id: str, user_info: dict[str, Any]) -> str: - """Save user info.""" - store = get_store() - store.put(("users",), user_id, user_info) - return "Successfully saved user info." - - store = InMemoryStore() - agent = create_agent( - model, - tools=[get_user_info, save_user_info], - store=store - ) - - # First session: save user info - agent.invoke({ - "messages": [{"role": "user", "content": "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev"}] - }) - - # Second session: get user info - agent.invoke({ - "messages": [{"role": "user", "content": "Get user info for user with id 'abc123'"}] - }) - # Here is the user info for user with ID "abc123": - # - Name: Foo - # - Age: 25 - # - Email: foo@langchain.dev - ``` - ::: - - :::js - - ```ts wrap expandable - import { z } from "zod"; - import { createAgent, tool } from "langchain"; - import { InMemoryStore } from "@langchain/langgraph"; - import { ChatOpenAI } from "@langchain/openai"; - - const store = new InMemoryStore(); - - const getUserInfo = tool( - async ({ user_id }) => { - const value = await store.get(["users"], user_id); - console.log("get_user_info", user_id, value); - return value; - }, - { - name: "get_user_info", - description: "Look up user info.", - schema: z.object({ - user_id: z.string(), - }), - } - ); - - const saveUserInfo = tool( - async ({ user_id, name, age, email }) => { - console.log("save_user_info", user_id, name, age, email); - await store.put(["users"], user_id, { name, age, email }); - return "Successfully saved user info."; - }, - { - name: "save_user_info", - description: "Save user info.", - schema: z.object({ - user_id: z.string(), - name: z.string(), - age: z.number(), - email: z.string(), - }), - } - ); - - const agent = createAgent({ - llm: new ChatOpenAI({ model: "gpt-4o" }), - tools: [getUserInfo, saveUserInfo], - store, - }); - - // First session: save user info - await agent.invoke({ - messages: [ - { - role: "user", - content: "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev", - }, - ], - }); - - // Second session: get user info - const result = await agent.invoke({ - messages: [ - { role: "user", content: "Get user info for user with id 'abc123'" }, - ], - }); - - console.log(result); - // Here is the user info for user with ID "abc123": - // - Name: Foo - // - Age: 25 - // - Email: foo@langchain.dev - ``` - ::: - - - ## Create tools ### Basic tool definition @@ -533,3 +136,375 @@ Define complex inputs with Pydantic models or JSON schemas: ::: +## Accessing Context + + +**Why this matters:** Tools are most powerful when they can access agent state, runtime context, and long-term memory. This enables tools to make context-aware decisions, personalize responses, and maintain information across conversations. + + +Tools can access different types of data: + +- **State** - Mutable data that flows through execution (messages, counters, custom fields) +- **Runtime** - Access to context, memory (store), and streaming capabilities via `get_runtime()` + +### State + +Use `InjectedState` to access and modify the agent's state during execution. State includes messages, custom fields, and any data your tools need to track. + + +**`InjectedState`**: An annotation that allows tools to access the current graph state without exposing it to the LLM. This lets tools read information like message history or custom state fields while keeping the tool's schema simple. + + +:::python +**Accessing state:** + +Tools can access the current graph state using the `InjectedState` annotation: + +```python wrap +from typing_extensions import Annotated +from langchain.tools import InjectedState + +# Access the current conversation state +@tool +def summarize_conversation( + state: Annotated[dict, InjectedState] +) -> str: + """Summarize the conversation so far.""" + messages = state["messages"] + + human_msgs = sum(1 for m in messages if m.__class__.__name__ == "HumanMessage") + ai_msgs = sum(1 for m in messages if m.__class__.__name__ == "AIMessage") + tool_msgs = sum(1 for m in messages if m.__class__.__name__ == "ToolMessage") + + return f"Conversation has {human_msgs} user messages, {ai_msgs} AI responses, and {tool_msgs} tool results" + +# Access custom state fields +@tool +def get_user_preference( + pref_name: str, + preferences: Annotated[dict, InjectedState("user_preferences")] # InjectedState parameters are not visible to the model +) -> str: + """Get a user preference value.""" + return preferences.get(pref_name, "Not set") +``` + + +State-injected arguments are hidden from the model. For the example above, the model only sees `pref_name` in the tool schema - `preferences` is *not* included in the request. + + +**Updating state:** + +Use `Command` to update the agent's state or control the graph's execution flow: + +```python wrap +from langgraph.types import Command +from langchain.messages import RemoveMessage +from langgraph.graph.message import REMOVE_ALL_MESSAGES +from langchain.tools import tool, InjectedToolCallId +from typing_extensions import Annotated + +# Update the conversation history by removing all messages +@tool +def clear_conversation() -> Command: + """Clear the conversation history.""" + + return Command( + update={ + "messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)], + } + ) + +# Update the user_name in the agent state +@tool +def update_user_name( + new_name: str, + tool_call_id: Annotated[dict, InjectedToolCallId] +) -> Command: + """Update the user's name.""" + return Command(update={"user_name": new_name}) +``` +::: + +### Runtime + +Use `get_runtime()` to access immutable configuration, context, memory (store), and streaming capabilities. The runtime provides access to data that persists across the agent's execution. + +#### Context + +Access immutable configuration and contextual data like user IDs, session details, or application-specific configuration. + +:::python +Tools can access runtime context through `get_runtime()`: + +```python wrap +from dataclasses import dataclass +from langchain_openai import ChatOpenAI +from langchain.agents import create_agent +from langchain.tools import tool +from langgraph.runtime import get_runtime + +USER_DATABASE = { + "user123": { + "name": "Alice Johnson", + "account_type": "Premium", + "balance": 5000, + "email": "alice@example.com" + }, + "user456": { + "name": "Bob Smith", + "account_type": "Standard", + "balance": 1200, + "email": "bob@example.com" + } +} + +@dataclass +class UserContext: + user_id: str + +@tool +def get_account_info() -> str: + """Get the current user's account information.""" + runtime = get_runtime(UserContext) + user_id = runtime.context.user_id + + if user_id in USER_DATABASE: + user = USER_DATABASE[user_id] + return f"Account holder: {user['name']}\nType: {user['account_type']}\nBalance: ${user['balance']}" + return "User not found" + +model = ChatOpenAI(model="gpt-4o") +agent = create_agent( + model, + tools=[get_account_info], + context_schema=UserContext, + system_prompt="You are a financial assistant." +) + +result = agent.invoke( + {"messages": [{"role": "user", "content": "What's my current balance?"}]}, + context=UserContext(user_id="user123") +) +``` +::: + +:::js +Tools can access an agent's runtime context through the `config` parameter: + +```ts wrap +import { z } from "zod" +import { ChatOpenAI } from "@langchain/openai" +import { createAgent } from "langchain" + +const getUserName = tool( + (_, config) => { + return config.context.user_name + }, + { + name: "get_user_name", + description: "Get the user's name.", + schema: z.object({}), + } +); + +const contextSchema = z.object({ + user_name: z.string(), +}); + +const agent = createAgent({ + model: new ChatOpenAI({ model: "gpt-4o" }), + tools: [getUserName], + contextSchema, +}); + +const result = await agent.invoke( + { + messages: [{ role: "user", content: "What is my name?" }] + }, + { + context: { user_name: "John Smith" } + } +); +``` +::: + +#### Memory (Store) + +Access persistent data across conversations using the store. The store is accessed via `get_runtime().store` and allows you to save and retrieve user-specific or application-specific data. + +:::python +Tools can access and update the store through `get_store()`: + +```python wrap expandable +from typing import Any +from langgraph.config import get_store +from langgraph.store.memory import InMemoryStore +from langchain.agents import create_agent +from langchain.tools import tool + +# Access memory +@tool +def get_user_info(user_id: str) -> str: + """Look up user info.""" + store = get_store() + user_info = store.get(("users",), user_id) + return str(user_info.value) if user_info else "Unknown user" + +# Update memory +@tool +def save_user_info(user_id: str, user_info: dict[str, Any]) -> str: + """Save user info.""" + store = get_store() + store.put(("users",), user_id, user_info) + return "Successfully saved user info." + +store = InMemoryStore() +agent = create_agent( + model, + tools=[get_user_info, save_user_info], + store=store +) + +# First session: save user info +agent.invoke({ + "messages": [{"role": "user", "content": "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev"}] +}) + +# Second session: get user info +agent.invoke({ + "messages": [{"role": "user", "content": "Get user info for user with id 'abc123'"}] +}) +# Here is the user info for user with ID "abc123": +# - Name: Foo +# - Age: 25 +# - Email: foo@langchain.dev +``` +::: + +:::js +```ts wrap expandable +import { z } from "zod"; +import { createAgent, tool } from "langchain"; +import { InMemoryStore } from "@langchain/langgraph"; +import { ChatOpenAI } from "@langchain/openai"; + +const store = new InMemoryStore(); + +// Access memory +const getUserInfo = tool( + async ({ user_id }) => { + const value = await store.get(["users"], user_id); + console.log("get_user_info", user_id, value); + return value; + }, + { + name: "get_user_info", + description: "Look up user info.", + schema: z.object({ + user_id: z.string(), + }), + } +); + +// Update memory +const saveUserInfo = tool( + async ({ user_id, name, age, email }) => { + console.log("save_user_info", user_id, name, age, email); + await store.put(["users"], user_id, { name, age, email }); + return "Successfully saved user info."; + }, + { + name: "save_user_info", + description: "Save user info.", + schema: z.object({ + user_id: z.string(), + name: z.string(), + age: z.number(), + email: z.string(), + }), + } +); + +const agent = createAgent({ + llm: new ChatOpenAI({ model: "gpt-4o" }), + tools: [getUserInfo, saveUserInfo], + store, +}); + +// First session: save user info +await agent.invoke({ + messages: [ + { + role: "user", + content: "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev", + }, + ], +}); + +// Second session: get user info +const result = await agent.invoke({ + messages: [ + { role: "user", content: "Get user info for user with id 'abc123'" }, + ], +}); + +console.log(result); +// Here is the user info for user with ID "abc123": +// - Name: Foo +// - Age: 25 +// - Email: foo@langchain.dev +``` +::: + +#### Stream Writer + +Stream custom updates from tools as they execute using `get_stream_writer()`. This is useful for providing real-time feedback to users about what a tool is doing. + +:::python +```python wrap +from langchain.tools import tool +from langgraph.config import get_stream_writer + +@tool +def get_weather(city: str) -> str: + """Get weather for a given city.""" + writer = get_stream_writer() + + # Stream custom updates as the tool executes + writer(f"Looking up data for city: {city}") + writer(f"Acquired data for city: {city}") + + return f"It's always sunny in {city}!" +``` + + +If you use `get_stream_writer()` inside your tool, the tool must be invoked within a LangGraph execution context. See [Streaming](/oss/langchain/streaming) for more details. + +::: + +:::js +```ts wrap +import { z } from "zod"; +import { tool } from "langchain"; + +const getWeather = tool( + ({ city }, config) => { + const writer = config.streamWriter; + + // Stream custom updates as the tool executes + writer(`Looking up data for city: ${city}`); + writer(`Acquired data for city: ${city}`); + + return `It's always sunny in ${city}!`; + }, + { + name: "get_weather", + description: "Get weather for a given city.", + schema: z.object({ + city: z.string(), + }), + } +); +``` +::: + From 95070ecc9832a0a2da860085da50deeecf7fda93 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 12:49:17 -0400 Subject: [PATCH 05/11] patterns --- src/oss/langchain/long-term-memory.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/src/oss/langchain/long-term-memory.mdx b/src/oss/langchain/long-term-memory.mdx index 5197f4c3f..f0230eb2e 100644 --- a/src/oss/langchain/long-term-memory.mdx +++ b/src/oss/langchain/long-term-memory.mdx @@ -105,7 +105,6 @@ from dataclasses import dataclass from langchain_core.runnables import RunnableConfig from langchain.agents import create_agent -from langgraph.config import get_store from langgraph.runtime import get_runtime from langgraph.store.memory import InMemoryStore From c1e302adacc4ce66326ef06e9887189f7c6b7146 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 13:04:49 -0400 Subject: [PATCH 06/11] some improvements --- src/oss/langchain/short-term-memory.mdx | 166 ++++++++++++++---------- 1 file changed, 94 insertions(+), 72 deletions(-) diff --git a/src/oss/langchain/short-term-memory.mdx b/src/oss/langchain/short-term-memory.mdx index 81e3394cb..4af99522d 100644 --- a/src/oss/langchain/short-term-memory.mdx +++ b/src/oss/langchain/short-term-memory.mdx @@ -197,56 +197,41 @@ Most LLMs have a maximum supported context window (denominated in tokens). One way to decide when to truncate messages is to count the tokens in the message history and truncate whenever it approaches that limit. If you're using LangChain, you can use the trim messages utility and specify the number of tokens to keep from the list, as well as the `strategy` (e.g., keep the last `maxTokens`) to use for handling the boundary. :::python -To trim message history in an agent, use the `@before_model` middleware decorator with the [`trim_messages`](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html) function: +To trim message history in an agent, use the `@before_model` middleware decorator: ```python -from langchain_core.messages.utils import trim_messages, count_tokens_approximately -from langchain.messages import BaseMessage +from langchain.messages import RemoveMessage +from langgraph.graph.message import REMOVE_ALL_MESSAGES from langgraph.checkpoint.memory import InMemorySaver from langchain.agents import create_agent, AgentState from langchain.agents.middleware import before_model from langgraph.runtime import Runtime -from langchain_core.runnables import RunnableConfig +from typing import Any @before_model -def trim_message_history(state: AgentState, runtime: Runtime) -> dict[str, list[BaseMessage]] | None: - """Trim messages before every model call to stay within context limits.""" - trimmed_messages = trim_messages( - state["messages"], - strategy="last", - token_counter=count_tokens_approximately, - max_tokens=384, - start_on="human", - end_on=("human", "tool"), - ) - # Return None if no trimming needed - if len(trimmed_messages) == len(state["messages"]): - return None - return {"messages": trimmed_messages} - - -checkpointer = InMemorySaver() -agent = create_agent( - "openai:gpt-5-nano", - tools=[], - middleware=[trim_message_history], - checkpointer=checkpointer, -) +def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: + """Keep only the last few messages to fit context window.""" + messages = state["messages"] -config: RunnableConfig = {"configurable": {"thread_id": "1"}} + if len(messages) <= 3: + return None # No changes needed -agent.invoke({"messages": "hi, my name is bob"}, config) -agent.invoke({"messages": "write a short poem about cats"}, config) -agent.invoke({"messages": "now do the same but for dogs"}, config) -final_response = agent.invoke({"messages": "what's my name?"}, config) + first_msg = messages[0] + recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:] + new_messages = [first_msg] + recent_messages -final_response["messages"][-1].pretty_print() -""" -================================== Ai Message ================================== + return { + "messages": [ + RemoveMessage(id=REMOVE_ALL_MESSAGES), + *new_messages + ] + } -Your name is Bob. You told me that earlier. -If you'd like me to call you a nickname or use a different name, just say the word. -""" +agent = create_agent( + model, + tools=tools, + middleware=[trim_messages] +) ``` ::: @@ -551,11 +536,11 @@ console.log(finalResponse.messages.at(-1)?.content); See [summarizationMiddleware](/oss/langchain/middleware#summarization) for more configuration options. ::: -## Access +## Access memory You can access and modify the short-term memory (state) of an agent in several ways: -### In tools +### Tools #### Read short-term memory in a tool @@ -759,7 +744,7 @@ await agent.invoke( ``` ::: -### In middleware (dynamic prompts) +### Prompt Access short term memory (state) in middleware to create dynamic prompts based on conversation history or custom state fields. @@ -904,48 +889,43 @@ for (const message of result.messages) { ``` ::: -### In middleware (message processing) +### Before model -Access short term memory (state) in middleware to process messages before or after model calls. +Access short term memory (state) in `@before_model` middleware to process messages before model calls. :::python ```python -from langchain_core.messages.utils import trim_messages, count_tokens_approximately -from langchain.messages import BaseMessage, RemoveMessage +from langchain.messages import RemoveMessage +from langgraph.graph.message import REMOVE_ALL_MESSAGES from langgraph.checkpoint.memory import InMemorySaver from langchain.agents import create_agent, AgentState -from langchain.agents.middleware import before_model, after_model +from langchain.agents.middleware import before_model from langgraph.runtime import Runtime +from typing import Any @before_model -def trim_message_history(state: AgentState, runtime: Runtime) -> dict | None: - """Trim messages before model call.""" - trimmed_messages = trim_messages( - state["messages"], - strategy="last", - token_counter=count_tokens_approximately, - max_tokens=384, - start_on="human", - end_on=("human", "tool"), - ) - if len(trimmed_messages) == len(state["messages"]): - return None - return {"messages": trimmed_messages} +def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: + """Keep only the last few messages to fit context window.""" + messages = state["messages"] -@after_model -def validate_response(state: AgentState, runtime: Runtime) -> dict | None: - """Remove messages containing sensitive words.""" - STOP_WORDS = ["password", "secret"] - last_message = state["messages"][-1] - if any(word in last_message.content for word in STOP_WORDS): - return {"messages": [RemoveMessage(id=last_message.id)]} - return None + if len(messages) <= 3: + return None # No changes needed + + first_msg = messages[0] + recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:] + new_messages = [first_msg] + recent_messages + + return { + "messages": [ + RemoveMessage(id=REMOVE_ALL_MESSAGES), + *new_messages + ] + } agent = create_agent( - model="openai:gpt-5-nano", - tools=[], - middleware=[trim_message_history, validate_response], - checkpointer=InMemorySaver(), + model, + tools=tools, + middleware=[trim_messages] ) ``` ::: @@ -968,6 +948,48 @@ const trimMessageHistory = createMiddleware({ }, }); +const agent = createAgent({ + model: "openai:gpt-5-nano", + tools: [], + middleware: [trimMessageHistory], +}); +``` +::: + +### After model + +Access short term memory (state) in `@after_model` middleware to process messages after model calls. + +:::python +```python +from langchain.messages import RemoveMessage +from langgraph.checkpoint.memory import InMemorySaver +from langchain.agents import create_agent, AgentState +from langchain.agents.middleware import after_model +from langgraph.runtime import Runtime + +@after_model +def validate_response(state: AgentState, runtime: Runtime) -> dict | None: + """Remove messages containing sensitive words.""" + STOP_WORDS = ["password", "secret"] + last_message = state["messages"][-1] + if any(word in last_message.content for word in STOP_WORDS): + return {"messages": [RemoveMessage(id=last_message.id)]} + return None + +agent = create_agent( + model="openai:gpt-5-nano", + tools=[], + middleware=[validate_response], + checkpointer=InMemorySaver(), +) +``` +::: +:::js +```typescript +import { RemoveMessage } from "@langchain/core/messages"; +import { createAgent, createMiddleware, type AgentState } from "langchain"; + const validateResponse = createMiddleware({ name: "ValidateResponse", afterModel: (state) => { @@ -984,7 +1006,7 @@ const validateResponse = createMiddleware({ const agent = createAgent({ model: "openai:gpt-5-nano", tools: [], - middleware: [trimMessageHistory, validateResponse], + middleware: [validateResponse], }); ``` ::: From 7c3cb8a1b093af809ee02297d9fabff759ecc29c Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 13:08:14 -0400 Subject: [PATCH 07/11] little --- src/oss/langchain/context-engineering.mdx | 5 ++--- src/oss/langchain/tools.mdx | 16 ++++++++-------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/src/oss/langchain/context-engineering.mdx b/src/oss/langchain/context-engineering.mdx index 7b082bb05..1b506830f 100644 --- a/src/oss/langchain/context-engineering.mdx +++ b/src/oss/langchain/context-engineering.mdx @@ -130,7 +130,6 @@ For instructions that depend on context (user profile, preferences, session data from dataclasses import dataclass from langchain.agents import create_agent from langchain.agents.middleware import dynamic_prompt, ModelRequest -from langgraph.config import get_store @dataclass class Context: @@ -142,7 +141,7 @@ def personalized_prompt(request: ModelRequest) -> str: user_id = request.runtime.context.user_id # Look up user preferences from long-term memory - store = get_store() + store = request.runtime.store user_prefs = store.get(("users",), user_id) # Access session state @@ -324,7 +323,7 @@ def search_documents( user_id = runtime.context.user_id # Access long-term memory for user preferences - store = get_store() + store = runtime.store search_prefs = store.get(("preferences", user_id), "search") # Access session state diff --git a/src/oss/langchain/tools.mdx b/src/oss/langchain/tools.mdx index e01a86576..af75ced2d 100644 --- a/src/oss/langchain/tools.mdx +++ b/src/oss/langchain/tools.mdx @@ -333,11 +333,11 @@ const result = await agent.invoke( Access persistent data across conversations using the store. The store is accessed via `get_runtime().store` and allows you to save and retrieve user-specific or application-specific data. :::python -Tools can access and update the store through `get_store()`: +Tools can access and update the store through `get_runtime().store`: ```python wrap expandable from typing import Any -from langgraph.config import get_store +from langgraph.runtime import get_runtime from langgraph.store.memory import InMemoryStore from langchain.agents import create_agent from langchain.tools import tool @@ -346,7 +346,7 @@ from langchain.tools import tool @tool def get_user_info(user_id: str) -> str: """Look up user info.""" - store = get_store() + store = get_runtime().store user_info = store.get(("users",), user_id) return str(user_info.value) if user_info else "Unknown user" @@ -354,7 +354,7 @@ def get_user_info(user_id: str) -> str: @tool def save_user_info(user_id: str, user_info: dict[str, Any]) -> str: """Save user info.""" - store = get_store() + store = get_runtime().store store.put(("users",), user_id, user_info) return "Successfully saved user info." @@ -458,17 +458,17 @@ console.log(result); #### Stream Writer -Stream custom updates from tools as they execute using `get_stream_writer()`. This is useful for providing real-time feedback to users about what a tool is doing. +Stream custom updates from tools as they execute using `get_runtime().stream_writer`. This is useful for providing real-time feedback to users about what a tool is doing. :::python ```python wrap from langchain.tools import tool -from langgraph.config import get_stream_writer +from langgraph.runtime import get_runtime @tool def get_weather(city: str) -> str: """Get weather for a given city.""" - writer = get_stream_writer() + writer = get_runtime().stream_writer # Stream custom updates as the tool executes writer(f"Looking up data for city: {city}") @@ -478,7 +478,7 @@ def get_weather(city: str) -> str: ``` -If you use `get_stream_writer()` inside your tool, the tool must be invoked within a LangGraph execution context. See [Streaming](/oss/langchain/streaming) for more details. +If you use `get_runtime().stream_writer` inside your tool, the tool must be invoked within a LangGraph execution context. See [Streaming](/oss/langchain/streaming) for more details. ::: From ffda58d6708cbd9514bb70a0fe8252a06b770b96 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 13:11:37 -0400 Subject: [PATCH 08/11] output snippet woo --- src/oss/langchain/short-term-memory.mdx | 30 +++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/src/oss/langchain/short-term-memory.mdx b/src/oss/langchain/short-term-memory.mdx index 4af99522d..ad8c61862 100644 --- a/src/oss/langchain/short-term-memory.mdx +++ b/src/oss/langchain/short-term-memory.mdx @@ -232,6 +232,21 @@ agent = create_agent( tools=tools, middleware=[trim_messages] ) + +config: RunnableConfig = {"configurable": {"thread_id": "1"}} + +agent.invoke({"messages": "hi, my name is bob"}, config) +agent.invoke({"messages": "write a short poem about cats"}, config) +agent.invoke({"messages": "now do the same but for dogs"}, config) +final_response = agent.invoke({"messages": "what's my name?"}, config) + +final_response["messages"][-1].pretty_print() +""" +================================== Ai Message ================================== + +Your name is Bob. You told me that earlier. +If you'd like me to call you a nickname or use a different name, just say the word. +""" ``` ::: @@ -927,6 +942,21 @@ agent = create_agent( tools=tools, middleware=[trim_messages] ) + +config: RunnableConfig = {"configurable": {"thread_id": "1"}} + +agent.invoke({"messages": "hi, my name is bob"}, config) +agent.invoke({"messages": "write a short poem about cats"}, config) +agent.invoke({"messages": "now do the same but for dogs"}, config) +final_response = agent.invoke({"messages": "what's my name?"}, config) + +final_response["messages"][-1].pretty_print() +""" +================================== Ai Message ================================== + +Your name is Bob. You told me that earlier. +If you'd like me to call you a nickname or use a different name, just say the word. +""" ``` ::: :::js From 5315e5d019a603eebeb8250f63ea995c8c1998ad Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 13:13:49 -0400 Subject: [PATCH 09/11] mermaids --- src/oss/langchain/short-term-memory.mdx | 54 +++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/src/oss/langchain/short-term-memory.mdx b/src/oss/langchain/short-term-memory.mdx index ad8c61862..0e8950ff0 100644 --- a/src/oss/langchain/short-term-memory.mdx +++ b/src/oss/langchain/short-term-memory.mdx @@ -908,6 +908,33 @@ for (const message of result.messages) { Access short term memory (state) in `@before_model` middleware to process messages before model calls. + +```mermaid +%%{ + init: { + "fontFamily": "monospace", + "flowchart": { + "curve": "basis" + }, + "themeVariables": {"edgeLabelBackground": "transparent"} + } +}%% +graph TD + S(["\_\_start\_\_"]) + PRE(before_model) + MODEL(model) + TOOLS(tools) + END(["\_\_end\_\_"]) + S --> PRE + PRE --> MODEL + MODEL -.-> TOOLS + MODEL -.-> END + TOOLS --> PRE + classDef blueHighlight fill:#0a1c25,stroke:#0a455f,color:#bae6fd; + class S blueHighlight; + class END blueHighlight; +``` + :::python ```python from langchain.messages import RemoveMessage @@ -990,6 +1017,33 @@ const agent = createAgent({ Access short term memory (state) in `@after_model` middleware to process messages after model calls. +```mermaid +%%{ + init: { + "fontFamily": "monospace", + "flowchart": { + "curve": "basis" + }, + "themeVariables": {"edgeLabelBackground": "transparent"} + } +}%% +graph TD + S(["\_\_start\_\_"]) + MODEL(model) + POST(after_model) + TOOLS(tools) + END(["\_\_end\_\_"]) + S --> MODEL + MODEL --> POST + POST -.-> END + POST -.-> TOOLS + TOOLS --> MODEL + classDef blueHighlight fill:#0a1c25,stroke:#0a455f,color:#bae6fd; + class S blueHighlight; + class END blueHighlight; + class POST greenHighlight; +``` + :::python ```python from langchain.messages import RemoveMessage From 473a07d0d057e57926b530c1227ae900399c5705 Mon Sep 17 00:00:00 2001 From: Sydney Runkle Date: Mon, 13 Oct 2025 21:45:11 -0400 Subject: [PATCH 10/11] update size --- src/oss/langchain/middleware.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/oss/langchain/middleware.mdx b/src/oss/langchain/middleware.mdx index 1e23a7954..b32721d96 100644 --- a/src/oss/langchain/middleware.mdx +++ b/src/oss/langchain/middleware.mdx @@ -15,7 +15,7 @@ The core agent loop involves calling a model, letting it choose tools to execute Core agent loop diagram From e063d0876826659629089050b9abc2f1b837c711 Mon Sep 17 00:00:00 2001 From: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com> Date: Mon, 13 Oct 2025 21:48:40 -0400 Subject: [PATCH 11/11] Update src/oss/langchain/middleware.mdx --- src/oss/langchain/middleware.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/oss/langchain/middleware.mdx b/src/oss/langchain/middleware.mdx index b32721d96..273557c8d 100644 --- a/src/oss/langchain/middleware.mdx +++ b/src/oss/langchain/middleware.mdx @@ -887,7 +887,7 @@ def log_before_model(state: AgentState, runtime: Runtime) -> dict[str, Any] | No return None # Node-style: validation after model calls -@after_model +@after_model(can_jump_to=["end"]) def validate_output(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: last_message = state["messages"][-1] if "BLOCKED" in last_message.content: