`wrap_model_call` middleware ToolStrategy narrowing has no effect — model still sees all structured output tools

[leforebs95]

Checked other resources

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Related Issues / PRs

Issue #36420

Reproduction Steps / Example Code (Python)

from typing import Literal, Union
from collections.abc import Callable

from pydantic import BaseModel, Field
from langchain.agents import create_agent
from langchain.agents.middleware import (
    ModelRequest,
    ModelResponse,
    wrap_model_call,
    before_agent,
)
from langchain.agents.structured_output import ToolStrategy


# Two possible response types
class DetailedAnswer(BaseModel):
    type: Literal["DetailedAnswer"] = "DetailedAnswer"
    content: str = Field(description="A detailed, thorough answer")

class BriefAnswer(BaseModel):
    type: Literal["BriefAnswer"] = "BriefAnswer"
    content: str = Field(description="A brief, one-sentence answer")

FullResponse = Union[DetailedAnswer, BriefAnswer]


# Middleware that should narrow to only DetailedAnswer
@wrap_model_call
async def force_detailed(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
    """Narrow response format to only allow DetailedAnswer."""
    narrowed = request.override(
        response_format=ToolStrategy(DetailedAnswer)
    )
    return await handler(narrowed)


agent = create_agent(
    model="anthropic:claude-haiku-4-5-20251001",
    tools=[],
    middleware=[force_detailed],
    response_format=ToolStrategy(FullResponse),
)

# Ask the model something — it CAN still choose BriefAnswer
# even though middleware narrowed to DetailedAnswer only
result = agent.invoke({
    "messages": [{"role": "user", "content": "What is 2+2? Be brief."}]
})

# BUG: model may return BriefAnswer despite middleware narrowing
print(type(result["structured_response"]))
# Expected: always DetailedAnswer
# Actual: may be BriefAnswer because both tools are bound to the model

Error Message and Stack Trace (if applicable)

Description

Description

When a @wrap_model_call middleware narrows response_format from a union ToolStrategy to a single-type subset via request.override(response_format=ToolStrategy(SubsetType)), the narrowing is validated (lines 1241-1248 of factory.py confirm the subset is valid) but never enforced. The model is still bound with all structured output tools from the original union, and can freely choose any of them.

This contradicts:

• The code comment at factory.py:1234-1239: "Middleware is allowed to change the response format to a subset of the original structured tools when using ToolStrategy"
• The discussion on Dynamic response_format with create_agent #34239 where this narrowing pattern is described as the intended approach for state-dependent structured output with ToolStrategy

Root Cause

structured_output_tools is a closure variable built once from the initial response_format at create_agent() time. Five locations in _get_bound_model() and the graph edge functions use this full set instead of the runtime-narrowed effective_response_format:

Location	Code	Impact
factory.py:1220	structured_tools = [info.tool for info in structured_output_tools.values()]	All original structured tools are added to final_tools and bound to the model, regardless of narrowed format
factory.py:1071	tc["name"] in structured_output_tools	Output handler accepts tool calls for types outside the narrowed format
factory.py:1100	structured_output_tools[tool_call["name"]]	Successfully parses responses of the wrong type
factory.py:1728	c["name"] not in structured_output_tools	Edge routing treats wrong-type calls as "handled" structured output
factory.py:1808	t.name in structured_output_tools	Agent loop exits on wrong-type structured response

The validation at lines 1241-1248 correctly confirms the narrowed format is a valid subset, but none of the downstream code filters to that subset.

Expected behavior

When middleware narrows response_format to ToolStrategy(SubsetType):

1. Only the structured output tools in the narrowed format should be bound to the model
2. The output handler should only accept tool calls matching the narrowed format
3. Edge routing should only treat narrowed-format tool calls as structured output

Actual behavior

All structured output tools from the original create_agent(response_format=...) are always bound to the model. The model can choose any of them regardless of middleware narrowing. The narrowed effective_response_format is validated but never used for filtering.

Suggested fix

In _get_bound_model(), filter structured output tools to only those present in effective_response_format.schema_specs:

# Current (line 1220): adds ALL structured tools
structured_tools = [info.tool for info in structured_output_tools.values()]

# Fix: filter to only tools in the narrowed format
if isinstance(effective_response_format, ToolStrategy):
    narrowed_names = {tc.name for tc in effective_response_format.schema_specs}
    structured_tools = [
        info.tool for name, info in structured_output_tools.items()
        if name in narrowed_names
    ]

The same filtering pattern would need to be applied to _handle_model_output (line 1071) and the edge functions (lines 1728, 1808) — though these are trickier since they're closures that don't have access to the runtime-narrowed format. One approach: store the effective_response_format in agent state so edge functions can read it, or restructure the edge functions to check both the full set and the state-stored narrowed set.

A simpler alternative for the edge functions: since _handle_model_output already receives effective_response_format, it could reject tool calls not in the narrowed set (returning them as "unhandled" so the agent loop retries), which would make the edge function changes unnecessary.

System Info

⎿  System Information
------------------
> OS: Darwin
> OS Version: Darwin Kernel Version 25.3.0: Wed Jan 28 20:54:46 PST 2026;
root:xnu-12377.91.3~2/RELEASE_ARM64_T6000
> Python Version: 3.13.1 (main, Jan 14 2025, 23:31:50) [Clang 19.1.6 ]

 Package Information
 -------------------
 > langchain_core: 1.2.25
 > langchain: 1.2.15
 > langsmith: 0.7.24
 > langchain_anthropic: 1.4.0
 > langchain_openai: 1.1.12
 > langgraph_api: 0.7.96
 > langgraph_cli: 0.4.19
 > langgraph_runtime_inmem: 0.27.0
 > langgraph_sdk: 0.3.12

 Optional packages not installed
 -------------------------------
 > deepagents
 > deepagents-cli

 Other Dependencies
 ------------------
 > anthropic: 0.85.0
 > blockbuster: 1.5.25
 > click: 8.3.1
 > cloudpickle: 3.1.2
 > croniter: 6.0.0
 > cryptography: 46.0.6
 > grpcio: 1.78.0
 > grpcio-health-checking: 1.78.0
 > grpcio-tools: 1.78.0
 > httpx: 0.28.1
 > jsonpatch: 1.33
 > jsonschema-rs: 0.29.1
 > langgraph: 1.1.6
 > langgraph-checkpoint: 3.0.1
 > openai: 2.26.0
 > opentelemetry-api: 1.38.0
 > opentelemetry-exporter-otlp-proto-http: 1.38.0
 > opentelemetry-sdk: 1.38.0
 > orjson: 3.11.7
 > packaging: 25.0
 > protobuf: 6.33.5
 > pydantic: 2.12.4
 > pyjwt: 2.12.0
 > pytest: 9.0.2
 > python-dotenv: 1.2.2
 > pyyaml: 6.0.3
 > requests: 2.33.0
 > requests-toolbelt: 1.0.0
 > rich: 14.2.0
 > sse-starlette: 3.3.2
 > starlette: 0.49.3
 > structlog: 25.5.0
 > tenacity: 9.1.2
 > tiktoken: 0.12.0
 > truststore: 0.10.4
 > typing-extensions: 4.15.0
 > uuid-utils: 0.12.0
 > uvicorn: 0.38.0
 > vcrpy: 7.0.0
 > watchfiles: 1.1.1
 > wrapt: 2.0.1
 > xxhash: 3.6.0
 > zstandard: 0.25.0

[Vishwaspatel2401]

Hi, Brandon Lefore (@leforebs95), I'd like to work on this. Here's my analysis and proposed fix:

Root cause: structured_output_tools is a closure captured at create_agent()
time, so all five downstream call sites (factory.py:1220, 1071, 1100, 1728, 1808)
always use the full original set — regardless of what middleware narrows
effective_response_format to at call time.

Proposed fix:

Add a helper that derives the effective tool list at call time by filtering
structured_output_tools to only those whose name appears in
effective_response_format.schema_specs:

def _get_effective_structured_tools(
    structured_output_tools: list,
    effective_response_format: ToolStrategy,
) -> list:
    narrowed_names = {
        spec.schema.__name__
        for spec in effective_response_format.schema_specs
    }
    return [t for t in structured_output_tools if t.name in narrowed_names]

[seuthootDev]

Hi, I'd like to work on this. Could a maintainer assign me?

I've already analyzed the root cause and have a fix ready. My approach covers two locations:

_get_bound_model — filter tools bound to the model:

narrowed_names = {spec.name for spec in effective_response_format.schema_specs}
structured_tools = [
    info.tool for name, info in structured_output_tools.items()
    if name in narrowed_names
]

_handle_model_output — filter accepted tool calls in output handling:

effective_structured_names = {
    spec.name for spec in effective_response_format.schema_specs
}
structured_tool_calls = [
    tc for tc in output.tool_calls if tc["name"] in effective_structured_names
]

I also have 4 regression tests written and passing locally. Happy to open a PR once assigned.

[aimvik07]

This appears to be a bug report about wrap_model_call middleware not properly narrowing which tools a model can see.

To help us investigate this issue, could you please provide:

• Your Python version
• The specific LangChain version you're using

This information will help us reproduce and debug the issue more effectively.

---

This is an automated triage comment.

[aimvik07]

This appears to be a bug report regarding wrap_model_call middleware not properly narrowing ToolStrategy when structured output tools are present.

Thanks for reporting this! To help us investigate, could you please provide:

• Your Python version
• Your exact LangChain version (e.g., langchain==0.1.0)

This additional context will help us reproduce and debug the issue more effectively.

---

This is an automated triage comment.

[aimvik07]

This appears to be a bug report about wrap_model_call middleware not properly narrowing ToolStrategy options when using structured output tools.

To help investigate this issue, could you please provide:

• Your Python version
• The exact LangChain version you're using

These details will help us reproduce and troubleshoot the problem more effectively.

---

This is an automated triage comment.

[lucasgodoy]

Hello, any news on that? I'm also experiencing the same bug. Thanks.