fix(run): keep input item order when collapsing duplicates

[hsusul]

Summary

deduplicate_input_items_preferring_latest collapses items that share a stable identifier (id, call_id, approval_request_id) and keeps the latest value. It is implemented by reversing the list, running the first-wins deduplicate_input_items, and reversing back:

return list(reversed(deduplicate_input_items(list(reversed(items)))))

That keeps the latest value, but it also relocates the surviving item to the position of its last duplicate rather than its first. Deduplication therefore reorders the conversation.

The concrete failure: when a function_call is repeated after its function_call_output, the call is moved behind the output. The Responses API rejects that ordering, and the same helper also decides the order items are written to a session, so an invalid ordering can be persisted and replayed on every later turn.

Affected component: src/agents/run_internal/items.py. The helper feeds three call sites:

• run_internal/run_loop.py — the final model input for streamed and non-streamed turns (applied to whatever RunConfig.call_model_input_filter returned).
• run_internal/session_persistence.py:287 — prepare_input_with_session.
• run_internal/session_persistence.py:423 — save_result_to_session, i.e. the persisted session history.

Minimal reproduction

Helper level:

from agents.run_internal.items import deduplicate_input_items_preferring_latest

call = {"type": "function_call", "call_id": "c1", "name": "t", "arguments": "{}"}
output = {"type": "function_call_output", "call_id": "c1", "output": "result"}

deduplicate_input_items_preferring_latest([call, output, dict(call)])
# [{'type': 'function_call_output', ...}, {'type': 'function_call', ...}]

Session level — save_result_to_session with an input list carrying a tool call plus its output, and a run item that replays the same call, persists ["function_call_output", "function_call"].

Run level — a call_model_input_filter whose returned list repeats an earlier function_call sends the output before the call to the model, in both Runner.run and Runner.run_streamed.

No API key, network access, or paid model call is involved; the repro uses FakeModel and the existing in-repo session test doubles.

Current behavior

The surviving item is emitted at the index of its final duplicate, so the item list is reordered and a function_call can end up after its function_call_output.

Corrected behavior

Duplicates collapse onto the first occurrence of their dedupe key, so caller order is preserved, while the last occurrence still supplies the value. Latest-value preference is unchanged.

Root cause

Reverse → first-wins dedupe → reverse is only equivalent to "prefer the latest value" when position does not matter. It conflates which value survives with where it survives.

Implementation

deduplicate_input_items_preferring_latest now records the latest item per dedupe key, delegates ordering to the existing deduplicate_input_items (first-wins, order-preserving), and substitutes the recorded latest value for each surviving key.

Why this is minimal

• One function body; no signature, no public API, and no call-site change.
• deduplicate_input_items stays the single source of truth for dedupe keys and ordering; no parallel dedupe path is introduced.
• Items without a dedupe key (messages, plain values) are still passed through untouched, so lists with no duplicates are returned unchanged.

Regression tests

tests/test_run_internal_items.py

• test_deduplicate_input_items_preferring_latest_keeps_original_order — a repeated function_call stays ahead of its output.
• test_deduplicate_input_items_preferring_latest_uses_latest_value_at_first_position — latest value, earliest position.
• test_deduplicate_input_items_preferring_latest_leaves_unique_items_untouched — no-duplicate lists are returned identically.

tests/test_call_model_input_filter.py

• test_call_model_input_filter_keeps_duplicate_item_order_non_streamed
• test_call_model_input_filter_keeps_duplicate_item_order_streamed

tests/test_agent_runner.py

• test_save_result_to_session_keeps_tool_call_before_its_output — persisted session order.

All five order assertions fail on main at 9f4292e5 and pass with this change. The existing latest-value tests added in #2411 (test_call_model_input_filter_prefers_latest_duplicate_outputs_non_streamed / _streamed, test_save_result_to_session_prefers_latest_duplicate_function_outputs, test_prepare_input_with_session_prefers_latest_function_call_output) still pass unchanged.

Execution modes covered

Runner.run and Runner.run_streamed for the model-input path, plus a direct save_result_to_session test for the persistence path. Runner.run_sync shares the Runner.run code path and needs no separate case.

Cleanup and lifecycle considerations

Pure function change: no tasks, streams, sessions, traces, or transports are created or closed, no global or per-run state is introduced, and caller-owned lists are not mutated (a new list is returned, as before).

Validation

Run from the repository root on macOS (Darwin 24.6.0), Python 3.12.13, uv 0.11.24, branch based on upstream/main at 9f4292e5:

Command	Result
make format	847 files left unchanged, All checks passed!
make lint	All checks passed!
make typecheck	mypy: Success: no issues found in 835 source files; pyright: 0 errors, 0 warnings, 0 informations
make tests	parallel: 6227 passed, 3 skipped; serial: 45 passed, 4 skipped
bash .agents/skills/code-change-verification/scripts/run.sh	code-change-verification: all commands passed.
git diff --check	clean
uv run pytest tests/test_run_internal_items.py tests/test_call_model_input_filter.py tests/test_agent_runner.py -q	240 passed (repeated 3x, stable)

Pre-fix confirmation: reverting only src/agents/run_internal/items.py and rerunning that focused command gives 5 failed, 235 passed, failing exactly on the five new order assertions.

Not run: the Python 3.10 matrix (UV_PROJECT_ENVIRONMENT=.venv_310), make coverage, make build-docs, and the integration-test profiles. The change adds no new syntax or dependency and touches no docs, examples, or generated files.

Compatibility considerations

No public API, signature, field order, import path, or __all__ change. The latest-value contract from #2411 is preserved; only the position of a collapsed duplicate changes, and only when the input actually contains duplicates. Lists without duplicate identifiers are returned exactly as before.

Non-goals

• No change to deduplicate_input_items (first-wins) or to the dedupe-key rules in _dedupe_key.
• No change to drop_orphan_function_calls, normalize_input_items_for_api, or any session backend.
• No broader reordering or validation of model input beyond restoring the caller's order.

Test plan

See Validation above. make format, make lint, make typecheck, and make tests all pass; the five new tests are demonstrated red on main and green with the fix.

Issue number

N/A — filed directly as a small fix, consistent with recent maintenance PRs in this repository.

Checks

• I've added new tests, if relevant
• I've run .agents/skills/code-change-verification/scripts/run.sh
• I've confirmed all verification steps pass
• If using Codex, I've run /review before submitting this PR

[PranavMishra28]

heads up, I think this swapped the failure rather than removing it, and it is on main now.

_dedupe_key keys tool items as call_id:{item_type}:{call_id} (items.py:675), so a function_call and its function_call_output get distinct keys and anchor independently. anchoring each key at its own first occurrence fixes the case where the stale duplicate is the call, but when the stale duplicate is the output the output is the one anchored first:

[function_call_output(c1,"stale"), function_call(c1), function_call_output(c1,"fresh")]
-> ["function_call_output", "function_call"]        value correctly "fresh"

which is the ordering the Responses API rejects, just mirrored. ran that against bdc294f to be sure rather than reading it off the diff.

it looks reachable rather than theoretical. in session_persistence.py:282-287 dedupe runs last in the chain, and neither pass before it repairs order: normalize_input_items_for_api (items.py:291) only coerces dicts and strips metadata, and drop_orphan_function_calls (items.py:165) only removes calls missing outputs, never an output whose call is missing or misplaced. the same helper also runs on user supplied input at run_loop.py:1554 and :2012 right after call_model_input_filter, which is the path the tests added here already exercise, so an output first list is easy to construct there.

anchoring function_call_output at max(first_occurrence, index_of_matching_call) instead of unconditionally at first occurrence would cover both directions. happy to send that as a follow up with the inverse case added to test_deduplicate_input_items_preferring_latest_keeps_original_order if useful.

one smaller thing worth knowing since the new test asserts it as correct: for [old_output, message, new_output] the result is [new_output, message], so the tool result is presented to the model as having happened before a user message that actually preceded it.

[seratch]

Thanks for catching this, and thanks for offering to send a follow-up. I agree that the first-occurrence placement introduced by #4140 is too broad: the inverse call/output case remains possible, and [old_output, message, new_output] shows that it can also change conversation chronology.

I will take the follow-up on our side so we can revisit the complete ordering contract rather than add another local condition. The intended direction is to preserve latest-occurrence placement for outputs and other items, matching the released behavior, while keeping duplicate tool-call items at their earliest call position so they cannot move behind the matching output. We will derive the call types from the existing _TOOL_CALL_TO_OUTPUT_TYPE mapping and cover all three orderings in the model-input and session-persistence paths.

Thanks again for the detailed report.