fix(run): fire on_llm_start / on_llm_end in Runner.run() for streaming & non-streaming (aligns with docs)

[erikanstine]

Summary

Fix a discrepancy between the documented lifecycle and the Python runner:

• on_llm_start and on_llm_end now emit from Runner.run() consistently in both streaming and non-streaming flows.
• Applies to _get_new_response and _run_single_turn_streamed.
• Behavior now matches the public API docs and other SDKs.

---

Behavior

• Start: on_llm_start fires immediately before an LLM request (sync/async/streaming).
• End: on_llm_end fires only on successful completion (after final chunk in streaming).
• Error: On exceptions, on_llm_end and on_agent_end are not emitted; errors propagate. This preserves current semantics.

Hook ordering (because run hooks and agent hooks dispatch concurrently):

• Start → RunHook, then AgentHook
• End → AgentHook, then RunHook

---

Compatibility

• Public API: No changes to method signatures or return types.
• Private API: _get_new_response signature updated to support unified flow. No other call sites/overrides in repo; considered safe as underscored/private.
• Behavioral: Bug fix, documented hooks now fire. Code that registered them will finally observe them.

---

Tests added

• Async run: verify on_agent_start, on_llm_start, on_llm_end, on_agent_end each fire once.
• Sync run: same assertions for Runner.run_sync.
• Streaming run: hook counts match expected values across a streamed call.
• Agent-level hooks: both run hooks and agent hooks emit as expected.
• Non-streaming error: patch FakeModel.get_response to raise; assert on_llm_start fires but on_llm_end / on_agent_end do not.
• Streaming error: BoomModel.stream_response yields once then raises; assert on_llm_start fires but on_llm_end / on_agent_end do not.

---

Changelog

Fix: Python runner now emits on_llm_start / on_llm_end as documented in both streaming and non-streaming paths. on_llm_end remains success-only; error paths propagate exceptions without firing end hooks.

Fixes #1612

[seratch]

Overall, this looks great. I will look into this early next week.

Can you make changes to the lifecycle example too?

diff --git a/examples/basic/lifecycle_example.py b/examples/basic/lifecycle_example.py
index f37380b..63ce3f8 100644
--- a/examples/basic/lifecycle_example.py
+++ b/examples/basic/lifecycle_example.py
@@ -1,10 +1,11 @@
 import asyncio
 import random
-from typing import Any
+from typing import Any, Optional
 
 from pydantic import BaseModel
 
 from agents import Agent, RunContextWrapper, RunHooks, Runner, Tool, Usage, function_tool
+from agents.items import ModelResponse, TResponseInputItem
 
 
 class ExampleHooks(RunHooks):
@@ -20,6 +21,22 @@ class ExampleHooks(RunHooks):
             f"### {self.event_counter}: Agent {agent.name} started. Usage: {self._usage_to_str(context.usage)}"
         )
 
+    async def on_llm_start(
+        self,
+        context: RunContextWrapper,
+        agent: Agent,
+        system_prompt: Optional[str],
+        input_items: list[TResponseInputItem],
+    ) -> None:
+        self.event_counter += 1
+        print(f"### {self.event_counter}: LLM started. Usage: {self._usage_to_str(context.usage)}")
+
+    async def on_llm_end(
+        self, context: RunContextWrapper, agent: Agent, response: ModelResponse
+    ) -> None:
+        self.event_counter += 1
+        print(f"### {self.event_counter}: LLM ended. Usage: {self._usage_to_str(context.usage)}")
+
     async def on_agent_end(self, context: RunContextWrapper, agent: Agent, output: Any) -> None:
         self.event_counter += 1
         print(

[seratch]

Please fix this one too:

uv run mypy .
tests/test_run_hooks.py:198: error: Unused "type: ignore" comment [unused-ignore]
Found 1 error in 1 file (checked 306 source files)
make: *** [Makefile:20: mypy] Error 1

[erikanstine]

Thanks for the 👀 @seratch - PR updated.

• ✅ Typing error fixed (uv run mypy .)
• ✅ Updated examples/basic/lifecycle_example.py

When updating lifecycle_example.py, I re-ran to get the updated expected output. To me it looks consistent with prior behavior (plus the new prints). One thing I noticed: the agent start and both LLM start/end hooks report Usage: 0 requests, 0 input tokens, 0 output tokens, 0 total tokens. Is that expected behavior for the Usage object? Not something I think this PR should change, but curious for your perspective.

[seratch]

can you move this right after L1422's usage update? It shouldn't bring any visible overhead in processing time and can provide better insights for the callback

[erikanstine]

Updated.

This fixes the issue I was seeing when running the lifecycle test. Thanks!