examples: add ModelBehaviorError handling patterns and self-healing agent - #3669
examples: add ModelBehaviorError handling patterns and self-healing agent#3669tahirkhan05 wants to merge 2 commits into
Conversation
…gent Adds two new examples to examples/agent_patterns/ that address the common question of what to do when the model calls a nonexistent tool or produces invalid output (ModelBehaviorError), referenced in issue 325. model_behavior_error_handling.py: Three practical patterns for handling ModelBehaviorError: 1. Return a safe fallback value for best-effort tasks. 2. Log and re-raise with enriched context for production pipelines. 3. Retry the same task with a more reliable fallback agent. self_healing_agent.py: A self-healing loop that catches ModelBehaviorError, appends the error as a correction to the conversation, and retries up to MAX_SELF_HEALS times. This gives the model a chance to fix its own mistake without restarting the full run from scratch. Also updates examples/agent_patterns/README.md to document both patterns.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7dcc5797c5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| # Build a fresh input that includes the correction. | ||
| # We restart from the original task plus the correction note so the | ||
| # model has full context without the broken tool call in its history. | ||
| messages = f"{task}\n\n[System note: {correction}]" |
There was a problem hiding this comment.
When ModelBehaviorError is raised after the run has already completed earlier model/tool turns, this resets the next attempt to only the original task plus the latest correction, discarding the history available on exc.run_data. In the long-running/expensive-task scenario this example advertises, copied code will rerun prior work and any tool side effects on every heal instead of continuing from the failed run with the completed turns preserved.
Useful? React with 👍 / 👎.
Previously the retry loop discarded all completed turns and restarted from the original task string, causing prior tool calls to be re-run on every heal attempt (including any tool side effects). Now the retry builds the next input from exc.run_data.new_items — the list of RunItems completed before the error — and appends only the correction message. This preserves the conversation history and avoids re-running prior work, which is the main point of the self-healing pattern for long-running tasks. Falls back to full restart if exc.run_data is None (should not occur in practice, but defensive).
|
Addressed the Codex's review feedback. The previous implementation discarded completed turns on each retry and restarted from the original task string, which would re-run prior tool calls (and their side effects) on every heal attempt. The updated implementation builds the next input from exc.run_data.new_items, the RunItem list of everything completed before the error, and appends only the correction message. This preserves the conversation history and avoids redundant work, which is the core value of the self-healing pattern for long-running tasks. Falls back to a full restart if exc.run_data is None (defensive, should not occur in practice). |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7d840a5be
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ) | ||
|
|
||
| if completed_items: | ||
| input = completed_items + [correction] |
There was a problem hiding this comment.
When a failure happens after at least one completed turn, new_items contains only the items produced by this run while RunErrorDetails.input holds the user input separately, so this retry starts the next run with assistant/tool items plus the correction but without the original task. In this version the fresh issue is that the attempted fix now preserves generated turns but drops the run input, and on later heals can also drop prior retry context; build the retry from exc.run_data.input plus the converted new_items before appending the correction.
Useful? React with 👍 / 👎.
|
Thanks for sharing the idea, but we don't plan to add these examples to this repo. Please feel free to host them in your own repo. |
2 participants
Adds two new examples to examples/agent_patterns/ that address the common
question of what to do when the model calls a nonexistent tool or produces
invalid output (ModelBehaviorError). Closes #325.
Changes
model_behavior_error_handling.py
Three practical patterns for handling ModelBehaviorError:
self_healing_agent.py
A self-healing loop that catches ModelBehaviorError, appends the error as a
correction to the conversation, and retries up to
MAX_SELF_HEALStimes —giving the model a chance to fix its own mistake without restarting the run
from scratch. Useful for long-running tasks where restarting is expensive.
examples/agent_patterns/README.md
Documents both new patterns alongside the existing ones.
Testing
Note: This PR is complementary to PRs #2957 and #3402, not a replacement. Those PRs propose adding retry logic inside the SDK core. This PR shows developers how to handle ModelBehaviorError in their own application code today, using patterns that work with the current SDK regardless of whether a built-in retry mechanism is added.