fix(guardrails): distinguish execution failure from invalid verdict

[xu-jia-ming]

Note to maintainers: this PR was authored with AI assistance. Per .github/CONTRIBUTING.md it should carry the llm-generated label — I cannot apply labels myself, could a maintainer add it? (happy to retitle/annotate as preferred)

Summary

• LLMGuardrail.__call__ returned (False, "Error while validating the task output: ...") for any exception, the exact shape of a genuine guardrail violation
• A provider outage, expired key or rate limit was therefore reported as a verdict about the agent's output: callers retried the agent on it (guardrail_max_retries) and finally raised "guardrail failed validation" for a failure that never judged anything
• An LLM failure now raises GuardrailExecutionError (new, in crewai.utilities.guardrail_types), which propagates out of the retry loops without spending a retry or appending the error to the conversation
• process_guardrail emits the terminal LLMGuardrailCompletedEvent for it before re-raising, mirroring the existing HookAborted handling, so the started event is always closed
• A guardrail that runs and judges the output invalid still returns (False, feedback) — that path is unchanged

Problem

Before, an infrastructure error and a validation verdict were indistinguishable:

# provider outage during the guardrail's own LLM call
guardrail(out)  # (False, 'Error while validating the task output: litellm.APIConnectionError: provider unavailable')

# genuine violation
guardrail(out)  # (False, 'too long by 40 words')

Both feed GuardrailResult(success=False), so Task._process_output / Agent._process_kickoff_guardrail append the text to the conversation, spend retries re-running the agent, and after guardrail_max_retries raise a validation failure naming feedback that was really an outage.

Root cause

LLMGuardrail.__call__'s blanket except Exception mapped "couldn't check" onto "check says it's bad". Plain callable guardrails already propagate exceptions through process_guardrail (which only catches HookAborted); LLMGuardrail was the odd one swallowing them.

Changes

• utilities/guardrail_types.py: new GuardrailExecutionError documenting the contract ("the guardrail could not run; not a statement about the output")
• tasks/llm_guardrail.py: the blanket except now raises GuardrailExecutionError("The LLM guardrail could not run: ...") from e instead of returning a false verdict; HookAborted still re-raises untouched
• utilities/guardrail.py: process_guardrail catches GuardrailExecutionError, emits the terminal completed event (success=False, real error), re-raises; docstring updated
• tests/test_task_guardrails.py:
  • test_llm_guardrail_outage_raises_execution_error — outage raises GuardrailExecutionError with the cause chained
  • test_llm_guardrail_violation_still_returns_verdict — a real violation still returns (False, feedback)
  • test_process_guardrail_propagates_execution_error_with_terminal_event — re-raise plus terminal event
  • updated test_guardrail_when_an_error_occurs (which pinned the old conflation) to expect GuardrailExecutionError

Testing

• uv run pytest lib/crewai/tests/test_task_guardrails.py -q — 25 passed (3 new + updated test included)
• New tests fail at the merge base (cannot even import GuardrailExecutionError) and pass at the PR tip
• uv run pytest lib/crewai/tests/hooks/ — 41 passed (2 Windows-only teardown errors from TemporaryDirectory SQLite file locks, also present on clean checkout)
• uv run ruff check / ruff format --check / uv run mypy on the changed files — clean

Tests not run: the full uv run pytest lib/crewai/tests/ -x -q suite (left to CI per time); locally on Windows the repo's default --block-network addopt breaks asyncio's socketpair fallback, so async-event tests were run with -o addopts overriding it.

Related issue

Fixes #7150

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7447ade3-43d6-4abd-99b0-61c18527a753

📥 Commits

Reviewing files that changed from the base of the PR and between da4daad and 491955a.

📒 Files selected for processing (4)

• lib/crewai/src/crewai/tasks/llm_guardrail.py
• lib/crewai/src/crewai/utilities/guardrail.py
• lib/crewai/src/crewai/utilities/guardrail_types.py
• lib/crewai/tests/test_task_guardrails.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

LLM guardrail execution failures now raise GuardrailExecutionError instead of returning validation feedback. process_guardrail emits a failed completion event and re-raises the error. Tests cover provider failures, validation failures, exception chaining, and event emission.

Changes

Guardrail execution error handling

Layer / File(s)	Summary
Execution error contract and LLM guardrail behavior lib/crewai/src/crewai/utilities/guardrail_types.py, lib/crewai/src/crewai/tasks/llm_guardrail.py	Adds GuardrailExecutionError. LLMGuardrail.__call__ raises it for execution failures while preserving HookAborted handling.
Guardrail processing and completion event lib/crewai/src/crewai/utilities/guardrail.py	process_guardrail emits a failed LLMGuardrailCompletedEvent and re-raises GuardrailExecutionError.
Execution and validation regression coverage lib/crewai/tests/test_task_guardrails.py	Tests provider failures, chained causes, genuine validation feedback, exception propagation, and failed completion events.

Sequence Diagram(s)

sequenceDiagram
  participant LLMGuardrail
  participant process_guardrail
  participant GuardrailCaller
  participant LLMGuardrailCompletedEvent

  LLMGuardrail->>process_guardrail: raise GuardrailExecutionError
  process_guardrail->>LLMGuardrailCompletedEvent: emit failed completion event
  process_guardrail-->>GuardrailCaller: re-raise execution error

Loading

Suggested reviewers: lucasgomide

Merge Risk: ⚪ Minimal · up to 49195

This change separates guardrail execution failures from genuine invalid-output verdicts, preventing outages or provider errors from consuming validation retries or entering agent feedback. No actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 72.73% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 4 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: distinguishing guardrail execution failures from invalid verdicts.
Description check	✅ Passed	The description directly explains the guardrail error-handling change, its motivation, implementation, and test coverage.
Linked Issues check	✅ Passed	The changes satisfy issue [#7150] by raising GuardrailExecutionError for LLM/provider failures, preserving the original cause, re-raising through process_guardrail, emitting the terminal event, and pr…
Out of Scope Changes check	✅ Passed	The code, documentation, and tests are limited to separating guardrail execution failures from invalid-output verdicts and verifying the required behavior for issue [#7150].

Full details: Linked Issues check

Explanation

The changes satisfy issue [#7150] by raising GuardrailExecutionError for LLM/provider failures, preserving the original cause, re-raising through process_guardrail, emitting the terminal event, and preserving genuine validation verdicts. This prevents provider errors from being treated as validation failures or consuming guardrail retries.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Vidit-Ostwal]

Thanks for this — the shape is right (raise GuardrailExecutionError instead of returning (False, …)).

#7151 already does that fix and has stronger coverage (scoped event handlers, passing-output control, and a test that the task does not spend guardrail_max_retries). Please follow along there.

Worth folding into #7151 if they take it: putting GuardrailExecutionError on guardrail_types.py and the "The LLM guardrail could not run:" message prefix.

[Vidit-Ostwal]

Closing as a duplicate of #7151. Same fix; follow along there.