Feature request: Dismissible hallucination alert banners

[githb-ac]

This feature is needed by users running multi-agent Hermes swarms. A reference implementation exists in the Hermes Control Interface repo. When an agent's hallucination rate exceeds a set threshold, please surface a prominent, dismissible alert banner at the top of the UI. The banner should name the agent, show the hallucination rate, and suggest a replacement model where possible. This is a critical safety feature for autonomous agent pipelines.

[nesquena-hermes]

Thanks for the feature request! Dismissible alert banners for hallucination threshold breaches would pair well with issue #688 (hallucination rate tracking).

The UI side is straightforward — a dismissible banner with agent name, rate, and a suggested replacement action. The prerequisite is the same as #688: a reliable hallucination rate signal needs to exist at the gateway/agent level before the WebUI can react to it with alerts.

Once #688 has a concrete signal design, this banner feature would be a natural next step. Logging both together for roadmap consideration.

[nesquena-hermes]

Thanks — logged as enhancement. Dismissible alert banners that fire when an agent exceeds its hallucination rate threshold will surface reliability issues without being permanently intrusive. We will design these in coordination with the hallucination rate tracking feature (#688).

[nesquena-hermes]

Closing this core issue because it depends on #688's undefined hallucination metric and would turn that unvalidated percentage into a global operational warning.

A safe alert needs more than a threshold: evaluator/method version, sample count/window, confidence, agent/model/task provenance, a stable event ID, drill-down evidence, dismiss/snooze semantics, rate limits, and an unknown/retracted state. None exists in the current Agent/WebUI contract. A banner saying an agent is “hallucinating at 6.2%” would otherwise look authoritative and could drive incorrect operational decisions.

The conditional optional direction is already tracked:

• 📊 Insights & observability — as extensions, not core panels (ideas / tracking) hermes-webui/hermes-webui-extensions#50

That observability-extension umbrella includes quality/hallucination monitoring and alert banners. If a validated external evaluator/Agent plugin later emits the required evidence-rich event, an extension may show it inside its own panel. A global banner should require an additional explicit opt-in and strict rate limiting.

Per the maintainer decision, no duplicate wishlist is being created. #688's evaluator contract and this alert surface must not be built independently. New evidence can reopen or re-triage the concept when a real evaluator backend exists.

Thanks @githb-ac for the threshold/dismiss UX proposal. It remains preserved as conditional extension work without adding speculative warning chrome to core.