Skip to content

Link agent diagnostic claims to navigable evidence #192

Description

@hellices

Goal

Make diagnostic answers traceable to the exact cluster evidence the agent used,
and make missing or contradictory evidence visible instead of hiding it behind
fluent text.

Current gap

The agent tool log shows calls and the UI can follow reads, but the final answer
is still ordinary model-generated markdown. A user must manually correlate
claims with earlier logs, events, and manifests. The product does not currently
distinguish clearly between:

  • a claim directly supported by fetched evidence;
  • a plausible hypothesis that needs another check;
  • an inference made from incomplete RBAC-visible data;
  • conflicting evidence from different timestamps or resources.

This limits trust precisely in the incident workflows where confidence matters.

Design

  • Assign stable per-turn evidence references to successful read results and
    deterministic findings.
  • Give references resource identity, timestamp/resourceVersion, source type,
    and a bounded display excerpt.
  • Require diagnostic claims to cite evidence references in the model protocol.
  • Render citations as navigable UI elements that open the relevant describe,
    event, log, or finding view.
  • Preserve explicit sections for evidence gaps, alternative hypotheses, and the
    next discriminating check.
  • Mark unsupported citations and uncited diagnostic claims visibly; do not
    silently remove or rewrite the model's text.

Acceptance criteria

  • Evidence references are generated by korvid, not invented by the provider.
  • A citation resolves only to evidence fetched in the current turn or retained
    with an explicit resource version/timestamp.
  • Selecting a citation opens the corresponding on-screen evidence where that
    source is still available.
  • Missing, stale, truncated, or RBAC-denied evidence is represented explicitly.
  • The UI distinguishes supported claim, hypothesis, contradiction, and unknown;
    it does not present a model-generated numeric confidence as objective fact.
  • Provider output with unknown references, duplicate references, or malformed
    citation syntax degrades visibly and safely.
  • Eval reports add citation precision/coverage and evidence-gap honesty without
    replacing existing diagnosis/evidence metrics.
  • Small-profile prompts and result caps remain bounded after citation metadata
    is added.

Related work

Out of scope

  • Proving causal correctness from citation presence alone.
  • Hiding an answer because it contains an uncited hypothesis.
  • Persisting sensitive raw evidence in conversation history indefinitely.

Metadata

Metadata

Assignees

Labels

area: agentEmbedded agent runtime, tools, prompts, and providersarea: uiTextual application, screens, widgets, and interaction designconcern: testingTest infrastructure, evaluation, and quality measurementenhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions