workflow analyze: text renderer hardcodes 'Agent target' as prompt agent, contradicting --format markdown

[placerda]

Summary

agentops workflow analyze prints a hardcoded Agent target value in its default text renderer. The value is a constant string, so it is printed regardless of what the analysis actually computed. Running the same command with --format markdown prints the correct, computed value.

This mislabels hosted-agent and azd-recipe projects as Foundry prompt agents, which is a different target kind with different CI requirements (prompt_agent_bootstrap, name:version addressing, cloud eval runner).

Root cause

agentops/services/workflow_analysis.py:

def _friendly_foundry_eval_text(check: str, text: str) -> str:
    if check == "Agent target":
        return "Foundry prompt agent (`name:version`)."   # <-- ignores `text`
    if check == "Evaluators":
        return _friendly_evaluator_list(text.split(", "))
    return _soften_text(text)

The text parameter is discarded in the Agent target branch. The call site does pass the real computed value:

def _render_text_foundry_eval_rows(rows: Sequence[Sequence[str]]) -> List[str]:
    ...
        _friendly_foundry_eval_text(str(check), str(explanation)),

and explanation originates from analysis.official_eval_reasons[0], which already holds the correct description.

Only the text renderer is affected. The markdown renderer emits explanation unmodified, which is why the two formats disagree.

Related: _soften_text does text.replace("foundry_prompt", "Foundry prompt agent"), the same prompt-agent-centric assumption applied more broadly.

Reproduction

Repo: an azd-managed Foundry app with a hosted agent. agentops.yaml:

version: 1
agent: https://<acct>.services.ai.azure.com/api/projects/<proj>/agents/helpdeskbot/versions/11
dataset: .agentops/data/helpdesk-smoke.jsonl
protocol: responses

The repo also contains an azd eval recipe at src/helpdeskbot/eval.yaml which declares the kind explicitly:

agent:
  name: helpdeskbot
  kind: hosted        # <-- hosted, not prompt
evaluators:
  - builtin.intent_resolution
  - builtin.task_adherence

Then:

$ agentops workflow analyze
Foundry eval
  ok   Agent target  Foundry prompt agent (`name:version`).      # WRONG

$ agentops workflow analyze --format markdown
| [x] | Agent target | Found azd eval recipe at `src\helpdeskbot\eval.yaml`. |   # CORRECT

Same command, same repo, same analysis. Only --format differs.

For completeness, agentops eval analyze on the same repo classifies the target correctly:

target    Foundry hosted agent

So the target kind is determined correctly upstream; only this renderer overrides it.

Impact

Cosmetic in the sense that no eval behaviour changes, but actively misleading for a first-time user following a hosted-agent tutorial:

1. The text renderer is the default. Most users never pass --format markdown, so the wrong label is the one they see.
2. workflow analyze is explicitly positioned as the planning step before workflow generate. Being told the target is a prompt agent points the user at the wrong CI shape.
3. The two output formats contradict each other, which undermines trust in the rest of the report.

Suggested fix

Return the computed text and keep the friendly rewrite only where it is actually a rewrite:

def _friendly_foundry_eval_text(check: str, text: str) -> str:
    if check == "Evaluators":
        return _friendly_evaluator_list(text.split(", "))
    return _soften_text(text)

A regression test asserting that workflow analyze text and markdown renderers agree on the Agent target row would prevent this class of drift.

Environment

• agentops-accelerator 0.8.2
• Python 3.12.9, Windows 11
• Target: Foundry hosted agent, protocol: responses

Related

• Docs/code drift: eval analyze reports model quality for hosted-agent targets that the docs classify as Conversational #363 — same family of issue in eval analyze (hardcoded/misapplied scenario label for hosted-agent targets). Different command and different code path, but the same underlying pattern of a display string that does not follow the computed target kind.

[placerda]

Fixed in #383 (merged to develop as 893152e).