A transparent HTTP proxy that captures Agent ↔ LLM API traffic, auto-splits sessions, builds structured conversation views, grades with multi-dimensional scoring, diagnoses behavioral issues via rule engine, and probes agent configuration for root causes — all with a built-in web dashboard.

透明 HTTP 代理，捕获 Agent ↔ LLM 的 API 流量，自动切分 session、构建结构化视图、多维自动评分、规则诊断行为问题、探针审查配置根因 —— 内置 Web 评测面板。

                         ┌──────────────────────────────┐
                         │   AgentEval (127.0.0.1:57633) │
                         └──────────────┬───────────────┘
                                        │
Agent ── HTTP ──► Proxy ── forward ──► Upstream LLM API
                    │
                    ├─ Raw traffic → logs/{stem}.jsonl
                    │  原始流量记录
                    ├─ Session detection (message rollback / idle timeout)
                    │  实时检测 session 边界
                    ├─ SessionView → logs/{session}.view.json
                    │  结构化会话视图
                    ├─ Auto-grade (rules + LLM judge) → logs/{session}.grade.json
                    │  自动评分（规则 + LLM 评审）
                    ├─ Diagnose (10 rule-based checks + LLM summary) → logs/{session}.diagnose.json
                    │  行为诊断（10条规则 + LLM 自然语言总结）
                    ├─ Probe (LLM agent reviews agent source config) → logs/{session}.probe.json
                    │  探针审查（LLM agent 审查被评测 agent 的 prompt/skills/tools 配置）
                    └─ Web Dashboard → http://127.0.0.1:57633/dashboard/
                       Web 评测面板

# Upstream LLM API / 上游 LLM 地址
AGENTEVAL_UPSTREAM=https://api.edgefn.net

# Proxy port / 代理监听端口
AGENTEVAL_PORT=57633

# Log directory / 日志目录
AGENTEVAL_LOG_DIR=./logs

# Judge LLM for grading, diagnosis summary, and probing / 评测 LLM（评分+诊断总结+探针共用）
AGENTEVAL_JUDGE_API_BASE=https://api.deepseek.com
AGENTEVAL_JUDGE_MODEL=deepseek-chat
AGENTEVAL_JUDGE_API_KEY=sk-xxx

# Source project directory for probe / 被探针审查的 agent 项目路径
PROBE_SOURCE_PROJECT_DIR=/path/to/your/agent/project

cargo run
# listening http://127.0.0.1:57633 -> https://api.edgefn.net
# dashboard http://127.0.0.1:57633/dashboard/

Point your Agent's BASE_URL to the proxy.

将 Agent 的 BASE_URL 指向代理地址。

# Claude Code
export ANTHROPIC_BASE_URL=http://127.0.0.1:57633

# OpenAI SDK
export OPENAI_BASE_URL=http://127.0.0.1:57633/v1

# Generic
MODEL_BASE_URL=http://127.0.0.1:57633/v1 your-agent-command

Open http://127.0.0.1:57633/dashboard/ in your browser.

浏览器打开上述地址查看评测面板。

# Run diagnosis on a session / 对某个 session 运行诊断
cargo run -- diagnose <session_id> [--format terminal|json]

# Run probe on a session (requires prior diagnosis) / 对某个 session 运行探针
cargo run -- probe <session_id>

Detection logic: Normal conversations grow messages turn-by-turn. A new conversation "shrinks" back to just the system prompt + new question. When common_prefix_len <= 1, it's treated as a new session.

正常对话 messages 逐轮增长，新对话 messages 会"回缩"。当 common_prefix_len <= 1 时判定为新 session。

Four weighted dimensions → overall 0–1 score. Rule metrics + LLM judge. Falls back to rule-based estimates if LLM is unavailable.

四个加权维度 → 0-1 总分。规则统计 + LLM 评审。LLM 不可用时自动降级。

10 rule-based checks across 4 categories. Pure rule engine (no LLM). After rules run, an LLM generates a 2-3 sentence natural-language summary (best-effort, skipped if no API key).

10 条规则检查，覆盖 4 个类别。纯规则引擎（不依赖 LLM）。规则运行后，LLM 生成 2-3 句自然语言总结（best-effort，无 API key 时跳过）。

An LLM agent with 4 read-only file tools enters the agent's source project directory (PROBE_SOURCE_PROJECT_DIR), reviews configuration files (CLAUDE.md, skills, prompts, tools), and identifies root causes for each diagnose issue. All recommendations are written to the report — never auto-applied.

携带 4 个只读文件工具的 LLM agent，进入被评测 agent 的项目目录，审查配置文件（CLAUDE.md、skills、prompts、tools），找到每个 diagnose issue 的配置根因。所有改进建议写入 report — 绝不自动修改代码。

Safety / 安全机制:

• All paths sandboxed: .. rejected, canonicalize verification / 路径沙箱：拒绝 .. + 二次校验
• Loop detection: 3 identical calls → warning injected / 循环检测：3 次重复调用 → 注入警告
• Max 30 steps / 最多 30 步
• Read-only: no write/execute tools / 只读不写
• LLM timeout: 300s / LLM 超时 300 秒

{AGENTEVAL_LOG_DIR}/
├── {stem}.jsonl                 ← Raw recorded traffic / 原始流量
├── {stem}_{N}.view.json         ← Session structured view / 结构化视图
├── {stem}_{N}.grade.json        ← Grade report / 评分报告
├── {stem}_{N}.diagnose.json     ← Diagnosis report / 诊断报告
└── {stem}_{N}.probe.json        ← Probe report / 探针报告