-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
Description
Context
Every EGOS agent should have an eval suite — a set of test cases that verify the agent produces correct findings. The SSOT Auditor is our most mature agent but has no eval suite yet.
What needs to be built
Create agents/evals/ssot_auditor.json with test cases:
{
"agent_id": "ssot_auditor",
"version": "1.0.0",
"cases": [
{
"name": "detects_duplicate_type",
"input": { "files": ["fixtures/dup-a.ts", "fixtures/dup-b.ts"] },
"expected": { "findings_min": 1, "severity": "warning" }
},
{
"name": "ignores_import_reexport",
"input": { "files": ["fixtures/import-only.ts"] },
"expected": { "findings_max": 0 }
}
]
}Also create the fixture .ts files in agents/evals/fixtures/.
How to test
bun agent:run ssot_auditor --eval(The eval runner doesn't exist yet — that's a separate issue. For now, just create the JSON + fixtures.)
Skills needed
- TypeScript basics
- Understanding of what "duplicate type definitions" means
- JSON
Difficulty
🟢 Beginner-friendly — no complex code, just test data
Reactions are currently unavailable