Skip to content

Add eval suite for SSOT Auditor agent #7

@enioxt

Description

@enioxt

Context

Every EGOS agent should have an eval suite — a set of test cases that verify the agent produces correct findings. The SSOT Auditor is our most mature agent but has no eval suite yet.

What needs to be built

Create agents/evals/ssot_auditor.json with test cases:

{
  "agent_id": "ssot_auditor",
  "version": "1.0.0",
  "cases": [
    {
      "name": "detects_duplicate_type",
      "input": { "files": ["fixtures/dup-a.ts", "fixtures/dup-b.ts"] },
      "expected": { "findings_min": 1, "severity": "warning" }
    },
    {
      "name": "ignores_import_reexport",
      "input": { "files": ["fixtures/import-only.ts"] },
      "expected": { "findings_max": 0 }
    }
  ]
}

Also create the fixture .ts files in agents/evals/fixtures/.

How to test

bun agent:run ssot_auditor --eval

(The eval runner doesn't exist yet — that's a separate issue. For now, just create the JSON + fixtures.)

Skills needed

  • TypeScript basics
  • Understanding of what "duplicate type definitions" means
  • JSON

Difficulty

🟢 Beginner-friendly — no complex code, just test data

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions