Skip to content

feat: add skills testing for Claude Code and OpenAI Codex#6

Merged
hidai25 merged 1 commit intomainfrom
feature/skills-testing
Dec 20, 2025
Merged

feat: add skills testing for Claude Code and OpenAI Codex#6
hidai25 merged 1 commit intomainfrom
feature/skills-testing

Conversation

@hidai25
Copy link
Owner

@hidai25 hidai25 commented Dec 20, 2025

Summary

  • Add evalview skill validate for structure validation
  • Add evalview skill test for behavior testing
  • Add evalview skill list to discover skills in directories
  • Support SKILL.md format (Claude Code, Codex CLI compatible)
  • Policy compliance checking (prompt injection, role hijacking detection)
  • Token size warnings (>5k tokens)
  • Best practices suggestions (examples, guidelines sections)
  • CI-friendly JSON output for all commands
  • Example skill and test suite included

New Commands

# Validate skill structure
evalview skill validate ./my-skill/SKILL.md
evalview skill validate ./skills/ -r

# Test skill behavior
evalview skill test tests/my-skill.yaml

# List skills
evalview skill list ~/.claude/skills/

Test plan

  • Validated against official Anthropic skills repo (17 skills pass)
  • CLI help displays correctly
  • JSON output works for CI integration

- Add `evalview skill validate` for structure validation
- Add `evalview skill test` for behavior testing
- Add `evalview skill list` to discover skills in directories
- Support SKILL.md format (Claude Code, Codex CLI compatible)
- Policy compliance checking (prompt injection, role hijacking detection)
- Token size warnings (>5k tokens)
- Best practices suggestions (examples, guidelines sections)
- CI-friendly JSON output for all commands
- Example skill and test suite included
@hidai25 hidai25 merged commit ecf2e8b into main Dec 20, 2025
6 of 7 checks passed
@hidai25 hidai25 deleted the feature/skills-testing branch December 20, 2025 21:17
muhammadrashid4587 pushed a commit to muhammadrashid4587/eval-view that referenced this pull request Mar 11, 2026
## Summary

Hash-based change detection for incremental reindexing.

**Stack: 6/9** — Depends on hidai25#27.

## Changes

- `tracker.py` with `ChangeTracker` class
- JSON state file: `{relative_path: sha256_hash}`
- Detects added, modified, deleted files
- Fault-tolerant: I/O errors logged and skipped

Closes hidai25#6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant