SkillEvaluator 0.1.0

SkillEvaluator 0.1.0 — Initial Public Release

SkillEvaluator is a multi-tier framework for evaluating AI agent skills:

• Tier 1 — Validation: deterministic quality gates (schema, PII, license, security scanners, and more)
• Tier 2 — Deduplication: semantic overlap and intra-skill redundancy checks
• Tier 3 — Live Evaluation: synthetic eval dataset generation and sandboxed agent runs via Harbor

Documentation: https://docs.nvidia.com/skills/skillevaluator/

Install

uv tool install --python 3.13 "skillevaluator[all] @ git+https://github.com/NVIDIA/SkillEvaluator.git@v0.1.0"
skillevaluator validate ./my-skill --checks schema,pii,license,quality,unicode,lint --no-dedup

The scoped check list above needs no API key. LLM-backed tiers and full security scanning require provider credentials and external tools — see the installation guide.

Highlights

• Full three-tier evaluation pipeline with a single CLI (skillevaluator)
• NVIDIA Build provider support with Nemotron defaults; also supports OpenAI, Anthropic, Bedrock, and OpenAI-compatible endpoints
• Tier 3 live evaluation via Harbor with OpenCode, Codex, and Claude Code paths
• Documentation site with 15 guides (installation, configuration, tiers, CI, CLI reference)
• Optional semantic-version validation in Tier 1 (--previous-version)
• DCO sign-off check in CI for contributor commits

Security

• Isolated NVIDIA Build credentials from vendor CLI and container processes
• Removed implicit host-side pytest execution from default Tier 1 code-integrity checks
• Hardened compatibility-bridge startup and local macOS Seatbelt policy

Full changelog

See CHANGELOG.md.