Repository navigation
SkillEvaluator v0.7.0
SkillEvaluator 0.7.0
SkillEvaluator 0.7.0 adds Python 3.14 support alongside Python 3.12 and 3.13, reusable Tier 1 CI integration, configurable quality policies, and clearer evaluation reports.
Highlights
- Python 3.14 support, with full-suite CI and built-package installation checks across Python 3.12, 3.13, and 3.14.
- Reusable GitHub Actions and GitLab CI checks, parallel skill scanning, and configurable quality headings and thresholds.
- Verdict-led HTML reports, case-first Tier 3 summaries, matching-judge comparisons, and actionable recommendations in
what_to_fix.json. - More precise Tier 2 completeness reporting, Tier 3 timeout/recovery and local-mode fixes, and an optional
--max-caseslimit for exploratory runs. - SkillSpector 2.13.3 installation and CI pins, preserved NVIDIA Build model-registry handling, and validated LLM provenance in JSON/HTML reports.
- Refreshed locked runtime and dashboard dependencies. The Python package's dependency ranges are unchanged.
Python package
PyPI publication: published and verified. Both the wheel and source distribution are available on PyPI and NVIDIA’s Python package index. Their downloaded SHA-256 hashes match the release files below.
Versioned package page: skillevaluator 0.7.0 on PyPI.
Create a virtual environment using Python 3.12, 3.13, or 3.14 and install the package:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install skillevaluator==0.7.0For the optional Python evaluation and security dependencies:
python -m pip install 'skillevaluator[all]==0.7.0'A standalone uv installation is also supported:
uv tool install --python 3.14 'skillevaluator==0.7.0'The all extra includes the Tier 2, Tier 3, and Python security dependencies. Full static security scanning additionally requires separately installed SkillSpector, Gitleaks, and Semgrep executables. Upgrade existing SkillSpector installations explicitly to v2.13.3; Claude 5.5 requires a supported explicit SKILLSPECTOR_MODEL with this scanner release. Live evaluations require provider credentials and the selected agent/sandbox runtime. The optional dashboard is installed with the separate dashboard extra.
See the installation guide for scanner and runtime setup.
Release files
This release uses the GitHub tag 0.7.0 at commit ad2815fe78c045db91d230681d32863698da08cf. The earlier v0.7.0 tag name was consumed by an immutable release and cannot be reused. The Python package version is 0.7.0.
The Assets section contains the installable Python wheel (skillevaluator-0.7.0-py3-none-any.whl), Python source distribution (skillevaluator-0.7.0.tar.gz), and their SHA-256 checksums. GitHub also supplies repository source archives and a release attestation. The wheel supports all three Python versions listed above.
Installation verification
Fresh PyPI installations were verified with pip using both wheel and source distributions on Python 3.12, 3.13, and 3.14, plus uv tool installation and the all extra on Python 3.14. Checks included dependency consistency, CLI execution, and real offline validation reports in JSON, HTML, Markdown, and SARIF formats. New provider-backed LLM or Tier 3 evaluations were not run for this package publication.
Package checksums
| File | SHA-256 |
|---|---|
skillevaluator-0.7.0-py3-none-any.whl |
886a9713ab33f0e904d848f649977d02214c2412901b4eecd9c01fe51b79340e |
skillevaluator-0.7.0.tar.gz |
c8634c39ce7bbbf5138bf6a7a99c841637fc40a8407caf88ae2c49cdc90f0a31 |
What's Changed
- Fix Windows subprocess decoding, reasoning-model token budget, and harbor bin resolution by @Minervaowl7 in #194
- fix(cli): include nested skills in catalog validation by @YMuskrat in #210
- fix(quality): keep minimum score failures after policy changes by @YMuskrat in #211
- fix(tier3): keep skill lift consistent when rebuilding reports by @YMuskrat in #212
- feat(tier1): score tier 1 quality checks via a configurable YAML file by @westonplatter in #213
- feat(tier3): let installed packages add custom execution backends by @rmalani-nv in #215
- fix(tier3): score unrecovered failures and align recovery verdicts by @mavericksea-ai in #217
- docs: simplify first runs with keyless checks and NVIDIA Build by @chrisknvidia in #218
- fix(judge): treat an "unavailable" schema option as a schema rejection by @MaxFreedomPollard in #219
- Pass Windows program dirs and Python UTF-8 mode to Harbor by @GodfredArhinful in #220
- fix(tier3): retain scores after verified timeout recovery by @chrisknvidia in #222
- fix(reporting): retain completed-agent results in Tier 3 reports by @chrisknvidia in #223
- docs: refresh evaluation guidance and expose dashboard guide by @rng1995 in #224
- fix(quality): score non-mapping metadata instead of crashing by @MaxFreedomPollard in #226
- fix(tier2): bound embedding chunks and preserve source evidence by @chrisknvidia in #227
- fix(tier3): avoid destructive scoring for plain cache cleanup by @chrisknvidia in #228
- fix(tier3): count explicit root searches as wasted tool calls by @chrisknvidia in #229
- fix(tier3): flag deletions that escape temporary paths by @YMuskrat in #230
- fix(tier3): flag destructive rm targets supplied by xargs by @YMuskrat in #231
- fix(tier3): gate nested cleanup findings on symlink evidence by @YMuskrat in #232
- docs(cli): compare direct tier workflows with validate --tiers by @chrisknvidia in #237
- feat(ci): add reusable Tier 1 scans and GitHub onboarding by @chrisknvidia in #238
- fix(tier3): preserve local verifier uploads and shared step files by @chrisknvidia in #239
- Add opt-in Unicode character facts for Tier 3 judges by @mavericksea-ai in #242
- fix(pii): recognize reserved placeholder email domains by @rng1995 in #246
- test: wait for descendant exit after streamed exec cleanup by @rng1995 in #247
- Add reusable GitLab skill evaluation and repository onboarding by @chrisknvidia in #248
- Share deterministic grading and add bounded input readers by @chrisknvidia in #249
- Add private manifest discovery and configuration checks by @chrisknvidia in #250
- fix(reporting): preserve unavailable Tier 3 evaluator scores by @rng1995 in #251
- fix(tier3): preserve local Claude Code instruction variables by @rng1995 in #252
- fix(rubric): preserve source and complete linked guidance by @rng1995 in #254
- feat(tier3): case-first CLI summary and case-based report findings by @chrisknvidia in #255
- perf(ci): faster Tier 1 scans and a clearer HTML report by @rng1995 in #256
- Compose private inventory, audits and catalog checks by @rng1995 in #257
- Share trace evidence, applicable scores and security checks by @rng1995 in #258
- fix: address review findings from #249 and #250 by @rng1995 in #260
- ci: run main test jobs in parallel (-n auto) by @rng1995 in #261
- fix(tier3): write generated test.sh and grader starters with LF endings by @rng1995 in #266
- fix(tier3): disable Claude Code background tasks in eval trials by @rng1995 in #268
- fix(quality): report the 5,000-token skill budget as advisory by @rng1995 in #272
- fix(tier3): launch the Bubblewrap binary that local mode detected by @rng1995 in #278
- fix(tier2): report context-deduplication work limits as incomplete by @rng1995 in #280
- feat(tier3): add opt-in --max-cases limit for exploratory runs by @rng1995 in #283
- fix(reporting): stop labeling recorded Tier 3 baselines as missing by @rng1995 in #285
- fix(security): keep SkillSpector's nv_build model registry for Build scans by @rng1995 in #288
- chore(deps): pin SkillSpector to v2.13.3 by @rng1995 in #292
- fix(security): redact credentials in declared component paths by @rng1995 in #301
- feat: support Python 3.14 by @chrisknvidia in #311
- feat(tier1): keep SkillSpector LLM provenance in reports by @rng1995 in #298
- fix: Git output decoding, PowerShell hook conditions and shared verifier redaction (#260 follow-ups) by @rng1995 in #312
- Prepare and qualify the 0.7.0 release, including dependency lock refresh and PyPI packages, in #314