Skip to content

SkillEvaluator v0.7.2

Choose a tag to compare

@chrisknvidia chrisknvidia released this 04 Nov 01:45
Immutable release. Only release title and notes can be modified.
f878f99

SkillEvaluator 0.7.2

Add hosted reports to an existing GitHub skill-checks workflow while keeping its shared component, folders, thresholds, and triggers. A centrally maintained component such as @v3 can now update its pinned evaluator without requiring an edit in every consumer repository. Each report retains the signed evaluator identity and resolved workflow revisions.

pip install --upgrade skillevaluator
skillevaluator ci init

PyPI package · CLI onboarding guide · Ready-to-use CI templates

GitHub onboarding and reporting

  • Preserve an existing supported scan workflow while adding hosted HTML and one updated PR comment. Direct evaluator calls and shared-component calls are supported.
  • Verify the shared component's identity permission before changing repository settings. Existing Actions restrictions, unrelated Pages sites, and deployment approvals remain protected.
  • Reuse an administrator-configured private Pages site in eligible GitHub Enterprise Cloud organization repositories. Verify its privacy and expected URL before retaining reports and again before deployment. If private hosting is unavailable, checks and downloadable reports remain available.
  • Improve repeat setup, custom scan job names, Windows workflow writes, and CRLF approval hashes.

Real consumer tests covered direct and shared-component setup, changed-skill selection, hosted HTML, updated PR comments, failing and recovered scans, central tag updates, and historical report backfills. Private scanning passed. Private Pages deployment was not tested end to end: an eligible Enterprise sandbox was unavailable.

Other Python package changes since 0.7.1

  • Add reproducible local-catalog decoy selection for Tier 3 with tier3 select-decoys, portable manifests, and verified staged content (#263).
  • Accept lists in metadata.domain, with explicit limits of 16 entries and 128 characters per entry (#279).
  • Fix Tier 2 self-match handling and chunk budgets (#296, #270).
  • Fix Tier 3 grading, source-error handling, Codex credential-directory cleanup, and benchmark display behavior (#265, #309, #287, #290, #276, #269).

Downloads

This single product release contains the Python wheel, source distribution, CI bundle, artifact manifest, and checksums.

The CI bundle reuses the existing verified scanner workflow 46a8cf269257d1ec56a0285c508ac9ee48010930 and image ghcr.io/nvidia/skillevaluator-tier1@sha256:a3baec51f48145cdf9de05a5ab054d9576f62f1de0f5276b788410f5c17747d4. No scanner image was rebuilt: the additional Python-package changes above do not change that image. The report publisher revision is 8096c6966a498f35f6f7dc1a596d1cc4da85cf11.

Contributors

Thanks to @chrisknvidia, @rng1995, and @YMuskrat for their contributions to this release.

Full changelog