Skip to content

SkillEvaluator v0.3.0

Choose a tag to compare

@rng1995 rng1995 released this 17 Sep 21:44
· 414 commits to main since this release
Immutable release. Only release title and notes can be modified.
ac0a049

What's Changed

  • docs: link the NVIDIA Verified Skills docs from SkillEvaluator by @mosheabr in #46
  • Fix Tier 2 scans for contained compatibility alias by @chrisknvidia in #36
  • fix: replace OSS boundary line allowlist with stable anchors by @chrisknvidia in #40
  • ci: skip heavy jobs for docs-only pull requests by @chrisknvidia in #48
  • fix(tier3): isolate per-task inputs and evaluator fixtures by @chrisknvidia in #44
  • feat(validate): replay explicit tier gating onto main by @rng1995 in #49
  • Ignore short env values in secure Docker redaction by @Hashim1999164 in #51
  • fix(quality): avoid substring false positives by @chrisknvidia in #45
  • chore(release): prepare v0.2.0 by @rng1995 in #52
  • chore: centralize frontier OpenAI/Anthropic chat defaults by @chrisknvidia in #38
  • feat(reporting): roll out unified benchmark publication card by @rng1995 in #47
  • fix(tier3): propagate judge model overrides to Harbor verifiers by @chrisknvidia in #54
  • fix(reporting): use commit SHA for pull request links by @danielgaskins in #60
  • fix(tier3): align Anthropic endpoints and fail closed on judge errors by @chrisknvidia in #56
  • Fix quality scoring with CommonMark structural parsing by @chrisknvidia in #53
  • fix(tier3): handle negative controls in insights by @rookepoole in #58
  • fix(tier3): parse SKILL.md frontmatter as YAML by @genkey6 in #68
  • chore(release): prepare v0.2.1 by @rng1995 in #76
  • fix(tier3): score Harbor single-step null results by @chrisknvidia in #74
  • fix(tier3): fail fast on rejected credentials by @chrisknvidia in #75
  • feat(tier3): report pass@k uncertainty and paired outcomes by @AdemVessell in #63
  • fix(ci): scope Gitleaks to checked-out history by @rioyu123 in #106
  • fix(tier3): avoid transcript runtime false positives by @rioyu123 in #84
  • fix: scan Markdown headings for PII instead of skipping them by @mimran-khan in #96
  • fix(tier3): retry malformed structured judge responses by @chrisknvidia in #78
  • fix(tier3): detect a results directory the Docker daemon cannot see by @genkey6 in #80
  • fix: skip gitleaks test dirs, not substring names by @mimran-khan in #102
  • fix(hygiene): use CommonMark for dead-link checks by @rioyu123 in #109
  • fix: keep --llm-verify file reads inside the skill by @mimran-khan in #101
  • feat: add SARIF reporter for GitHub Code Scanning by @mimran-khan in #117
  • fix: do not let frontmatter MIT hide a GPL LICENSE file by @mimran-khan in #97
  • fix: treat tools/ as an executable directory by @mimran-khan in #103
  • fix: flag Windows PII usernames that start with s by @mimran-khan in #95
  • fix: evaluate full SPDX license expressions by @mimran-khan in #98
  • fix(tier3): decode native Codex exec calls by @Tomauskasz in #111
  • fix(hygiene): detect unpinned dependencies with environment markers by @rioyu123 in #107
  • fix: accept UTF-8 BOM in SKILL.md frontmatter by @mimran-khan in #99
  • Honor SkillSpector analysis completeness by @Tomauskasz in #112
  • feat(model-catalog): allow plain HTTP for named hosts by @Tomauskasz in #135
  • Fix empty process-control environment resets by @RyaliNvidia in #133
  • fix: map Harbor trial folders to eval case ids by @mimran-khan in #104
  • fix(cli): report malformed policy files cleanly by @rioyu123 in #129
  • feat(cli): write catalog-summary.json after catalog validate by @mimran-khan in #121
  • feat(cli): parallel catalog validation and fleet rollup by @mimran-khan in #123
  • fix(security): accept fully covered documentation scans by @deepujain in #143
  • fix(tier3): prevent false positives in network exfiltration checks by @kweinmeister in #141
  • fix(embedding): reject non-progressing chunk windows by @deepujain in #142
  • feat(reporting): record the evaluated source identity on benchmark cards by @MaxFreedomPollard in #131
  • chore(release): prepare v0.3.0 by @rng1995 in #144

New Contributors

Full Changelog: v0.1.0...v0.3.0