Repository navigation
SkillEvaluator v0.3.0
·
414 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
What's Changed
- docs: link the NVIDIA Verified Skills docs from SkillEvaluator by @mosheabr in #46
- Fix Tier 2 scans for contained compatibility alias by @chrisknvidia in #36
- fix: replace OSS boundary line allowlist with stable anchors by @chrisknvidia in #40
- ci: skip heavy jobs for docs-only pull requests by @chrisknvidia in #48
- fix(tier3): isolate per-task inputs and evaluator fixtures by @chrisknvidia in #44
- feat(validate): replay explicit tier gating onto main by @rng1995 in #49
- Ignore short env values in secure Docker redaction by @Hashim1999164 in #51
- fix(quality): avoid substring false positives by @chrisknvidia in #45
- chore(release): prepare v0.2.0 by @rng1995 in #52
- chore: centralize frontier OpenAI/Anthropic chat defaults by @chrisknvidia in #38
- feat(reporting): roll out unified benchmark publication card by @rng1995 in #47
- fix(tier3): propagate judge model overrides to Harbor verifiers by @chrisknvidia in #54
- fix(reporting): use commit SHA for pull request links by @danielgaskins in #60
- fix(tier3): align Anthropic endpoints and fail closed on judge errors by @chrisknvidia in #56
- Fix quality scoring with CommonMark structural parsing by @chrisknvidia in #53
- fix(tier3): handle negative controls in insights by @rookepoole in #58
- fix(tier3): parse SKILL.md frontmatter as YAML by @genkey6 in #68
- chore(release): prepare v0.2.1 by @rng1995 in #76
- fix(tier3): score Harbor single-step null results by @chrisknvidia in #74
- fix(tier3): fail fast on rejected credentials by @chrisknvidia in #75
- feat(tier3): report pass@k uncertainty and paired outcomes by @AdemVessell in #63
- fix(ci): scope Gitleaks to checked-out history by @rioyu123 in #106
- fix(tier3): avoid transcript runtime false positives by @rioyu123 in #84
- fix: scan Markdown headings for PII instead of skipping them by @mimran-khan in #96
- fix(tier3): retry malformed structured judge responses by @chrisknvidia in #78
- fix(tier3): detect a results directory the Docker daemon cannot see by @genkey6 in #80
- fix: skip gitleaks test dirs, not substring names by @mimran-khan in #102
- fix(hygiene): use CommonMark for dead-link checks by @rioyu123 in #109
- fix: keep --llm-verify file reads inside the skill by @mimran-khan in #101
- feat: add SARIF reporter for GitHub Code Scanning by @mimran-khan in #117
- fix: do not let frontmatter MIT hide a GPL LICENSE file by @mimran-khan in #97
- fix: treat tools/ as an executable directory by @mimran-khan in #103
- fix: flag Windows PII usernames that start with s by @mimran-khan in #95
- fix: evaluate full SPDX license expressions by @mimran-khan in #98
- fix(tier3): decode native Codex exec calls by @Tomauskasz in #111
- fix(hygiene): detect unpinned dependencies with environment markers by @rioyu123 in #107
- fix: accept UTF-8 BOM in SKILL.md frontmatter by @mimran-khan in #99
- Honor SkillSpector analysis completeness by @Tomauskasz in #112
- feat(model-catalog): allow plain HTTP for named hosts by @Tomauskasz in #135
- Fix empty process-control environment resets by @RyaliNvidia in #133
- fix: map Harbor trial folders to eval case ids by @mimran-khan in #104
- fix(cli): report malformed policy files cleanly by @rioyu123 in #129
- feat(cli): write catalog-summary.json after catalog validate by @mimran-khan in #121
- feat(cli): parallel catalog validation and fleet rollup by @mimran-khan in #123
- fix(security): accept fully covered documentation scans by @deepujain in #143
- fix(tier3): prevent false positives in network exfiltration checks by @kweinmeister in #141
- fix(embedding): reject non-progressing chunk windows by @deepujain in #142
- feat(reporting): record the evaluated source identity on benchmark cards by @MaxFreedomPollard in #131
- chore(release): prepare v0.3.0 by @rng1995 in #144
New Contributors
- @mosheabr made their first contribution in #46
- @Hashim1999164 made their first contribution in #51
- @danielgaskins made their first contribution in #60
- @rookepoole made their first contribution in #58
- @genkey6 made their first contribution in #68
- @AdemVessell made their first contribution in #63
- @rioyu123 made their first contribution in #106
- @mimran-khan made their first contribution in #96
- @Tomauskasz made their first contribution in #111
- @RyaliNvidia made their first contribution in #133
- @deepujain made their first contribution in #143
- @kweinmeister made their first contribution in #141
- @MaxFreedomPollard made their first contribution in #131
Full Changelog: v0.1.0...v0.3.0