You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
v0.5.0
Adds typed capability, regression, and product evaluations; portable manifests; strict grading; promotion readiness; history, JUnit, and SARIF reporting; and the composite action.
Documented 0.x breaking change: default shell checks now allow only constrained jq/printf/wc -l pipelines. Migrate other checks to typed graders or use unsafe_shell: true / EVAL_ALLOW_UNSAFE_SHELL=1 only with trusted case content; this is not an OS sandbox.
Adds the Janus 0.1.10 offline benchmark. The matched file-output case did not invoke Janus's candidate executor, while checks: [] was accepted as PASS with zero checks; native adapter recommendation remains no-go until both behaviors are fixed.
Benchmark used deterministic stubs. No live model quality, cost, latency, or reliability measurements were made.