Repository navigation
Releases
Releases · orq-ai/evaluatorq
v1.100.0
What's Changed
🚀 Features
- feat(tracing): emit the OTel gen_ai.evaluation.result event on the evaluated job span (RES-1816) by @Baukebrenninkmeijer in #458
Full Changelog: v1.99.1...v1.100.0
v1.99.1
What's Changed
🐛 Bug Fixes
- fix: stop a run on a model the Anthropic client cannot serve (RES-1879) by @KarinaKKarinaK in #440
Full Changelog: v1.99.0...v1.99.1
v1.99.0
What's Changed
🚀 Features
- feat(integrations): add a Claude Agent SDK red-team target (RES-1876) by @KarinaKKarinaK in #443
Full Changelog: v1.98.0...v1.99.0
v1.98.0
What's Changed
🐛 Bug Fixes
- fix(simulation): name repetition counts and fields consistently and restore repetitions on replay (RES-1818) by @Baukebrenninkmeijer in #457
Full Changelog: v1.97.1...v1.98.0
v1.97.1
What's Changed
🐛 Bug Fixes
- fix(simulation): spot a placeholder persona or scenario list and ask again through a tool call (RES-1849) by @KarinaKKarinaK in #444
📚 Documentation
- docs: require dashboard updates when result semantics change by @Baukebrenninkmeijer in #455
🧰 Maintenance
- refactor(dashboard): check missing-union-attribute in dashboard source by @Baukebrenninkmeijer in #456
- ci: tighten ruff, ty and OpenCodeReview rules from past review findings (RES-1915) by @Baukebrenninkmeijer in #448
- build(deps): bump langchain from 1.3.9 to 1.4.10 by @dependabot[bot] in #450
- build(deps): bump loguru from 0.7.3 to 0.8.0 by @dependabot[bot] in #452
- build(deps-dev): bump pytest-asyncio from 1.3.0 to 1.5.0 by @dependabot[bot] in #453
Full Changelog: v1.97.0...v1.97.1
v1.97.0
What's Changed
🚀 Features
- feat(simulation): add repetitions to run each case as several fresh conversations (RES-1818) by @KarinaKKarinaK in #438
🧰 Maintenance
- build(deps-dev): bump ty from 0.0.90 to 0.0.100 by @dependabot[bot] in #423
Full Changelog: v1.96.6...v1.97.0
v1.96.6
What's Changed
🐛 Bug Fixes
- fix(redteam): stop writing the duplicated orq.redteam.jury.* span attributes (RES-1237) by @KarinaKKarinaK in #445
Full Changelog: v1.96.5...v1.96.6
v1.96.5
What's Changed
🐛 Bug Fixes
- fix(jury): reseat presets after weekly drift by @github-actions[bot] in #426
📚 Documentation
- docs: forbid local pytest-xdist runs in the agent instructions by @Baukebrenninkmeijer in #447
New Contributors
- @github-actions[bot] made their first contribution in #426
Full Changelog: v1.96.4...v1.96.5
v1.96.4
What's Changed
🐛 Bug Fixes
- fix: budget decision-model state at 3 chars per token (RES-1882) by @Baukebrenninkmeijer in #431
Full Changelog: v1.96.2...v1.96.4
v1.96.2
What's Changed
📚 Documentation
- docs: document the insights() coding_labels, run_name and runs_dir arguments (BACK-2425) by @KarinaKKarinaK in #441
🧰 Maintenance
- refactor: rename Jev to decision model across code, CLI and docs (RES-1900) by @KarinaKKarinaK in #442
- refactor(common): group catalogue rates into ModelInfo.pricing and bill incomplete tiers at standard rates (RES-1346) by @Baukebrenninkmeijer in #433
Full Changelog: v1.96.1...v1.96.2