Repository navigation
Releases: YasserManss/modeljury
Releases · YasserManss/modeljury
Release list
v0.3.0
Breaking changes
OPENAI_API_KEYis only sent to the endpoint it belongs to:OPENAI_BASE_URL, or OpenAI when that is unset. A juror with any otherbase_urlneeds its ownapi_key=, and a 401 from such a juror now says so. Servers that don't check keys are unaffected.- A panel-wide
max_tokensnow applies to Claude jurors too; it used to leave them at 16000. Thinking counts toward it, so a small cap can leave no answer. Give the Claude juror its ownparams={"max_tokens": ...}, which still wins. - Options that differ only by case or surrounding spaces raise
ValueErrorinstead of merging silently. - A reply with no vote now fails as
no vote in reply: ..., showing the start and end of long replies, instead ofunparseable reply: ...orchoice not in options: ''.
Fixes
- A draft inside a
<think>block is no longer counted as the vote, and a reply cut off mid-thought is no vote. - A vote followed by another JSON object is no longer lost, and a broken final vote wins over an earlier draft, with its fields salvaged.
- Options with stray surrounding spaces now match.
Docs
- The Timeouts section now explains what
timeoutbounds: calls don't stream, so a server that sends nothing until it is done times out aftertimeoutseconds of work, while one that sends keep-alive bytes never does. - The benchmark claims are corrected: hate speech moved from the wins to the losses, "beats each of its own models" is qualified, and the headline panel is marked as not yet confirmed on a fresh sample.
v0.2.1
- Salvage the juror's choice from replies with broken JSON (e.g. an unquoted reason), instead of counting the vote as failed.
- The source distribution now contains only the library, tests and docs.
v0.2.0
v0.1.0
Full Changelog: https://github.com/YasserManss/modeljury/commits/v0.1.0