Skip to content

Releases: YasserManss/modeljury

v0.3.0

Choose a tag to compare

@YasserManss YasserManss released this 21 Oct 08:01

Breaking changes

  • OPENAI_API_KEY is only sent to the endpoint it belongs to: OPENAI_BASE_URL, or OpenAI when that is unset. A juror with any other base_url needs its own api_key=, and a 401 from such a juror now says so. Servers that don't check keys are unaffected.
  • A panel-wide max_tokens now applies to Claude jurors too; it used to leave them at 16000. Thinking counts toward it, so a small cap can leave no answer. Give the Claude juror its own params={"max_tokens": ...}, which still wins.
  • Options that differ only by case or surrounding spaces raise ValueError instead of merging silently.
  • A reply with no vote now fails as no vote in reply: ..., showing the start and end of long replies, instead of unparseable reply: ... or choice not in options: ''.

Fixes

  • A draft inside a <think> block is no longer counted as the vote, and a reply cut off mid-thought is no vote.
  • A vote followed by another JSON object is no longer lost, and a broken final vote wins over an earlier draft, with its fields salvaged.
  • Options with stray surrounding spaces now match.

Docs

  • The Timeouts section now explains what timeout bounds: calls don't stream, so a server that sends nothing until it is done times out after timeout seconds of work, while one that sends keep-alive bytes never does.
  • The benchmark claims are corrected: hate speech moved from the wins to the losses, "beats each of its own models" is qualified, and the headline panel is marked as not yet confirmed on a fresh sample.

v0.2.1

Choose a tag to compare

@YasserManss YasserManss released this 05 Oct 07:37
  • Salvage the juror's choice from replies with broken JSON (e.g. an unquoted reason), instead of counting the vote as failed.
  • The source distribution now contains only the library, tests and docs.

v0.2.0

Choose a tag to compare

@YasserManss YasserManss released this 04 Oct 12:27

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Choose a tag to compare

@YasserManss YasserManss released this 04 Oct 11:12