Skip to content

v0.3.0

Choose a tag to compare

View all tags
@ShreyamMaity ShreyamMaity released this 03 Oct 13:35
· 1 commit to main since this release
v0.3.0
3a75e84

What's Changed

  • Improved model configuration

Highlights

  • Per-group routing policy: sticky, strict or failover, with sticky scope (conversation, day or until changed), TTL, failover trigger (errors only or errors and timeouts), retries on the same model, first-token and total timeouts, and cooldown
  • Sticky routing keeps a conversation on one model and only moves on a real failure, so answers stay consistent
  • Optional pin to an exact model per request
  • Responses report the model that answered, whether it was pinned, and any failover with its reason (headers and stream events)
  • Test traffic keeps its own cooldowns, so test runs never affect real traffic
  • /config page and API: edit policies, create and delete groups, list groups, view and clear pins
  • Longer default timeouts with stream keepalives for long tool-heavy answers

Full Changelog: v0.2.0...v0.3.0

What's Changed

Full Changelog: v0.2.0...v0.3.0

What's Changed

Full Changelog: v0.2.0...v0.3.0

What's Changed

Full Changelog: v0.2.0...v0.3.0

What's Changed

Full Changelog: v0.2.0...v0.3.0

Contributors

ShreyamMaity
Assets 6
Loading