Skip to content

literouter v0.3.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 26 Sep 15:11
· 56 commits to main since this release
v0.3.0
c036ba9

v0.3.0 — Relay Round-Robin Balancing

Highlights

  • Round-robin relay balancing: Set server.routing_policy to round_robin to rotate the first relay between requests while keeping each relay's model fallback chain together. Equal relays converge on an even traffic split as requests accumulate.
  • Health and affinity remain authoritative: Open circuit breakers are still skipped before healthy relays, and session affinity still moves the relay that already served a conversation to the front.
  • Safer model fallback: A relay's advertised models now stay grouped during routing, including the single-relay case, so model fallback order is not interleaved across relays.
  • Console and documentation: The built-in Web Console exposes the new policy in multiple languages, with configuration and routing documentation updated in both Chinese and English.

Upgrade Notes

  • The configuration schema is unchanged. Existing configurations continue to load with routing_policy: "priority" and the same behavior as v0.2.0.

  • To enable balanced routing, edit the server configuration:

    {
  "server": {
    "routing_policy": "round_robin"
  }
}

  • round_robin is equal-weight rotation. It is not weighted traffic splitting; provider weight remains a tie-breaker in priority ordering.

Installation

Download the archive for Linux x86_64, macOS arm64, or Windows x86_64 from the GitHub Release, verify its .sha256 file, and run literouter serve. Linux users can also install the latest binary with:

curl -fsSL https://raw.githubusercontent.com/YangQi0408/literouter/main/scripts/install.sh | bash

Then open http://127.0.0.1:8787/ui/ to use the built-in Web Console.

Assets 20
Loading