Skip to content

TokenRouter 0.5.3

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Oct 23:17
· 88 commits to main since this release
v0.5.3
f428ca1

TokenRouter 0.5.3 fixes Gemini streaming compatibility, completion checks, and usage retention, and updates the project branding.

Upgrade notes

  • Gemini SSE responses now require valid JSON and a completion signal. Empty streams and EOF after unfinished content report failures; usage or heartbeats alone do not establish success. Check custom Gemini-compatible upstreams that previously relied on EOF alone. These rules cover Gemini REST, OAuth, and Vertex streams, their Messages, Chat Completions, and Responses conversions, and Antigravity's native Gemini streaming and buffered output.
  • gateway.max_line_size now bounds the complete frame on these Gemini SSE paths, including comments and metadata. This release adds no database migrations.

Improvements

  • Replace the default TokenRouter logo with a cyan routing mark in the web application and repository assets.
  • Refresh both READMEs with localized animated gateway banners, a static view for reduced-motion preferences, consistent badges, and quick navigation. Dark-mode screenshots are visible by default; light-mode screenshots are in a collapsed section.
  • Point official-site links in both READMEs and branding notices to https://tokenflux.com.

Bug fixes

  • Keep Gemini streams alive while waiting for upstream headers or the first response event by deferring response-header updates until delivery. Native generation and admission waits use empty SSE events that Google SDKs can ignore. Empty candidate placeholders no longer create SDK chunks or first-event timing observations; payloads with additional response metadata remain available.
  • Parse complete Gemini SSE events across LF, CRLF, and multiline data, including valid final JSON without a trailing blank line. Native output contains complete, compact JSON data frames, preserves unknown fields after envelope extraction, and consumes upstream comments, transport metadata, and [DONE] without forwarding them to SDK decoders.
  • Track completion across the requested candidate count so one finished candidate cannot end another candidate's stream. Explicit prompt blocking and [DONE] after actual candidate content remain valid completion signals. Stop heartbeats when the validated terminal is delivered, including large terminal frames beyond the metadata inspection prefix, and detect short writes instead of recording successful delivery.
  • Apply the upstream idle deadline consistently to these stream readers using gateway.stream_data_interval_timeout (180 seconds by default, 0 disables it). Actual upstream bytes, including heartbeats, renew the deadline; local heartbeats do not. After valid completion, collect trailing usage for at most five seconds. Tail transport loss or expiry preserves the completed result and observed usage and logs gemini.stream_tail_incomplete; an explicit upstream error still fails the request. Cancellation and read completion close the response body and wait for the reader to exit.
  • Surface Google errors embedded in HTTP 200 responses and reject malformed, oversized, empty, or unfinished streams without synthesizing successful closing events. Local read and protocol failures use status 502 with the existing provider policy. Heartbeat-only responses remain eligible for retries or provider switching; delivered protocol output closes that window. Final errors use JSON before HTTP commitment and the client's SSE format afterward, with one error presentation. Native SSE errors include a numeric code, sanitized message, and Google status even when a heartbeat already committed HTTP 200.
  • Retain observed usage and partial attempt results when a Gemini stream fails. Sparse usage tails update only present, non-null fields, preserving earlier input and cache counters while accepting explicit zero values. Existing settlement eligibility rules still apply.
  • Preserve separate candidates when collecting upstream SSE for a buffered response, including their content, finish reasons, and unknown fields. Adjacent text merges only when all other metadata matches, preserving distinct thought signatures, and repeated terminal candidates no longer duplicate completed content.

📥 Installation

Docker:

# GitHub Container Registry
docker pull ghcr.io/tokenflux/tokenrouter:0.5.3

One-line install (Linux):

curl -sSL https://raw.githubusercontent.com/TokenFlux/TokenRouter/main/deploy/install.sh | sudo bash

Manual download:
Download the appropriate archive for your platform from the assets below.

📚 Documentation

中文翻译

TokenRouter 0.5.3 修复 Gemini 流式兼容性、完成判定和用量保留，并更新项目品牌设计。

升级说明

  • Gemini SSE 响应现在必须包含有效 JSON 和完成信号。空流以及内容尚未完成时的 EOF 会报告失败；只有用量或心跳不能判定成功。请检查此前仅依赖 EOF 结束响应的自定义 Gemini 兼容上游。这些规则覆盖 Gemini REST、OAuth 和 Vertex 流、它们的 Messages、Chat Completions 和 Responses 转换，以及 Antigravity 的 Gemini 原生流式和缓冲输出。
  • 在这些 Gemini SSE 路径中，gateway.max_line_size 现在限制完整帧的大小，包括注释和元数据。本次发布不新增数据库迁移。

改进

  • 将 Web 应用和仓库资源中的默认 TokenRouter Logo 替换为青色路由图形。
  • 两种语言的 README 新增本地化网关动画横幅，并为偏好减少动态效果的用户显示静态画面；统一徽章样式并增加快捷导航。默认展示深色模式截图，浅色模式截图放入折叠区。
  • 两种语言的 README 和品牌声明中的官网链接改为 https://tokenflux.com。

问题修复

  • 将响应头更新推迟至实际交付时，使 Gemini 流在等待上游响应头或首个响应事件期间仍能保活。原生生成和准入排队使用 Google SDK 可忽略的空 SSE 事件。空候选占位帧不再生成 SDK 响应块，也不再被记为首个事件；带有其他响应元数据的报文仍会保留。
  • 按完整事件解析 Gemini SSE，支持 LF、CRLF、多行 data，以及缺少末尾空行但 JSON 有效的最后一帧。原生输出使用完整、紧凑的 JSON data 帧，解包后保留未知字段，并消费上游注释、传输元数据和 [DONE]，不将它们传给 SDK 解码器。
  • 按请求中的候选数量跟踪完成状态，避免一个候选结束时提前终止其他候选的流。明确的提示阻止，以及实际候选内容之后的 [DONE]，仍可作为有效完成信号。在交付经过验证的终态时停止心跳，包括终态信息位于元数据检查前缀之外的大帧；短写会被检测为失败，不再记录为成功交付。
  • 这些流读取器统一使用 gateway.stream_data_interval_timeout 限制上游空闲时间（默认 180 秒，设为 0 可关闭）。实际收到的上游字节会刷新截止时间，包括心跳；本地心跳不会刷新。有效完成后，继续收集用量尾帧，最多等待五秒。尾部传输中断或超时会保留已完成的结果和已观测用量，并记录 gemini.stream_tail_incomplete；明确的上游错误仍会使请求失败。取消或读取结束时关闭响应体，并等待读取协程退出。
  • 正确呈现 HTTP 200 响应中嵌入的 Google 错误，拒绝格式错误、超限、空白或未完成的流，不再生成表示成功的结束事件。本地读取和协议故障按状态码 502 进入既有提供商策略。只发送过心跳的响应仍可重试或切换提供商；交付协议输出后关闭该重试窗口。最终错误在 HTTP 提交前使用 JSON，提交后使用客户端对应的 SSE 格式，并且只呈现一次。即使心跳已经提交 HTTP 200，原生 SSE 错误仍包含数值 code、经过敏感信息清理的 message 和 Google status。
  • Gemini 流失败时保留已观测用量和部分尝试结果。稀疏用量尾帧只更新实际出现且非 null 的字段，保留此前的输入和缓存计数，并接受显式零值。结算资格继续遵循既有规则。
  • 为缓冲响应汇总上游 SSE 时，分别保留各候选的内容、结束原因和未知字段。只有其他全部元数据一致时才合并相邻文本，从而保留不同的 thought signature；重复出现的终态候选不再重复追加已完成的内容。

📥 安装

Docker：

# GitHub Container Registry
docker pull ghcr.io/tokenflux/tokenrouter:0.5.3

一键安装（Linux）：

curl -sSL https://raw.githubusercontent.com/TokenFlux/TokenRouter/main/deploy/install.sh | sudo bash

手动下载：
从下方附件下载适合所用平台的归档。

📚 文档

Assets 8
Loading