Skip to content

Releases: openJiuwen-ai/agent-infer

Release list

AgentInfer 0.2.0

AgentInfer 0.2.0 Latest
Latest

Choose a tag to compare

View all tags
@openjiuwen-release-bot openjiuwen-release-bot released this 15 Oct 06:00
0.2.0
485edce

Highlights / 亮点

  • One-flag agent-aware serving: vllm serve MODEL --agentinfer now injects the agent-aware scheduler (async or sync), identity & lifecycle middleware, and the embedded Progress-TTL controller in a single step. Explicit long-form options remain supported.
  • Trace Replay benchmarks: AgentBench replay reconstructs sessions, actors, token targets, and dependencies from a captured requests.jsonl — no prompt plaintext stored — with cold-start reproducibility validation, now supporting TraceLab and AgentX traces plus bundled default trace datasets.
  • AgentRouter WASM affinity: new agent_hint_affinity WASM guest for the upstream Router --wasm-middleware host (native patch kept only for agent_hint_token_offsets).
  • Bilingual Diátaxis docs: mirrored 中文/English documentation with new deployment guides (vLLM-Router, vLLM-Semantic-Router, CI/CD).

Installation

Install from PyPI into the same Python environment as vLLM (Python 3.10+):

python -m pip install agentinfer

Then start agent-aware serving with one flag:

vllm serve MODEL --agentinfer

Added

  • vllm serve MODEL --agentinfer single-flag serving path; the lifecycle socket defaults when unset and conflicting --scheduler-cls options exit with code 2. Verified end-to-end on GPU (tests/agentcache/entrypoints/test_serve_flag_e2e.py); parser factory, scheduler schedule(), and prefix-lookup observer tolerate current and older vLLM layouts.
  • AgentBench: full benchmark + trace replay migration; TraceLab and AgentX replay support; default trace datasets; replay counts complete conversations with the configured chat template via a validated local tokenizer.
  • Calibration tools for vLLM prefill and decode profiles under tools/calibration/.
  • WASM agent_hint_affinity guest under agentinfer/agentrouter/agent_hint_affinity; optional native patch for agent_hint_token_offsets under agentinfer/agentrouter/patches.
  • Mirrored Chinese/English docs reorganized with the Diátaxis framework; vLLM-Router / vLLM-Semantic-Router / CI-CD deployment docs.
  • Migrated E2E use-case test suite.

Changed

  • Replay current-turn calibration preserves historical messages and requires exact input lengths; prefix clipping checks longest-to-shortest prefixes to handle nonmonotone token counts; nonzero calibration tolerance is now rejected instead of silently normalized.
  • Converted Trace IR manifests identify the converter by name without a separate version label.
  • Refined the Progress-TTL policy and stabilized force-resume deadlines.
  • Reorganized the README into overview, core features (with architecture diagram), requirements, installation, and Quick Start.

Fixed

  • DSH hot-module-reload error.
  • Benchmark runs now preserve token targets and validate complete runs; tokenizer discovery errors close the HTTP client before propagating.

Removed (breaking)

  • Removed the legacy AgentAwareScheduler/AgentAwareQueue compatibility surfaces, the import-time EngineArgs patch that defaulted scheduler_cls, and the agentinfer.LLM passthrough subclass. Enable agent-aware serving explicitly via vllm serve MODEL --agentinfer or the long-form scheduler/middleware options; other commands delegate to upstream vLLM unchanged.

Upgrade notes: if you relied on the implicit agent-aware scheduler default or agentinfer.LLM, switch to the --agentinfer flag or explicit --scheduler-cls/middleware options. Install the agentinfer-0.2.0 wheel into your vLLM environment.

中文说明

安装

在 vLLM 所在的 Python 环境（Python 3.10+）中从 PyPI 安装：

python -m pip install agentinfer

随后一键启动智能体感知服务：

vllm serve MODEL --agentinfer

主要变化

  • 单标志智能体感知服务：vllm serve MODEL --agentinfer 一键注入智能体感知调度器（同步/异步）、身份与生命周期中间件及内嵌 Progress-TTL 控制器；显式长选项仍可用，冲突的 --scheduler-cls 以退出码 2 报错。
  • 轨迹回放基准：从采集的 requests.jsonl 重建会话、执行者、token 目标与依赖关系（不存取原始 Prompt 明文），支持冷启动可复现校验；新增 TraceLab 与 AgentX 回放支持，并内置默认轨迹数据集。
  • AgentRouter WASM 亲和：新增 agent_hint_affinity WASM guest，对接上游 Router --wasm-middleware 宿主；原生补丁仅保留 agent_hint_token_offsets。
  • 中英双语文档：按 Diátaxis 框架重组，新增 vLLM-Router / vLLM-Semantic-Router / CI-CD 部署文档。

破坏性变更

  • 移除旧版 AgentAwareScheduler/AgentAwareQueue 兼容入口、默认 scheduler_cls 的导入期 EngineArgs 补丁以及 agentinfer.LLM 透传子类。请改用 --agentinfer 标志或显式调度器/中间件长选项启用；其余命令原样透传上游 vLLM。

完整的更新日志：0.1.0...0.2.0

Assets 5
Loading