ageval v0.8.1

Evaluators can return optional checks (helper evaluation_check). Parent writes evaluation/checks.json when the list is well-formed; bind still reads only status / score / metrics. Verifier shows those rows in the file-tree preview. Hub / opens the Leaderboard plaza. Unchanged Attempt images on GHCR are retagged instead of rebuilt. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Optional evaluation checks: evaluator.py may return a checks list. SDK evaluation_check(...) builds a row (id required; exec_result fills exit / stdout / stderr). Parent persists ageval.evaluation.checks/1 as evaluation/checks.json. Omit checks and the file is not written. A FAIL row with a PASS return is legal observation. slim and Hub archives keep checks.json and still drop evaluator_raw.json.
• Verifier XOR: When both evaluation/observation.jsonl and evaluation/checks.json exist, a Trajectory / Files toggle sits on the tab row (trajectory first). Selecting checks.json renders counts, expandable rows, stdout/stderr, and the package script. There is no separate Checks panel. Agent Trajectory still reads only root trajectory.jsonl.
• Hub root plaza: / and unknown paths land on /leaderboard (not /datasets). Login with no return path follows the same default.
• Attempt GHCR publish: Release plans skip, retag, or rebuild. Same-tag re-entry is a no-op. Unchanged bake inputs copy the previous multi-arch list. Inspect errors fail closed and rebuild instead of guessing a retag.

Fix

• Trajectory open: Opening an Attempt keeps messages at the start. Tab focus no longer centers #main.
• File preview languages: Viewer tokenizes Python, YAML, and shell instead of forcing plain text.
• Light code wells: Light code-bg sits on canvas-soft so Trace and script wells stay recessed.
• Check script paths: Non-finite scores are dropped so checks.json stays valid JSON. View script rejects traversal / absolute / .ageval paths.
• Hub org Secrets: The organization Secrets empty state is gone (not a Registry feature).

Docs

• README leads with Leaderboard and trajectory; Verifier checks are in the extra stills. Scoring copy names script checks versus LLM-as-judge.
• Author pages and skills show evaluation_check return shape. Local Viewer / results pages describe the XOR toggle and in-tree preview.

Chore

• Example agent packs that duplicated the builtin catalog are gone. ACP stays --agent pi with no install; dsh / nooa / miniswe overlays remain.

中文

Evaluator 可以多返回可选 checks（helper evaluation_check）。列表合格时 parent 写入 evaluation/checks.json；bind 仍只读 status / score / metrics。Verifier 在文件树预览里渲染这些行。Hub 的 / 打开 Leaderboard plaza。Attempt 镜像 bake 输入没变时 GHCR 只 retag、不重建。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 可选评测 checks: evaluator.py 可返回 checks 列表。SDK evaluation_check(...) 组一行（必须有 id；exec_result 补 exit / stdout / stderr）。parent 把 ageval.evaluation.checks/1 写成 evaluation/checks.json。省略 checks 则不写文件。某一行 FAIL、顶层 PASS 是合法观察。slim 与 Hub 归档保留 checks.json，仍丢掉 evaluator_raw.json。
• Verifier XOR: 同时有 evaluation/observation.jsonl 与 evaluation/checks.json 时，页签行最右侧是 Trajectory / Files 切换（默认轨迹）。选中 checks.json 渲染计数、可展开行、stdout/stderr、dataset 脚本。没有单独的 Checks 面板。Agent Trajectory 仍只读根 trajectory.jsonl。
• Hub 根路径 plaza: / 与未知路径落到 /leaderboard，不再落到 /datasets。登录未带返回路径时同样默认。
• Attempt GHCR 发布: 发布计划为 skip / retag / rebuild。同 tag 再进是 no-op。bake 输入没变则复制上一份多架构 list。inspect 失败则失败关闭并 rebuild，不猜 retag。

修复

• 打开轨迹: 打开 Attempt 时消息留在开头。页签焦点不再把 #main 滚到中间。
• 文件预览语言: Viewer 对 Python / YAML / shell 做高亮，不再强制纯文本。
• 浅色代码井: 浅色 code-bg 落到 canvas-soft，Trace 与脚本井不再贴画布。
• check 脚本路径: 非有限 score 丢弃，保证 checks.json 仍是合法 JSON。View script 拒绝穿越 / 绝对路径 / .ageval。
• Hub 组织 Secrets: 去掉组织 Secrets 空状态（Registry 没有这个功能）。

文档

• README 以 Leaderboard 与轨迹打头；Verifier checks 放进附加截图。打分文案区分脚本 checks 与 LLM-as-judge。
• 作者页与 skills 给出 evaluation_check 返回形状。本地 Viewer / 结果页写明 XOR 切换与文件树预览。

杂项

• 与内置目录重复的 example agent pack 已删。ACP 仍是 --agent pi、不必 install；dsh / nooa / miniswe overlay 保留。

What's Changed

• feat(release): retag ageval-attempt when bake inputs are unchanged by @ffy6511 in #228
• feat(webui): share Hub/Viewer overlap through apps/shared by @ffy6511 in #230
• feat(runtime): structured checks on Verifier for deterministic evaluators by @ffy6511 in #231

Full Changelog: v0.8.0...v0.8.1

ageval v0.8.0

The public Hub is https://ageval.zjureal.com: the CLI uses that origin when AGEVAL_REGISTRY_URL is unset. Catalog opens on a Leaderboard plaza that indexes Dataset, Agent, and Model boards. Pareto plots cheaper on the left with a mute frontier; table cells can show lab and product marks. The docs site defaults to English. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Public Hub origin: Unset AGEVAL_REGISTRY_URL and credentials url resolve to https://ageval.zjureal.com. --registry-url still overrides; login does not require a URL first. The docs-site Hub link matches.
• Leaderboard plaza: Catalog leads with /leaderboard. It indexes listed Dataset suites and Agent / Model Performance — not a fourth ranking. The Dataset view groups by dataset and reuses Table / Pareto / Waffle per group; Agent / Model stay tables. Signed-out is OK; internal, draft-bound, and unlisted suites stay on the dataset tab.
• Pareto and board marks: Pareto’s x-axis increases to the right (cost, tokens, or time); undominated points join with a mute frontier, and switching the metric morphs the same suite points. Model cells prepend a lab mark after a unique pin join; Harness cells prepend the builtin product mark. In-app Harness / Model jumps stay ink until hover.
• Docs site English default: / redirects to /en/; Chinese stays at /zh-CN/. Unsuffixed MDX is English.

Fix

• Bind-tree symlinks: Docker harvest copies symlink nodes without following them, so workspace venv links that point at in-box interpreters no longer abort download or evaluate upload.
• Plaza reads: The Dataset view stays up when the derived Performance GET is missing. Official datasets are labeled as datasets, not plugins.

Docs

• Reader install / Hub / CLI pages name https://ageval.zjureal.com as the public origin. Dataset READMEs spell the default job (profiles.yaml) versus extra profiles*.yaml that still enter packageDigest.
• Hub docs cover plaza grouping, Catalog order, and left-cheap Pareto. The landing nav splits into chapter jump links; GitHub social preview is a 1280×640 English OG image.

中文

公开 Hub 是 https://ageval.zjureal.com：未设 AGEVAL_REGISTRY_URL 时 CLI 连这个源。Catalog 以 Leaderboard plaza 开场，索引 Dataset / Agent / Model 三块已有看板。Pareto 左侧更便宜，带淡色 frontier；表格单元格可加 lab / 产品标。文档站默认英文。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 公开 Hub 源: 未设 AGEVAL_REGISTRY_URL 与 credentials url 时解析到 https://ageval.zjureal.com。--registry-url 仍可覆盖；登录不必先填 URL。文档站 Hub 链接同步。
• Leaderboard plaza: Catalog 以 /leaderboard 打头。它是已上榜 Dataset suite 与 Agent / Model Performance 的索引，不是第四套排名。Dataset 视图按 dataset 分组，组内复用 Table / Pareto / Waffle；Agent / Model 仍是表。未登录可看；internal / draft / 未上榜 suite 只留在 dataset 页。
• Pareto 与看板标: Pareto 横轴向右变贵（cost / tokens / time）；未被支配的点用淡色 frontier 连起来，切度量时同一批 suite 点做 morph。Model 单元格在 pin 唯一命中后加 lab 标；Harness 单元格加内置产品标。站内 Harness / Model 跳转默认 ink，悬停再加深。
• 文档站默认英文: / 重定向到 /en/；中文仍在 /zh-CN/。无后缀 MDX 是英文。

修复

• 绑定树符号链接: Docker 收割只复制 symlink 节点、不跟随目标。指向盒内解释器的 workspace venv 链接不再让 download / evaluate upload 中止。
• Plaza 读取: 派生 Performance GET 缺失时 Dataset 视图仍可用。官方 dataset 标成 dataset，不再标成插件。

文档

• 读者向安装 / Hub / CLI 页写明公开源是 https://ageval.zjureal.com。Dataset README 区分默认 job（profiles.yaml）与仍会进入 packageDigest 的额外 profiles*.yaml。
• Hub 文档覆盖 plaza 分组、Catalog 顺序、左侧更便宜的 Pareto。落地页导航拆成章节跳转；GitHub 社交预览是 1280×640 英文 OG 图。

What's Changed

• feat(hub): Leaderboard plaza indexes Dataset / Agent / Model performance by @ffy6511 in #225

Full Changelog: v0.7.3...v0.8.0

ageval v0.7.3

Docker Attempt runs resolve the official base from GHCR at the CLI version, then fall back to the recipe packed in the wheel. Jobs can add extra LLM hostnames with egress_allow. Hub leaderboards gain waffle and Pareto views, suite runs open on their own page, and attach uses dashed fill-in slots. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Official Attempt image from GHCR: Missing local ageval-attempt:base pulls ghcr.io/zju-real/ageval-attempt:<cli-ver> and retags it locally. A miss builds once from the packaged attempt/ Dockerfile in the wheel — not from the checkout cwd. The historical l1 tag is gone; recipes and the default tag are ageval-attempt:base only. The tag workflow also pushes this image. Non-default CPython uses <ver>-py3.13; CI only publishes the default 3.12, so other minors miss GHCR and build locally.
• egress_allow extra LLM hosts: On docker, environment_options.egress_allow is a hostname list that must sit next to egress: llm on the same map. The box allowlist is bound base_url hosts ∪ these extras. Agent extras stay off the scoring list and scoring extras stay off the agent list; non-docker kinds and non-hostnames fail at lock.
• Hub leaderboard charts: Boards switch Table / Pareto / Waffle. Waffle paints per-trial pass/fail/error; Pareto plots pass rate against cost, tokens, or time. Suite summaries persist token and cost usage; Pareto prefers that stored estimate, and cost is the sum of every job's terminal usage (ACP usage is per-turn).
• Suite run detail page: A suite opens at /datasets/:id/suites/:suiteRunId instead of a modal. The jobs tab searches and shows Started / Duration from summary.json. ?suite= still redirects.
• Attach fill-in slots: Role, agent, and model are dashed underlines. Hover expands a quick-pick list; click still opens the search palette.
• Hub chrome: Official and builtin packages use a bundled REAL Lab mark. Sign out is an icon button. Disabled Share / Inbox / org actions explain what to fill on hover. Declared slots get an info hint. Field labels are sentence-case ink.

Fix

• Viewer navigation: SPA clicks no longer leave the URL ahead of the tree. Trajectory hides permission_decision rows when the policy is batch_auto_approve; sealed jsonl is unchanged.
• Share model picker: The model field is a searchable scrolling list (search plus rows, plus a typed custom name) instead of a clipped Select.
• Suite cost: Rolling usage into the suite summary sums every terminal turn. Re-upload refreshes metrics.usage from local run dirs.

Docs

• Reader pages keep glossary names, stay in one language, and resolve in-site links as relative MDX paths so the locale prefix is kept. Install copy states that [all] is e2b/daytona/nooa/dsh/miniswe and that Hub still uses uv sync --extra registry. egress_allow and the leaderboard views are documented next to the existing knobs.

中文

Docker Attempt 运行按 CLI 版本从 GHCR 解析官方基座，没有再用打进 wheel 的配方构建。job 可用 egress_allow 追加 LLM 主机。Hub 排行榜增加 waffle / Pareto，suite run 有独立详情页，attach 改为虚线填空槽。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 官方 Attempt 镜像走 GHCR: 本地没有 ageval-attempt:base 时，pull ghcr.io/zju-real/ageval-attempt:<cli-ver> 再标成本地 tag。再没有则用 wheel 里打包的 attempt/ Dockerfile 构建一次，不读 checkout cwd。历史 l1 tag 去掉，配方与缺省 tag 只认 ageval-attempt:base。打 tag 的 workflow 同时推这张镜像。非缺省 CPython 的 GHCR tag 是 <ver>-py3.13；CI 只推默认 3.12，其它 minor pull 不到就本地构建。
• egress_allow 额外 LLM 主机: docker 上 environment_options.egress_allow 是 hostname 列表，必须与同一 map 的 egress: llm 同写。放行名单 = 绑定 base_url 主机 ∪ 这些 extras。agent extras 不进打分名单，打分 extras 不进 agent 名单；非 docker kind 与非 hostname 在 lock 失败。
• Hub 排行榜图: 看板在 Table / Pareto / Waffle 间切换。Waffle 按 trial 画 pass/fail/error；Pareto 画通过率对 cost / tokens / time。suite 摘要落 token 与费用；Pareto 优先用上传时写入的估计，费用是每道 job 全部 terminal usage 之和（ACP usage 按 turn）。
• Suite run 详情页: suite 打开 /datasets/:id/suites/:suiteRunId，不再用 modal。Jobs 页可搜索，Started / Duration 来自 summary.json。?suite= 仍重定向。
• Attach 填空槽: Role / agent / model 是虚线下划线。悬停展开快选列表；点击仍打开搜索面板。
• Hub 壳层: 官方与 builtin 包用打包的 REAL Lab 标。Sign out 收成图标按钮。禁用的 Share / Inbox / 组织操作悬停说明缺什么。Declared slots 有说明。字段标签改为句首大写的 ink。

修复

• Viewer 导航: SPA 点击不再让 URL 跑到树前面。Trajectory 在策略为 batch_auto_approve 时隐藏 permission_decision 行；封存的 jsonl 不变。
• Share 模型选择: 模型字段改为可搜索的滚动列表（搜索 + 行，可键入自定义名），不再是被裁切的 Select。
• Suite 费用: 写入 suite 摘要时累加每一次 terminal turn。重新上传会从本地 run 目录刷新 metrics.usage。

文档

• 读者页跟 glossary 用词、单页单语，站内链接改相对 MDX 路径以保留 locale 前缀。安装说明写明 [all] 是 e2b/daytona/nooa/dsh/miniswe，Hub 仍用 uv sync --extra registry。egress_allow 与排行榜视图写在既有旋钮旁边。

What's Changed

• feat(runtime): extra hostname allowlist on egress: llm by @ffy6511 in #222
• refactor: two internal structure RFCs land (suite document + registry aggregate stores) by @ffy6511 in #223

Full Changelog: v0.7.2...v0.7.3

ageval v0.7.2

Docker ACP runs bake the bound coding-agent entry onto the task image, so invoke never npm i. Jobs can pick the official Attempt base CPython with environment_options.python_version, and the CLI talks to the public Hub without setting a registry URL first. Product docs ship as a static GitHub Pages site. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• ACP bound-entry bake: On docker, host.start() bakes lock options.entry onto the task image via ACP image_layers. after_environment_ready probes and skips install when the bake already matches. Switching model does not rebuild; switching entry does.
• Selectable Attempt CPython: environment_options.python_version (shape ^\d+\.\d+$; omit = 3.12) builds a versioned tag such as ageval-attempt:py3.13 and rewrites FROM ageval-attempt:base. Scoring hosts inherit python_version like platform / user. Empty / latest / 3 fail once; a base that cannot be pulled does not fall back to 3.12. Host AGEVAL_PIP_INDEX is passed as PIP_INDEX_URL on plugin-layer builds.
• Public Hub as CLI default: Unset AGEVAL_REGISTRY_URL and credentials url fall back to the public Hub. --registry-url still overrides; login no longer requires a URL first.
• Default scoring.exec host: Omit evaluation.environments and scoring.exec hits the run environment, or the isolated singular host. A named map still requires an exact name; unknown names fail once and do not start.
• Published example ids: minimal-demo ships as official/minimal-demo; the airline cut lives at examples/datasets/tau3-airline-5. The demo drops the extra e2b / miniswe profile files and binds dsh roles with *.
• Docs site on GitHub Pages: The website is a static export at https://zju-real.github.io/ageval/ (in-browser search). Hub Models search shows an OS-specific Cmd/Ctrl+F hint.

Fix

• Codex docker bake: Codex native RUN uses bash process substitution (BuildKit's default shell is dash). Probe initialize gets the same HOME / XDG as attach_stdio. FROM --platform=… ageval-attempt:base keeps dockerfile flags so it lands on the versioned base.
• dsh invoke: Each invoke gets a unique session id, so a second call no longer collides with a persisted log. The executor base_url defaults to the official DeepSeek API instead of a LiteLLM URL.
• Registry compose ports: Postgres and rustfs stay on the compose network; Hub 8080 is the only host-published port of the stack.
• Hub org table: Role and Actions columns keep real widths under table-fixed, so member action buttons are not clipped.

Docs

• README and the docs site lead with uv tool install ageval-cli and Hub refs; clone is the contributor path. Public copy uses glossary names (zh 参考答案, Core; no slot / fail-closed jargon). Reader pages state facts inline. The bilingual README Docs section points at the live site.

中文

Docker 上的 ACP 运行会把绑定的 coding-agent entry bake 进题图，invoke 时不再 npm i。job 可用 environment_options.python_version 选择官方 Attempt 基座的 CPython。CLI 默认连公开 Hub，不必先设 Registry URL。产品文档以静态站发布到 GitHub Pages。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• ACP 绑定 entry bake: docker 在 host.start() 把 lock 的 options.entry 经 ACP image_layers 写入题图。after_environment_ready 探测命中则跳过安装。换 model 不重建；换 entry 会。
• 可选 Attempt CPython: environment_options.python_version（形状 ^\d+\.\d+$；省略 = 3.12）构建版本化 tag（如 ageval-attempt:py3.13），并把 FROM ageval-attempt:base 解析过去。打分 Host 像 platform / user 一样继承 python_version。空串 / latest / 3 一次失败；上游 tag 拉不到不回退 3.12。宿主 AGEVAL_PIP_INDEX 作为 PIP_INDEX_URL 传进插件 bake 层。
• CLI 默认公开 Hub: 未设 AGEVAL_REGISTRY_URL 与 credentials url 时回落到公开 Hub。--registry-url 仍可覆盖；登录不必先填 URL。
• 缺省 scoring.exec Host: 省略 evaluation.environments 时，scoring.exec 打 run 环境或 isolated 那一只。有名表仍须点名；未知名一次失败，不 start。
• 公开示例 id: minimal-demo 发布为 official/minimal-demo；airline 缩略在 examples/datasets/tau3-airline-5。demo 去掉额外的 e2b / miniswe profiles，dsh 用 * 绑定角色。
• 文档站上 GitHub Pages: website 静态导出到 https://zju-real.github.io/ageval/（浏览器内搜索）。Hub Models 搜索按系统显示 Cmd/Ctrl+F。

修复

• Codex docker bake: Codex 原生 RUN 走 bash 进程替换（BuildKit 默认 dash）。probe initialize 使用与 attach_stdio 相同的 HOME / XDG。FROM --platform=… ageval-attempt:base 保留 dockerfile flag，落到版本化基座。
• dsh invoke: 每次 invoke 使用唯一 session id，第二次调用不再撞已持久化的 log。executor base_url 默认官方 DeepSeek API，不再误走 LiteLLM URL。
• Registry compose 端口: Postgres 与 rustfs 只留在 compose 网络；整栈对外只发布 Hub 8080。
• Hub 组织表: Role / Actions 在 table-fixed 下保有真实宽度，成员操作按钮不再被裁切。

文档

• README 与文档站以 uv tool install ageval-cli 和 Hub ref 开路；clone 是贡献者路径。公开文案跟 glossary（中文写参考答案、称 Core；不用 slot / fail-closed）。读者页把事实写在当页。双语 README 的文档入口指向线上文档站。

What's Changed

• feat(runtime): forward AGEVAL_PIP_INDEX to plugin bake layers by @ffy6511 in #215
• feat(docker): environment_options.python_version selects the Attempt base FROM by @ffy6511 in #219
• feat(acp): bake the bound ACP entry onto the task image by @ffy6511 in #221

Full Changelog: v0.7.1...v0.7.2

ageval v0.7.1

ageval is now installable from PyPI as a single ageval-cli wheel: the SDK merges into the same distribution, the Viewer SPA ships inside the wheel and sdist, and a tag-triggered Trusted Publishing workflow publishes it. Capability timeouts now judge as FAIL (score 0, metrics.reason=timeout) instead of ERROR, scoring hosts are bound from evaluate-phase graphs under their own network allowlist, and engine overlays are written from lock at environment prepare. The CLI shows the live attempt phase with elapsed time, and the Viewer gains a running pass rate plus placeholders for unrun tasks. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Single-wheel PyPI distribution ageval-cli: one wheel ships both ageval and ageval_sdk top-level packages — the sdk/ tree is deleted, and "ageval_sdk must not import ageval" becomes the architecture red line in place of package-graph isolation. ae registers as a shorthand entry beside ageval; the all meta-extra pulls nooa/dsh/miniswe/e2b/daytona, never registry; MIT license and project metadata included. Version metadata queries the new distribution name.
• Viewer SPA in the wheel: the release build bundles apps/viewer into src/ageval/viewer/static; the wheel ships it and the sdist force-includes it, so sdist→wheel rebuilds stay SPA-carrying. Building an sdist without the SPA fails on purpose.
• Publish pipeline: a tag-triggered v* workflow builds the SPA → wheel → publishes to PyPI via Trusted Publishing (OIDC). It refuses tags that disagree with VERSION before building anything, and a workflow_dispatch target publishes to TestPyPI for operator dry runs.
• Timeouts judge as FAIL: run/evaluate time budgets (task_run_timeout, wall time, exec/verifier timeouts) bind to FAIL score 0 with metrics.reason=timeout and timeout_phase. Environment start failures and other phase failures stay ERROR.
• Scoring box from evaluate-phase graphs: evaluate_host.environment_options declares the scoring host's own network/egress/platform/user allowlist; unknown keys and missing isolated fail closed at lock. Singular and named scoring hosts are constructed at evaluate phase after seal — image_layers union the evaluate-phase profile graphs so judge plugins bake in while solver-only plugins stay out, and scoring llm egress builds from evaluate-phase base_urls.
• Engine overlays from lock: environment prepare writes Pi/OpenCode/Codex/Claude HOME files from job_overlay; Claude ANTHROPIC_MODEL and Codex INITIAL_AGENT_MODE pin on attach. Host catalogs are not copied, and locator values stay out of overlay files.
• Live run progress: the single-task spinner gains elapsed time; the suite bar shows the live attempt phase ("task (phase) +N"), per-task elapsed, and the suite total. Non-TTY output and --json are unchanged.
• Viewer live suite: a live summary.json is rewritten under the progress lock from settled attempts — the running pass rate's denominator is settled tasks, and created_at locks on first write. The suite detail page joins the planned task axis and renders muted RUNNING/PENDING placeholder rows for unrun tasks. Uploads refuse running or cancelling snapshots.
• Trajectory rendering: step bodies render as markdown (user/agent/thought) or highlighted JSON (tool args, observations) with per-type icon tones; step headers stick below the invoke bar, fold scroll-safely, and show a 2.5-line fading preview when collapsed.
• Phase observer: an optional ctx.on_phase observer fires started/finished for lock, preflight, and each phase — observational only, and the default keeps behavior byte-for-byte.
• Examples layout: public datasets nest under examples/datasets (minimal-demo becomes dataset example/minimal-demo); the tau3-airline cut is trimmed to five tasks whose default profiles bind openai-http (GLM Coding Plan). README, website, tests, and smokes point at the new paths.

Fix

• Codex tool args: READ/list tool calls whose rawInput is omitted recover arguments from locations or title; execute keeps its title fallback.
• Codex docker bake: nested @openai/codex-linux-* native packages are installed and verified at bake — the npm -g copy vendored by codex-acp missed them.
• multiagent-env-min PostgreSQL: the example's attempt image installs PostgreSQL, setup starts it and loads the seed data, and db_query connects over TCP via the handoff meta host (the apt socket directory is not writable by the box user).

Docs

• Design and reader docs sync with the above: scoring-box network policy and evaluate-phase bind, FAIL/ERROR semantics in the glossary and the ARCHITECTURE outcome table, the SDK pinned to the single ageval-cli distribution, ageval_sdk paths under src/, and a sweep of repo links to ZJU-REAL/AgEval. The landing hero recenters on a write-once / run-anywhere harness rotate line (Claude Code, Codex, Pi, OpenCode, DSH, NOOA, mini-SWE-agent) with an N×M composition diagram on the bilingual READMEs; the Hub brand catalog adds mini-SWE-agent.

中文

ageval 现在以单个 ageval-cli wheel 发布到 PyPI：SDK 合并入同一发行版，Viewer SPA 打进 wheel 与 sdist，由 tag 触发的 Trusted Publishing workflow 发布。能力超时改判为 FAIL（score 0，metrics.reason=timeout）而非 ERROR，打分 Host 改为从 evaluate 相位图绑定并拥有自己的网络白名单，engine overlay 在环境准备期从 lock 写出。CLI 显示实时 attempt 相位与耗时，Viewer 新增进行中通过率与未跑任务的占位行。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 单 wheel PyPI 发行 ageval-cli: 一只 wheel 同时带 ageval 与 ageval_sdk 两个顶层包——sdk/ 树删除，「ageval_sdk 不得 import ageval」取代包图隔离成为架构红线。ageval 旁注册 ae 简写入口；all meta-extra 拉 nooa/dsh/miniswe/e2b/daytona，绝不含 registry；附 MIT license 与项目元数据。版本元数据改为查询新发行名。
• Viewer SPA 进 wheel: 发版构建把 apps/viewer 打进 src/ageval/viewer/static；wheel 携带、sdist force-include，sdist→wheel 重建仍带 SPA。没有 SPA 时构建 sdist 故意失败。
• 发布流水线: tag 触发的 v* workflow 按 SPA → wheel → Trusted Publishing（OIDC）发布到 PyPI。构建前拒绝与 VERSION 不一致的 tag；另设 workflow_dispatch 目标发 TestPyPI 供操作员演练。
• 超时判 FAIL: run/evaluate 的时间预算（task_run_timeout、墙钟、exec/verifier 超时）绑定为 FAIL score 0，带 metrics.reason=timeout 与 timeout_phase。环境启动失败与其他相位失败仍是 ERROR。
• 打分 Host 从 evaluate 相位图绑定: evaluate_host.environment_options 声明打分 Host 自己的 network/egress/platform/user 白名单；未知名与缺 isolated 在 lock fail-closed。单只与点名打分 Host 都在 seal 后的 evaluate 相位构造——image_layers 取 evaluate 相位 profile 图的并集，judge 插件烤进镜像、solver 专属插件排除在外，打分 llm egress 从 evaluate 相位 base_urls 构建。
• Engine overlay 从 lock 写出: 环境准备期按 job_overlay 写 Pi/OpenCode/Codex/Claude 的 HOME 文件；attach 时钉住 Claude ANTHROPIC_MODEL 与 Codex INITIAL_AGENT_MODE。不复制宿主目录，locator 值不进 overlay 文件。
• 实时运行进度: 单任务 spinner 显示耗时；suite 进度条显示实时 attempt 相位（「task (phase) +N」）、单任务耗时与套件总计。非 TTY 输出与 --json 不变。
• Viewer 实时套件: 在 progress 锁下重写实时 summary.json——进行中通过率分母是已结算任务，created_at 首写即锁定。套件详情页接入计划任务轴，为未跑任务渲染弱化的 RUNNING/PENDING 占位行。上传拒绝 running / cancelling 快照。
• 轨迹渲染: 步骤正文按 markdown（user/agent/thought）或高亮 JSON（工具参数、observation）渲染，配按类型着色的图标；步骤头吸底在 invoke 条下方、滚动安全折叠，收起时显示 2.5 行渐隐预览。
• 相位观察器: 可选的 ctx.on_phase 观察器对 lock、preflight 与五个相位发 started/finished——纯观测，默认行为逐字节不变。
• Examples 布局: 公开数据集收进 examples/datasets（minimal-demo 变为 dataset example/minimal-demo）；tau3-airline 剪成五题切片，默认 profiles 绑 openai-http（GLM Coding Plan）。README、website、测试与 smoke 指向新路径。

修复

• Codex 工具参数: READ/list 工具调用缺 rawInput 时从 locations 或 title 恢复参数；execute 保留 title 兜底。
• Codex docker bake: 嵌套的 @openai/codex-linux-* 原生包在 bake 期安装并校验——npm -g 复制进 codex-acp 的拷贝此前会漏掉它们。
• multiagent-env-min PostgreSQL: 示例的 attempt 镜像自带 PostgreSQL，setup 负责启动与种子数据装载，db_query 经 handoff meta 主机走 TCP 连接（apt 的 socket 目录对 box 用户不可写）。

文档

• 设计与读者文档随上述变更同步：打分盒网络策略与 evaluate 相位绑定、glossary 与 ARCHITECTURE 结果表的 FAIL/ERROR 语义、SDK 钉在单一 ageval-cli 发行、ageval_sdk 路径改到 src/、仓库链接清扫到 ZJU-REAL/AgEval。落地页 hero 改为 write-once / run-anywhere 的 harness 轮换主线（Claude Code、Codex、Pi、OpenCode、DSH、NOOA、mini-SWE-agent），双语 README 附 N×M 组合图；Hub 品牌目录新增 mini-SWE-agent。

What's Changed

• feat(release): PyPI-installable ageval-cli — single-wheel SDK merge, bundled viewer SPA, publish pipeline by @ffy6511 in #212
• feat(runtime): live suite summary and evaluate-phase scoring bind by @ffy6511 in #213

Full Changelog: v0.7.0...v0.7.1

ageval v0.7.0

Tasks can declare named sequential scoring hosts (evaluation.environments): the evaluator execs them by name and ACP attaches only after gold. A first-party anthropic-http executor joins the builtin mechanism cards. Hub grows a Models plaza pinned from models.dev, org-grouped dataset tables with owner-editable descriptions, and org icons. HTTP executors and miniswe accept options.extra_body. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Named scoring hosts: evaluation.environments in task.yaml maps sequential docker stages (dockerfile or docker_image); lock fails closed unless the job is isolated docker and every file exists. The evaluator starts a named host lazily on first use and execs it via inputs["scoring"].exec(<name>, argv); cleanup stops started hosts. ACP Agent.session(..., environment=<name>) rebinds attach — refused before gold is uploaded, required when the map exists; unknown names fail closed without starting. HTTP judges ignore environment=. Omitting the map keeps today's single isolated scoring host. scoring.exec env values may name a parent locator (resolved on the parent; missing ones are omitted; literal values are rejected so the worker cannot push secrets).
• anthropic-http executor: first-party Anthropic Messages executor (POST /messages with x-api-key; tool_use maps onto AgentResult.tool_calls). Builtin mechanism card --agent anthropic-http needs no install; Hub Explore lists the plugin and mechanism card.
• Hub Models plaza: sidebar Models renders per-lab hairline tables from a versioned models.dev snapshot (request paths never fetch). Modality tabs filter rows; Cmd/Ctrl+F opens a search palette; /models/{canonical} is the encyclopedia (Overview spec + Performance, PDF badge, HF mark only with a weights URL). Harness pages reuse ModelItem for a flat model directory. Overlay invoke ids unique-join canonicals longest-first; Performance attach and Inbox approval pick a canonical bucket, and approving without one fails closed.
• Dataset descriptions and org icons: datasets list as org-grouped tables; org owners edit the description in place or via ageval registry set-description (override wins over the manifest copy; clearing falls back; no republish). Orgs gain icon_key / icon_github with a Choose-icon modal; org heads show icons and official orgs sort first. Brand catalog adds 15 Lobe lab logos kept in lockstep with the registry allowlist.
• HTTP executor options: options.extra_body merges vendor-native keys into openai-http, anthropic-http, and miniswe (reserved keys rejected; extra_body wins on clash). miniswe forwards optional reasoning_effort to LiteLLM.
• Fog theme: light and dark canvas families retuned (light anchored #F1F3F5, dark #1B1E26) and synced across the token doc, Viewer, Hub, and website.

Fix

• Trajectory seal: trajectory.jsonl still seals after an evaluate ERROR, so Viewer keeps layer C; the first phase_failed stands and a later record failure degrades to record_warning.
• miniswe tool events: bash actions emit fold-compatible tool start/update events so Viewer shows tool_call / observation rows.
• Scoring robustness: scoring.exec clamps to remaining wall time (expired wall returns task_run_timeout without starting a host); evaluator stderr drains while serving frames.
• Registry persistence: Postgres data survives container recreation via a named volume.

Docs

• Reader pages cover named scoring hosts and scoring.exec (evaluator, environments, config fields), dataset descriptions, the org-grouped Hub list, and registry set-description. The Hub page adds the Models plaza. Plugin READMEs stay the parameter contract for extra_body / reasoning_effort.

中文

题包可声明多只、按序点名的打分 Host（evaluation.environments）：evaluator 按名 exec，ACP 只在 gold 之后 attach。第一方 anthropic-http executor 进内置机制卡。Hub 新增 Models plaza（models.dev pin）、按组织分组的 Dataset 表与可改描述、组织图标。HTTP executor 与 miniswe 支持 options.extra_body。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 点名打分 Host: task.yaml 的 evaluation.environments 是按序 docker stage 表（dockerfile 或 docker_image）；非 isolated docker 或文件缺失 lock fail-closed。evaluator 首次使用才 start，经 inputs["scoring"].exec(<name>, argv) exec；cleanup 停掉已启的 Host。ACP Agent.session(..., environment=<name>) 重绑 attach——gold 上传前拒绝，有表时必须点名；未知名 fail-closed 且不 start。HTTP judge 忽略 environment=。省略该表 = 今日单只 isolated 打分 Host。scoring.exec 的 env 可以写 in-box 名引用 parent locator（在 parent 侧解析，缺失的省略；字面量拒绝，worker 不能塞密钥）。
• anthropic-http executor: 第一方 Anthropic Messages executor（POST /messages + x-api-key；tool_use 映射到 AgentResult.tool_calls）。内置机制卡 --agent anthropic-http 不必 install；Hub Explore 列出插件与机制卡。
• Hub Models plaza: 侧栏 Models 用随版本 pin 的 models.dev snapshot 画每 lab 一张细线表（请求路径不外拉）。模态 tab 过滤；Cmd/Ctrl+F 开搜索面板；/models/{canonical} 是模型百科（Overview 规格 + Performance，PDF 徽章，有 weights 链接才显示 HF 标）。harness 页用 ModelItem 画扁平 Model 区。overlay invoke id 走 longest-unique join 到 canonical；Performance attach 与 Inbox 批准要挑 canonical 桶，缺桶批准 fail-closed。
• Dataset 描述与组织图标: Dataset 按组织分组成表；org owner 可在详情页就地改描述，或用 ageval registry set-description（override 优先于 manifest，清空回退，不重发包）。组织支持 icon_key / icon_github 与 Choose-icon 弹窗；org 组头显示图标，官方组织排最前。品牌目录新增 15 个 Lobe lab 标，与 registry allowlist 同步。
• HTTP executor 选项: options.extra_body 把厂商原生键合并进 openai-http、anthropic-http 与 miniswe（保留键拒绝；冲突时 extra_body 赢）。miniswe 把可选 reasoning_effort 透传给 LiteLLM。
• 雾色调色板: 亮 / 暗两套 canvas 色族重定基准（亮 #F1F3F5、暗 #1B1E26），token 文档、Viewer、Hub、website 同步。

修复

• 轨迹密封: evaluate ERROR 后 trajectory.jsonl 仍密封，Viewer 保留层 C；首个 phase_failed 保持，之后的 record 失败降级为 record_warning。
• miniswe 工具事件: bash action 发出 fold 兼容的 tool start/update 事件，Viewer 能看到 tool_call / observation 行。
• 打分健壮性: scoring.exec 按剩余墙钟封顶（超时返回 task_run_timeout 且不 start Host）；evaluator 边服务边排空 stderr。
• Registry 持久化: Postgres 改挂命名卷，容器重建数据不丢。

文档

• 读者向页面覆盖点名打分 Host 与 scoring.exec（evaluator、环境、config 字段）、Dataset 描述、按组织分组的 Hub 列表与 registry set-description；Hub 页新增 Models plaza。插件 README 仍是 extra_body / reasoning_effort 的参数合同。

Full Changelog: v0.6.3...v0.7.0

ageval v0.6.3

Shipped coding harnesses bind with --agent pi (no install). --model is a run override. Hub Agent pages collect Performance, and a Leaderboard row opens a suite inspector. evaluator.py runs as a parent worker like run.py; docker jobs may isolate a second scoring environment. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Shipped harness cards: Mechanism cards (pi, opencode, codex, claude-code, grok-build, openai-http) sit first on Hub Explore. CLI --agent pi projects the in-tree catalog. Custom overlay packs stay org/name@version after install. --model (with --agent) patches binding.model on the bound roles; omit it to keep the package default.
• Agent install plugins: ageval agent install then installs binding.extensions[].plugin (skips contrib). Missing plugin or unsatisfied host_requires fail-closes the whole install.
• Evaluate environment: evaluator.py is a parent subprocess on the same JSON-RPC mouth as run.py, not a second host.exec into the environment. Opt-in evaluate_host.isolated: true (Current: docker) starts a second scoring environment so gold never enters the Agent environment. LLM-as-judge rows go to evaluation/observation.jsonl (no user lines) and Viewer Verifier; they are not PASS.
• Hub Performance: Agent detail is Overview / Performance / Files. Mechanism cards default-collect official plaza (Maintainer Collect); custom cards still need consented agent_ref. Attach picks an overlay role. Remove unstamps that role only. Inbox filters by Kind / Dataset and can hide processed history for this user.
• Leaderboard inspector: Row click opens Profiles / Plugin / Jobs (Share for the owner). YAML is Shiki-highlighted with a hairline fence. A version Select filters suites; header ?v= still only switches README / Tasks. Score rings sit left of pass rate / mean; comparable tables sort from headers.
• ACP idle timeout: options.idle_timeout_seconds is a stall ceiling during session/prompt. Session updates or permission reset it. Unset / ≤0 keeps wall-clock invoke timeout only. Fires acp_idle_timeout, still capped by the Parent wall.
• Loopback HTTP keys: Empty api_key is allowed on host-only loopback URLs for HTTP executors (openai-http / dsh / nooa / miniswe). Remote and RFC1918 URLs still fail closed. --probe must not report credential_missing for loopback HTTP.

Fix

• ACP timeout events: Keep in-flight session/update rows and dump vendor acp_events.jsonl instead of wiping the turn.
• Plaza attach: Builtin short ids (pi, openai-http) stamp on delayed attach and plaza Leaderboard without Agent-org consent. Custom org/name still needs consent.
• Viewer / Hub: Attempt Model reads job_overlay. Truncated cells get tooltips again. Write failures toast instead of filling the dialog. Long README inline code wraps.

Docs

• Reader CLI / Hub / evaluator pages: --agent pi, --model, Performance, parent evaluator.py, isolated scoring environment, idle_timeout_seconds. Plugin ACP README is the parameter contract. Chinese docs call the isolation host 环境.

中文

内置 coding harness 用 --agent pi 绑定（不必 install）。--model 是这次 run 的覆盖。Hub Agent 页采集 Performance，Leaderboard 点行打开 suite inspector。evaluator.py 与 run.py 一样是 parent 子进程；docker job 可另起打分环境。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• 机制卡: Hub Explore 前排是 pi / opencode / codex / claude-code / grok-build / openai-http。CLI --agent pi 投影仓内目录。定制 overlays 包仍是装完再 --agent org/name@version。--model（须配合 --agent）改已绑角色的 binding.model；省略用包缺省。
• Agent install 插件: ageval agent install 接着装 binding.extensions[].plugin（跳过 contrib）。缺插件或 host_requires 不满足则整条 fail-closed。
• 打分环境: evaluator.py 在 parent 子进程跑，不再 host.exec 进环境。job evaluate_host.isolated: true（Current：docker）另起打分环境，gold 不进 Agent 环境。LLM-as-judge 写入 evaluation/observation.jsonl（省略 user），Viewer Verifier 可看；那不是 PASS。
• Hub Performance: Agent 详情是 Overview / Performance / Files。机制卡默认采集官方 plaza（Maintainer 可改 Collect）；定制卡仍要同意的 agent_ref。Attach 先选 overlay role。Remove 只剥该 role。Inbox 按 Kind / Dataset 过滤，History 可对本用户隐藏已处理行。
• Leaderboard inspector: 点行打开 Profiles / Plugin / Jobs（owner 另有 Share）。YAML 按语言高亮并带 hairline 框。版本选择器过滤 suite；页头 ?v= 仍只切 README / Tasks。Pass rate / mean 左侧是分数环；可比表点表头排序。
• ACP 静默超时: options.idle_timeout_seconds 是 session/prompt 的静默上限。session/update 或 permission 会重置。省略或 ≤0 只用墙钟 invoke timeout。触发 acp_idle_timeout，仍受 Parent 墙钟封顶。
• Loopback HTTP 空钥: 仅 host-only loopback 允许 HTTP executor（openai-http / dsh / nooa / miniswe）空 api_key。远端与 RFC1918 仍 fail-closed。loopback HTTP 的 --probe 不得报 credential_missing。

修复

• ACP 超时事件: 保留进行中的 session/update，并 dump 厂商 acp_events.jsonl，不再整轮抹掉。
• Plaza 挂载: 机制卡短 id（pi、openai-http）延后 attach 与 plaza Leaderboard 不经 Agent org 同意。定制 org/name 仍要同意。
• Viewer / Hub: Attempt 的 Model 读 job_overlay。截断单元格恢复 tooltip。写入失败改 toast，不再灌进对话框。README 长行内 code 会换行。

文档

• 读者向 CLI / Hub / Evaluator：--agent pi、--model、Performance、parent evaluator.py、isolated 打分环境、idle_timeout_seconds。插件 ACP README 仍是参数合同。中文文档把隔离宿主叫做环境。

What's Changed

• feat(agents): treat agent packages as harnesses; add --model by @ffy6511 in #196
• feat(hub): open Leaderboard Harness and Model on the Agent package by @ffy6511 in #197
• feat: loopback HTTP empty keys and agent-install plugins by @ffy6511 in #201
• feat(eval): optional evaluator SDK invoke records judge observation by @ffy6511 in #203
• feat(runtime): isolated evaluate host, tree artifacts, and judge transport by @ffy6511 in #205

Full Changelog: v0.6.2...v0.6.3

ageval v0.6.2

Hub adds an Inbox for listing and appearance, marketplace stars and package marks, first-party contrib as builtin catalog cards, and GHCR images for Hub and Registry. On a TTY the CLI prints a short recap instead of dumping JSON; run / view / upload-suite accept id@version (checkout under --dir). After seal, the durable evidence tree is slim; trajectory rows carry first-class usage and profile_id. Docker can run as a chosen user, including root. Evidence grade stays runnable-mvp. Default CI does not claim live Agent / e2b / ssh / Daytona runs.

Feat

• Hub Inbox and attach: Dataset and Agent org owners batch-approve listing and appearance. Share attaches a published agent_ref (direct write if you own the agent org, otherwise Inbox). Public Leaderboard still lists only complete, release-bound, owner-listed suites.
• Hub catalog: Plugin and agent cards show stars and download counts, uploader avatars / owner icon picker, and vendor color marks. Explore is the default list. First-party contrib (local / docker / e2b / ssh / daytona / acp / openai-http) appears as builtin overlay cards without a blob. Release tags push ghcr.io/zju-real/ageval-hub and ageval-registry.
• TTY recap: Commands print a short recap on a TTY (no digest / sha256). Pipes stay JSON. ageval run --json forces the full document.
• Registry refs: ageval run official/demo@0.1.0 --dir tmp installs at tmp/official/demo/ and reuses a matching child. ageval view and results upload-suite take a path or id@version; a unique verified cache hit does not contact Hub. tasks remains a local directory.
• Sealed evidence: Layer C terminal.usage holds first-class token/cost fields; leftover vendor keys sit in sibling extra. summary_enrich writes Attempt summary.extra. Every sealed row stamps profile_id. After a successful seal the durable tree drops vendor raw unless --keep-vendor-raw. Viewer / Hub show usage chips; missing quantities stay omitted.
• Runtime knobs: environment_options.user sets docker run / exec / attach_stdio (default 10001:10001; root / 0 / 0:0 map to root; no-new-privileges stays). openai-http honors options.reasoning_effort. Overlay locators expand from the host .env chain; base_url locators stay in the lock like api_key.

Fix

• ACP install: Skip invoke-time npm i when the entry is already on PATH. Floor prepare_home at 120s. Raise Attempt image buildx timeout to one hour.
• Hub / Registry: Restore multi-file overlay sources; hide private job stats; keep executor chips when environment is also set; show unlisted complete suites so listing can be requested; require dataset_version on suite upload; empty GitHub OAuth allowlist allows any login.
• Trajectory read: Fold overflow-only rows and keep multi-role order as invoke order. Slim trees without metadata.json still export. Viewer backfills profile_id by turn_index on old jsonl.

Docs

• Landing hero uses a pixel owl, a clone/uv sync snippet, and Hub links (docs chrome too). Plugin READMEs are the public contract: capability and parameter tables. Hub docs record Inbox, stars, builtin cards, and delayed agent_ref attach.

中文

Hub 增加上榜 / 出场 Inbox、市场 star 与包标记，并把 first-party contrib 画成 builtin 目录卡；发版镜像进 GHCR。TTY 下 CLI 打短 recap，不再倾倒 JSON；run / view / upload-suite 接受 id@version（--dir 检出）。密封后默认可持久树变瘦；轨迹行带一等 usage 与 profile_id。docker 可指定用户，包括 root。证据等级仍是 runnable-mvp。默认 CI 不声称真 Agent / e2b / ssh / Daytona 运行。

新功能

• Hub Inbox 与挂载: Dataset / Agent org owner 批量批准上榜与出场。Share 挂 published agent_ref（自己是该 Agent org owner 则直写，否则进 Inbox）。Public Leaderboard 仍只列完备、绑定 release、且 owner 已批准上榜的 suite。
• Hub 目录: 插件 / Agent 卡片展示 star 与下载量、上传者头像 / owner 图标、厂商色标。列表默认 Explore。first-party contrib（local / docker / e2b / ssh / daytona / acp / openai-http）以 builtin overlay 出卡，没有 blob。发版 tag 推 ghcr.io/zju-real/ageval-hub 与 ageval-registry。
• TTY recap: TTY 打短 recap（不含 digest / sha256）；管道仍是 JSON。ageval run --json 强制完整文档。
• Registry ref: ageval run official/demo@0.1.0 --dir tmp 装到 tmp/official/demo/，子目录已匹配则复用。ageval view 与 results upload-suite 接受路径或 id@version；唯一已校验缓存命中不打 Hub。tasks 仍只收本地目录。
• 密封证据: 层 C terminal.usage 是一等 token/cost；厂商剩余进兄弟 extra。summary_enrich 写 Attempt summary.extra。密封行带 profile_id。seal 成功后默认可持久树丢掉 vendor raw，除非 --keep-vendor-raw。Viewer / Hub 画 usage chips；没报的量省略，不编 0。
• 运行时旋钮: environment_options.user 同时用于 docker run / exec / attach_stdio（默认 10001:10001；root / 0 / 0:0 为 root；仍 no-new-privileges）。openai-http 尊重 options.reasoning_effort。overlay locator 从宿主 .env 链展开；base_url locator 与 api_key 一样进 lock。

修复

• ACP 安装: 入口已在 PATH 时不再 invoke 时 npm i。prepare_home 下限 120s。Attempt 镜像 buildx 超时提到一小时。
• Hub / Registry: 恢复多文件 overlay 源；隐藏私有 job 统计；environment 已设时仍保留 executor chips；未上榜的完备 suite 可被申请上榜；suite 上传必须带 dataset_version；GitHub OAuth 白名单为空则允许任一登录。
• 轨迹读取: 只折叠溢出行，多角色按 invoke 序。没有 metadata.json 的瘦树仍能导出。Viewer 对旧 jsonl 按 turn_index 回填 profile_id。

文档

• 落地页 hero 改为像素猫头鹰、clone / uv sync 片段，以及 Hub 链接（文档站 chrome 同样）。插件 README 是公开合同：必须用表格写 capabilities 与参数。Hub 文档记录 Inbox、star、builtin 卡与延迟 agent_ref 挂载。

What's Changed

• feat(hub): uploader GitHub avatars and a color catalog picker by @ffy6511 in #178
• feat(hub): marketplace favorites, owner-first members, prune ignored e2e by @ffy6511 in #176
• feat(hub): builtin contrib catalog and GHCR deploy images by @ffy6511 in #184
• feat(hub): delayed agent_ref attach and listing Inbox by @ffy6511 in #183
• feat(hub): job settings overflow, Explore-first lists, grayscale timing by @ffy6511 in #185
• feat(runtime): first-class trajectory usage, Hub chips, slim evidence by @ffy6511 in #190
• feat(runtime): summary.extra, summary_enrich, and dataset_id@version labels by @ffy6511 in #192

Full Changelog: v0.6.1...v0.6.2

ageval v0.6.1

Environments add Daytona on the same Protocol. Coding agents can run in-box through acp-oneshot when attach_stdio is missing. openai-http carries native tools= / tool_calls; tau2-class harnesses execute them. Hub shows plugin and agent catalog cards, bios, and download counts. Evidence grade stays runnable-mvp. Default CI does not claim live e2b / ssh / Daytona Agent runs.

Feat

• Daytona kind: environment: daytona uses the same Protocol as local / docker / e2b / ssh. Snapshot from a Dockerfile or a concrete OCI tag; attach_stdio is frozen after a live ACP initialize. Missing DAYTONA_API_KEY fails --probe.
• In-box ACP oneshot: executor: acp-oneshot runs one host.exec per invoke; ACP frames stay in the box. Lock requires exec, not attach_stdio. The default coding inlet remains parent ACP plus attach_stdio.
• Native tool calls: openai-http posts tools= and returns structured tool_calls. session.invoke accepts optional tools / messages; ACP ignores the catalog. tau2-class harnesses run ToolSet / Environment and record_observation. Parent does not execute package tools and does not spend quota on them.
• ACP install probe: After the environment is ready, probe the entry name, pin the package, and stdio-initialize. A same-named TCP binary counts as missing; install_command re-runs (with sudo fallback) when the probe misses.
• Hub catalog and identity: Plugin and agent marketplace uses cards with description and download_count. Users and orgs can set a bio; the GitHub link is derived from login. download_count increments on a successful package content GET, not per-version and not PASS.

Fix

• Daytona sandbox: Keep the snapshot PATH (do not copy the host PATH); inject AGEVAL_* on exec; map timeout; delete sandboxes; mkdir /attempt without sudo first.
• Artifact harvest: Download publishable files from /attempt/workspace/<basename>. terminal-jsonl-agg publishes from disk so truncated chat cannot occupy dest.
• ACP oneshot result: The wrapper writes the envelope to a box file when exec stdout is empty.
• Trajectory fold: Fold thought parts and skip empty assistant rows after tools. Viewer uses Brain for thought and SquareTerminal for invoke-end / bash.

Docs

• README and landing treat Harness as a first-class eval axis, locked with the Model and the environment. Language switcher and badges sit on one centered row. Design, website, and skills list Daytona. write-run notes tools / tool_calls. Hub docs record catalog cards, bios, and download counts.

中文

环境槽增加 Daytona，仍走同一套 Protocol。没有 attach_stdio 时，coding agent 可走盒内 acp-oneshot。openai-http 走原生 tools= / tool_calls；tau2-class harness 在题包内执行这些调用。Hub 的插件 / Agent 市场改为卡片，并展示 bio 与下载量。证据等级仍是 runnable-mvp。默认 CI 不声称 e2b / ssh / Daytona 的真 Agent 运行。

新功能

• Daytona kind: environment: daytona 与 local / docker / e2b / ssh 同一 Protocol。从 Dockerfile 或带具体 tag 的 OCI 编 snapshot；attach_stdio 在一次真实 ACP initialize 后冻结。缺 DAYTONA_API_KEY 时 --probe 失败。
• 盒内 ACP oneshot: executor: acp-oneshot 每次 invoke 一次 host.exec，ACP 帧留在盒内。lock 只要 exec，不要 attach_stdio。默认 coding 入口仍是 parent ACP + attach_stdio。
• 原生 tool call: openai-http POST tools=，返回结构化 tool_calls。session.invoke 可传 tools / messages；ACP 忽略工具目录。tau2-class harness 在题包内跑 ToolSet / Environment，并用 record_observation 挂回包。parent 不执行题包工具，也不为此扣配额。
• ACP 安装探测: 环境就绪后探入口名、钉死包版本、stdio initialize。同名 TCP 二进制视为缺失；探测失败则重跑 install_command（可 sudo）。
• Hub 卡片与身份: 插件 / Agent 市场改为带说明和 download_count 的卡片。用户 / 组织可写 bio；GitHub 链由 login 推导。download_count 在成功拉取包 content 时加一，不是 per-version，也不是 PASS。

修复

• Daytona sandbox: 保留 snapshot PATH（不把宿主 PATH 拷进盒）；exec 注入 AGEVAL_*；映射 timeout；删除 sandbox；先不以 sudo 建 /attempt。
• 产物回收: 从 /attempt/workspace/<basename> 下载可发布文件。terminal-jsonl-agg 只从磁盘 publish，截断的聊天不能占 dest。
• ACP oneshot 结果: exec stdout 为空时，从盒内结果文件读 envelope。
• 轨迹折叠: 折叠 thought；工具轮后不再画空 assistant 行。Viewer 用 Brain 标 thought，用 SquareTerminal 标 invoke-end / bash。

文档

• README 与 landing 把 Harness 作为一等评测维度，与 Model、运行环境一并锁定。语言切换与徽章同一行居中。设计、website、skills 列出 Daytona。write-run 说明 tools / tool_calls。Hub 文档记录卡片、bio 与下载量。

What's Changed

• feat(hub): profile bios, GitHub links, and marketplace download counts by @ffy6511 in #172
• feat(openai-http): native tools= on invoke for tau2-class harnesses by @ffy6511 in #175
• feat(runtime): acp-oneshot executor and Daytona environment by @ffy6511 in #173

Full Changelog: v0.6.0...v0.6.1

ageval v0.6.0

This release is the ageval outer runtime: lock a dataset, open one environment, run run.py, and bind PASS only from an independent evaluator. The CLI, packages, and formats are ageval. Coding agents enter through ACP; other mechanisms install as plugins and bind from job profiles. Environments are the exclusive slot local / docker / e2b / ssh. Agent packages (ageval.agent/1) bind a job with --agent. There is no bora shim. Evidence grade stays runnable-mvp. Default CI does not claim live e2b / ssh Agent runs.

Breaking

• Product rename: CLI ageval, import ageval / ageval_sdk, home ~/.ageval, env AGEVAL_*. Formats are ageval.dataset/1, ageval.task/1, ageval.plugin/1, ageval.profiles/1, ageval.agent/1. Unknown format is one invalid_format error. No bora command, no BORA_* alias, no dual-read. GitHub is ZJU-REAL/ageval.
• Attempt pipeline: The L0/L1 lifecycle is gone. Phases live in src/ageval/attempt/: environment → run → evaluate → record, cleanup in finally. Task run.py runs in a child process bound to the parent Agent Service. Gold (evaluation/) enters the environment at evaluate. Trajectory / RunTerminal.completed is never PASS.
• Environment and plugins: The environment is environment: on profiles.yaml, not provider.kind. ACP takes attach_stdio from the environment; parent is the only JSON-RPC client. dsh / nooa / miniswe invoke through host.exec. --agent and --profiles are mutually exclusive. Named examples are examples/journeys, examples/tau3-airline, and examples/agents; examples/core and examples/l1 are gone.

Feat

• Environment kinds: docker / e2b / ssh sit on the same Protocol as local. Plugin image layers bake from the plugin package. Directory upload replaces the dest tree. SSH A still does not support live ACP stdio.
• Agent Hub: ageval.agent/1 publish / install / cache. ageval agent plus --agent on lock / run / campaign. Listed overlay files ship with the Agent package. Official public suite appearances fold into Agent detail.
• Plugin slots: Exclusive evaluation_runtime and trajectory_seal default to the in-environment evaluator and layer-C trajectory. plugin.yaml may set a Hub description. ageval plugin publish --replace overwrites the same package_id@version. plugin_requires, home-files, and agent-skills are opt-in chain slots.
• Campaign and suite: ageval campaign is a task-axis matrix; a failed cell stays one cell. Named finished suite slots can be replaced and re-uploaded without rewriting the whole row.
• Hub and Viewer: Create orgs from the list page; manage members and transfer owner. Org owners flip visibility, delete a version, or release a draft. Uploaders share, hide, or delete results. Plugin description sits above Install. Job Harness shows the ACP entry (pi), not the transport acp. Viewer reads the Attempt trajectory and the real environment kind.

Fix

• Fail-closed scoring: Verdict bind accepts only PASS / FAIL / ERROR. A failed or timed-out task worker is a run-phase ERROR and does not fall into evaluate.
• ACP and environments: Advertised models bind by exact id. E2B ACP uses real stdio pipes. SSH copies directory contents on upload/download. DSH drops empty OpenAI tool-call fields before the official adapter. Artifact harvest looks up the declared relative path.

Docs

• Public copy: README, website, and skills speak ageval + environments. Smoke commands use examples/journeys. Clone and Hub links are ZJU-REAL/ageval.

中文

本版本是 ageval 外层评测运行时：锁定 dataset，打开一个环境，运行 run.py，PASS 只来自独立 evaluator。CLI、包和 format 都是 ageval。Coding agent 走 ACP；其它机制以插件安装，再由 job profiles 绑定。环境是独占槽 local / docker / e2b / ssh。Agent 包（ageval.agent/1）用 --agent 绑定一次 job。没有 bora 兼容层。证据等级仍是 runnable-mvp。默认 CI 不声称 e2b / ssh 的真 Agent 运行。

破坏性变更

• 产品更名: CLI ageval，import ageval / ageval_sdk，家目录 ~/.ageval，环境变量 AGEVAL_*。Format 为 ageval.dataset/1、ageval.task/1、ageval.plugin/1、ageval.profiles/1、ageval.agent/1。未知 format 只报一个 invalid_format。没有 bora 命令、没有 BORA_* 别名、没有双读。GitHub 为 ZJU-REAL/ageval。
• Attempt 流水线: L0/L1 生命周期删除。相位在 src/ageval/attempt/：environment → run → evaluate → record，cleanup 在 finally。题包 run.py 在子进程里运行，绑到父进程 Agent Service。gold（evaluation/）在 evaluate 才进入环境。轨迹 / RunTerminal.completed 不是 PASS。
• 环境与插件: 环境写在 profiles.yaml 的 environment:，不是 provider.kind。ACP 从环境拿 attach_stdio；parent 是唯一 JSON-RPC client。dsh / nooa / miniswe 经 host.exec 调用。--agent 与 --profiles 互斥。保留的例子是 examples/journeys、examples/tau3-airline 和 examples/agents；examples/core 与 examples/l1 已删除。

新功能

• 环境 kind: docker / e2b / ssh 与 local 同一 Protocol。插件镜像层从插件包 bake。目录 upload 替换目标树。SSH A 仍不支持 live ACP stdio。
• Agent Hub: ageval.agent/1 的 publish / install / 本地缓存。ageval agent，以及 lock / run / campaign 上的 --agent。声明的 overlay 文件随 Agent 包走。官方公开 suite 出场收进 Agent 详情。
• 插件槽: 独占槽 evaluation_runtime / trajectory_seal 默认是环境内 evaluator 和层 C 轨迹。plugin.yaml 可写 Hub description。ageval plugin publish --replace 可覆盖同一 package_id@version。plugin_requires、home-files、agent-skills 是可选链槽。
• Campaign 与 suite: ageval campaign 是 task 轴矩阵；失败的一格仍是一格。已完成的具名 suite 槽可以替换后再上传，不必整行重写。
• Hub 与 Viewer: 列表页可建组织；管理成员、转让 owner。组织 owner 可改可见性、删版本、发 release。上传者可分享 / 隐藏 / 删除结果。插件说明画在 Install 上方。Job 的 Harness 显示 ACP entry（如 pi），不是传输名 acp。Viewer 读 Attempt 轨迹和真实 environment kind。

修复

• 打分 fail-closed: 裁决只接受 PASS / FAIL / ERROR。失败或超时的题包 worker 记为 run 相位 ERROR，不会掉进 evaluate。
• ACP 与环境: 广告模型按精确 id 绑定。E2B ACP 使用真 stdio 管道。SSH 上传/下载复制目录内容。DSH 在官方 adapter 之前丢掉空的 OpenAI tool-call 字段。产物回收按声明的相对路径查找。

文档

• 对外文案: README、website 与 skills 使用 ageval 与「环境」。公开烟测命令使用 examples/journeys。克隆与 Hub 链接为 ZJU-REAL/ageval。

What's Changed

• feat(hub): official user identity and first-class org plugins by @ffy6511 in #123
• feat(registry,hub): transfer org owner and manage members by @ffy6511 in #124
• feat(registry): public concurrency and whole-object streaming by @ffy6511 in #125
• feat(plugins): per-plugin options and home_overlay by @ffy6511 in #134
• feat(eval): declare L1 clean-eval /tmp size on evaluation.tmpfs_mb by @ffy6511 in #133
• feat(hub): derive Runtime plaza from official public Leaderboards by @ffy6511 in #131
• feat(suite): append a replacement attempt onto an existing suite by @ffy6511 in #127
• feat(eval): L1 evaluation.network and optional same-Attempt eval by @ffy6511 in #138
• feat(plugins): plugin_requires graph and agent-skills overlay by @ffy6511 in #141
• feat(acp): move first-party ACP into the plugin; grok-build argv bind by @ffy6511 in #143
• feat(hub): binding.overlays for Runtime plaza file tree by @ffy6511 in #145
• feat: Agent Hub — first-class bora.agent/1 object (publish / install / run via --agent) by @tricktreat in #147
• feat(agent-hub): ship overlay files and fold Runtime plaza by @ffy6511 in #155
• fix(acp): allowlist child env, credential health, side-by-side agents by @ffy6511 in #156
• docs: strip leftover bora product-name traces by @ffy6511 in #159
• feat(plugins): invoke dsh and nooa only through the environment protocol by @ffy6511 in #160
• feat(runtime): exclusive eval/seal slots and miniswe Protocol inject by @ffy6511 in #166
• feat: Hub owner ops and fail-closed core fixes by @ffy6511 in #167
• refactor: trim unused core surface and cut lock/run I/O by @ffy6511 in #168
• feat!: promote canary — ageval Attempt pipeline, environment slots, Hub owner ops by @ffy6511 in #169

New Contributors

• @tricktreat made their first contribution in #147

Full Changelog: v0.5.2...v0.6.0