Skip to content

fix(ci): never render a budget FAIL for a run that measured nothing - #3198

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-3152-bundle-analysis-cancelled-fail
Aug 2, 2026
Merged

fix(ci): never render a budget FAIL for a run that measured nothing#3198
os-zhuang merged 1 commit into
mainfrom
claude/issue-3152-bundle-analysis-cancelled-fail

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #3152

问题

.github/workflows/performance-budget.yml 在运行被取消时仍会发一条 ❌ FAIL 评论,而预算从未被测量过。工作流设了 concurrency: cancel-in-progress: true,这意味着每个连推两次提交的 PR 都会收到这条假警报。

同一份代码上的对照:运行 30699128418(cancelled)判 FAIL 且三项指标全空;运行 30699202638(success)判 PASS,实测 28.1 KB / 350 KB —— 只用掉预算的 8%。

两处机制:

  1. 发评论的步骤条件是 if: ... && always()always() 在运行被取消时照样触发;而渲染逻辑把「非 pass」等同于 FAIL(status === 'pass' ? '✅' : '❌'),于是「没测到」被渲染成了「测到了且超标」。
  2. Generate package size report 同样是 if: always(),取消时它对着只构建了一部分的 packages/*/dist 跑完并输出 —— 报告少了 7 个包(app-shell、plugin-calendar、plugin-designer、plugin-gantt、plugin-kanban、plugin-report、plugin-view)却看不出缺失,让那条假 FAIL 更像一份可信的完整报告。

改动

门禁语义:FAIL 必须带着数字。

  • 评论步骤改用 !cancelled() 而不是 always()。被取消的运行不再发任何评论 —— 取代它的那次运行几十秒后就会给出真实结论。
  • 只有「测量到了超标」才渲染 FAIL:渲染器要求 budget_statuspass/fail,并且 gzip 大小、预算、入口文件名三项都在。真实超标必然带着这三个数字,这正是 issue 里点出的可判别信号。
  • 「没测到」改发一条中性说明,正文明确写 This is not a budget violation,并列出 Build packages / Budget check 两步的 outcome 与失败原因。

让生产者声明结果,而不是让消费者从「空」去猜。

  • budget 步骤原本有两条退出路径(dist 目录不存在、找不到 JS 文件)什么 output 都不写。现在它们显式写 budget_status=error 和一条 budget_message。消费者读到的是一个声明过的结果,而不是靠空字符串反推。同理,渲染器用的是显式白名单 MEASURED_STATUSES,而不是 status !== 'pass' 那个取反 —— 正是那个取反把「没有数据」变成了「FAIL」。

半截报告。

  • Generate package size report 改成 if: !cancelled() && steps.build_packages.outcome == 'success',只在包全部构建完成后才生成。生成不出来时评论里会写明「没有体积报告」,而不是静默省略 —— 因为一份被截断的报告读起来和完整的一模一样。

渲染逻辑搬出 YAML。

  • 新增 scripts/render-budget-comment.mjs,评论步骤改为 node scripts/render-budget-comment.mjs > budget-comment.md,step outputs 经 env: 传入而不是插值进 JS 字符串字面量。这个 bug 本身就是一个渲染 bug,而内联在 script: 块里的逻辑没有任何测试能覆盖它。现在它由 scripts/__tests__/render-budget-comment.test.ts 覆盖(17 个用例),其中 6 个是钉住工作流本身的 pin test(禁止 always()!cancelled() 的三处、报告的构建门、env 契约、budget_status 的三种取值)。

每条退出路径的预期行为

场景 Build packages budget_status 体积报告 PR 评论
运行被取消(连推两次) 部分完成 缺失(步骤没跑) 跳过 不发评论
包构建失败 failure 缺失(步骤 skipped) 跳过 ℹ️ not measured
console 构建失败 success 缺失(步骤 skipped) 生成 ℹ️ not measured + 完整报告
dist 目录不存在 success error 生成 ℹ️ not measured + 原因
dist 里没有 JS success error 生成 ℹ️ not measured + 原因
未超预算 success pass 生成 ✅ PASS + 实测数字
超预算 success fail 生成 FAIL + 实测数字(行为不变)

验证

1. YAML 可解析,步骤条件符合预期

YAML PARSE: OK, steps = 12
- Build packages [id=build_packages]
- Check console performance budget [id=budget]
- Generate package size report [id=size-report]  if: ${{ !cancelled() && steps.build_packages.outcome == 'success' }}
- Render performance budget comment [id=render_comment]  if: ${{ github.event_name == 'pull_request' && !cancelled() }}
- Comment PR with results  if: ${{ github.event_name == 'pull_request' && !cancelled() }}

2. 把 budget 步骤的 shell 原样从 YAML 里抽出来执行,逐条验证 output

== PATH A: dist 目录不存在 ==        exit=1   budget_status=error
                                             budget_message=Build output not found at apps/console/dist/assets
== PATH B: dist 存在但没有 JS ==      exit=1   budget_status=error
                                             budget_message=No JS files found in apps/console/dist/assets
== PATH C: 未超预算 ==                exit=0   gzip_kb=40.1 budget_kb=350
                                             entry_file=index-abc123.js  budget_status=pass
== PATH D: 超预算 ==                  exit=1   gzip_kb=701.8 budget_kb=350
                                             entry_file=index-over.js    budget_status=fail

3. 把 PATH D 的 output 原样喂给渲染器(端到端):真实超标信号完全保留

## ❌ Console Performance Budget

| Metric | Value | Budget |
|--------|-------|--------|
| Main entry (gzip) | **701.8 KB** | 350 KB |
| Entry file | `index-over.js` | — |
| Status | **FAIL** | — |

4. 取消 / 未测量路径:不再有 ❌,也不再有 FAIL

## ℹ️ Console Performance Budget — not measured

This run did not produce a console bundle to measure, so there is no pass/fail verdict.

**This is not a budget violation.** Nothing was measured — the numbers a real violation would carry are simply absent.

| Step | Outcome |
|------|---------|
| Build packages | `failure` |
| Check console performance budget | `skipped` |

5. 测试与静态检查

$ pnpm exec vitest run --project unit scripts/__tests__/render-budget-comment.test.ts
 Test Files  1 passed (1)
      Tests  17 passed (17)

$ pnpm exec vitest run --project unit          # 确认 vitest.config.mts 的 include 改动没有影响其它文件
 Test Files  332 passed (332)
      Tests  4657 passed | 1 skipped (4658)

$ pnpm exec tsc --noEmit --ignoreConfig --strict ... scripts/__tests__/render-budget-comment.test.ts
 (无输出,0 错误)

$ pnpm exec eslint scripts/render-budget-comment.mjs scripts/__tests__/render-budget-comment.test.ts
 ESLINT_EXIT=0

$ node scripts/check-changeset-fixed.mjs
 ✅  All workspace packages are in the changeset fixed group.

注:本 PR 只改动 .github/**scripts/**vitest.config.mts 和 docs,不匹配 performance-budget.yml 自己的 paths 过滤器(packages/**apps/console/**pnpm-lock.yaml),所以这个工作流不会在本 PR 上运行。这也正是为什么渲染逻辑被搬进了可单测的文件,而不是留在 YAML 里靠 CI 碰运气。

关于 changeset

按 AGENTS.md §9「纯 bug 修复不需要 changeset」,且本次改动不触及任何已发布包的源码(只有 CI 工作流、仓库级 CI 脚本、vitest 配置和文档),没有版本需要 bump。仓库里唯一的 changeset 门禁 scripts/check-changeset-fixed.mjs 校验的是 fixed group 成员资格,不要求每个 PR 都带 changeset,已通过。

顺带发现(未在本 PR 修)


🤖 Generated with Claude Code

https://claude.ai/code/session_01PRJtkgUAaVG11FsJQbvZWA


Generated by Claude Code

A cancelled Bundle Analysis run posted "❌ Console Performance Budget /
Status: FAIL" with all three metrics empty. `cancel-in-progress` kills the
first run on every second push, so every PR pushed twice within one build
got a fake budget alarm — run 30699128418 (cancelled) said FAIL while
30699202638 (success, same commit) measured 28.1 KB against a 350 KB budget.

Cause: the comment step was gated on `always()`, which fires on cancellation
too, and its renderer treated "not pass" as FAIL — so an absent measurement
became a verdict.

- Gate the comment on `!cancelled()`, not `always()`: a cancelled run posts
  nothing, because the superseding run posts the real verdict moments later.
- Render FAIL only when the bundle was measured AND over budget. A run that
  produced nothing measurable gets a neutral "not measured" note that states
  it is not a budget violation.
- Make the budget step declare `budget_status=error` on its two "nothing to
  measure" exits instead of leaving every output empty, so the renderer reads
  an outcome rather than inferring one from silence.
- Generate the package size report only from a complete package build. Under
  `always()` it ran against a partially built `packages/*/dist` and emitted a
  report that was silently missing 7 packages, which made the fake FAIL look
  like a complete, credible report.

Rendering moves out of the YAML into scripts/render-budget-comment.mjs so it
can be unit tested — the bug was purely a rendering bug, and logic inlined in
a `script:` block cannot be covered by any test.

Fixes #3152

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRJtkgUAaVG11FsJQbvZWA
@vercel

vercel Bot commented Aug 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectui Ignored Ignored Aug 2, 2026 11:14am

Request Review

@os-zhuang
os-zhuang marked this pull request as ready for review August 2, 2026 11:31
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 2, 2026
Merged via the queue into main with commit 4f66d83 Aug 2, 2026
15 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-3152-bundle-analysis-cancelled-fail branch August 2, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bundle Analysis:被取消的运行会发出一条 ❌ FAIL 评论,三项指标全为空——每个连推两次的 PR 都会收到假警报

2 participants