Skip to content

test(e2e): state the seed-visibility precondition instead of depending on it (#665) - #668

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-665-e2e-seed-precondition
Aug 4, 2026
Merged

test(e2e): state the seed-visibility precondition instead of depending on it (#665)#668
os-zhuang merged 1 commit into
mainfrom
claude/issue-665-e2e-seed-precondition

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #665

按 PM 裁定实现方案 3:把 e2e 套件一直隐式依赖、却从未声明的前置条件——「种子记录此刻对套件登录的账号可见」——在 e2e/global-setup.ts 里显式断言。不满足时整轮 run 以一条说明真实成因和可执行补救的错误中止,而不是让 11 个 spec 带着 no seeded accounts returned / the demo seed did not load 级联失败(那句话描述的是一个加载得好好的种子)。

套件证明的东西没有任何变化:不加共享授权、不加权限集、不改用 seeded dev admin。方案 1 / 2 明确不在本 PR 范围,方案 2 已单独立案(见文末)。

两个探针,而不是一个

只读一次 crm_account 无法区分「种子被别人认领了」和「种子根本没播种」——两者都返回 0 行,而补救方式相反。所以多花一个请求,读一个可见性与 ownership 无关的对象:

探针 OWD demo_bootstrap 扫描 种子被认领后
crm_account private 归零
crm_product public_read 始终可见

两者都在 CrmSeedData 里。于是「有 product、没有 account」只能是种子在、但属于别人;「两者都没有」只能是根本没播种。这两条前提不是注释,test/e2e-seed-precondition.test.ts 直接对元数据断言:把 crm_product 加进 demo_bootstrapCLAIMED_OBJECTS,或改掉任一 sharingModel,测试就红——否则守卫会悄悄退化成把「被认领」误报成「没播种」,正是 #665 要消灭的那种误诊。

为什么这条守卫不可能把 CI 变红

三层保证,前两层是结构性的:

  1. 只在套件本来就要失败的状态里才可能触发。 第一个探针读到任何一行就立刻返回,连第二个请求都不发。CI 现在能过 smoke.spec.tsrecords.length > 0,就意味着这个探针在 CI 上恒 > 0。
  2. 仍在播种中的冷库会被等待,而不是被判成 absent。 waitForQuiet 用 health 延迟推断「播种风暴过去了」,那是代理指标不是事实。守卫最多轮询 30s,行一出现立即返回。
  3. 实测 CI 那条路径的机制。 CI 的 webServerobjectstack start,它不播种 dev admin:套件注册的账号就是组织里的第一个用户(sys_user 恰好 1 行),demo_bootstrap 于是把种子认领给它自己——冷库上实测每条 crm_accountowner_id 都等于该账号 id。CI 一直绿是因为这个,不是因为运气。而 pnpm dev 播种 admin@objectos.ai,第一个用户变成那个 admin,下一个整十分钟边界一到,种子就被从套件手里认领走。

验证(全部在本机对真实 booted server 跑出来,不是推理)

A. objectstack dev + 已被认领的种子 → 守卫按预期响亮失败(即 #665 的复现态)。用真实的 e2e/global-setup.ts(sign-up/sign-in + 守卫)打一台 pnpm demo:reset 后启动的 dev server:

=== GLOBAL SETUP FAILED ===

e2e precondition failed: the demo seed is loaded but INVISIBLE to e2e-admin@hotcrm.test.

`crm_account` returned 0 rows while `crm_product` returned rows — so the seed
is there, and the private records are simply owned by another user. ...

Remedy — run the suite the way CI does: a cold database served by `objectstack start`.

    pnpm demo:reset          # rm -rf .objectstack/data && rebuild
    pnpm start               # terminal 1 — NOT `pnpm dev`, see below
    pnpm test:e2e            # terminal 2

以 dev admin 身份核对成因:9 条 crm_accountowner_id 全是 dev admin 的 id(server 于 :49 启动,:50 的 sweep 已认领完毕)。

B. objectstack start 冷库(CI 路径)→ 守卫不触发:

=== GLOBAL SETUP PASSED — the guard did not fire ===

同一台 server 上以套件账号读:sys_user 只有 1 行(e2e-admin@hotcrm.test),sweep 跑过之后 crm_account.owner_id 全部等于该账号 id——即 CI 上 sweep 前后都可读。

C. 真正的 playwright test 走一遍(复用同一台 server,含真实 global setup + 守卫):

✓ 1 [chromium] › e2e/smoke.spec.ts:42:5 › the CRM data API is auth-gated (42ms)
✓ 2 [chromium] › e2e/smoke.spec.ts:51:5 › REST API serves seeded hotcrm records to an authenticated caller (118ms)
✓ 3 [chromium] › e2e/smoke.spec.ts:62:5 › every core CRM object is queryable (802ms)

  3 passed (4.3s)

容器里没有 chromium,page 驱动的那几个 spec 只能交给 CI 跑;它们不读种子行,与本改动无关。

D. 单元与静态检查:

pnpm typecheck                     → 通过(tsc --noEmit,e2e/ 与 test/ 都在 include 内)
pnpm test                          → 49 files / 1180 passed | 1 skipped
pnpm exec vitest run test/e2e-seed-precondition.test.ts → 15 passed
pnpm hygiene                       → clean

新测试用结构化的请求上下文接口对着 stub 跑,所以「绝不能在 CI 触发的那条分支」有测试覆盖,却不需要浏览器、server 或数据库。

仍会被跑到的两处措辞

守卫只在 global setup 跑一次,而 demo_bootstrap 每十分钟一次,所以 sweep 可能在一轮 run 中途认领种子——e2e/smoke.spec.tsseededAccountId() 因此仍可达。这两处现在带上同一句成因(SEEDS_UNREADABLE_MID_RUN),而不再指控种子加载器。

后续

方案 2(给 e2e 账号显式授权,或让 spec 自持数据,从而彻底摆脱对种子 ownership 的依赖)已作为未指派的独立 issue 立案,引用 #665 与本 PR。


Generated by Claude Code

…g on it (#665)

`pnpm test:e2e` against a dev server that had been up for more than ten minutes
failed 11 of 16 specs on "no seeded accounts returned" and "no seeded
crm_account — the demo seed did not load". The seed had loaded. `global-setup.ts`
signs UP `e2e-admin@hotcrm.test`, which lands as a plain org member owning
nothing and holding no sharing grant; every seeded row starts out ownerless,
which under `sharingModel: 'private'` is the only reason the suite could read it.
Once `demo_bootstrap` (or `pnpm demo:staff`) claims those rows for the first
user, the suite reads zero — and blamed the seed loader.

Global setup now asserts the precondition. Two `?limit=1` reads separate the two
states that both look like zero rows: `crm_account` is `private` and swept by
`demo_bootstrap`, so it goes dark the moment the seeds are claimed; `crm_product`
is `public_read` and in no sweep, so no ownership state can hide it. Products but
no accounts means the seeds are there and claimed, and the run aborts with that
sentence plus the remedy; neither means nothing was seeded, and says so.

The guard cannot turn a passing run red: it returns on the first readable row
without issuing the second probe, and waits out a seed that is still loading
rather than calling it absent. Measured on both server modes — `objectstack
start` (CI's webServer) seeds no dev admin, so the e2e account is the org's first
user and the sweep claims the seeds FOR it; `objectstack dev` seeds
admin@objectos.ai and the sweep claims them away.

What the suite proves is unchanged: no sharing grant, no permission set, no
switch to the seeded dev admin. `test/e2e-seed-precondition.test.ts` pins the
guard's metadata premises against the real objects and flow, so adding
`crm_product` to `CLAIMED_OBJECTS` fails rather than silently degrading the
diagnosis back to the misleading one.
@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hotcrm Ignored Ignored Aug 4, 2026 12:58am

Request Review

@github-actions github-actions Bot added the ci/cd CI plumbing and the verification pipeline label Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

补一条本地实测,补齐 PR 正文里「守卫不会把 CI 变红」的最后一格——在 CI 那条路径上,demo_bootstrap 真的跑过之后守卫依然不触发

同一台 objectstack start server(冷库,pnpm demo:reset 之后启动),跨过 01:00 这个整十分钟的 sweep 边界再跑一次真实的 e2e/global-setup.ts:

01:00:43
--- guard after the 01:00 demo_bootstrap tick (production mode) ---

=== GLOBAL SETUP PASSED — the guard did not fire ===

原因就是正文第 3 点的机制,这里是它的直接读数(以套件账号 e2e-admin@hotcrm.test 读):

GET /api/v1/data/sys_user?limit=5&fields=id,email
{"records":[{"id":"zrw7HSesLI0ugNC8ak2rRSdQZXG3sFrS","email":"e2e-admin@hotcrm.test"}],"total":1}

GET /api/v1/data/crm_account?limit=3&fields=id,name,owner_id
{"records":[
  {"name":"Initech Solutions","owner_id":"zrw7HSesLI0ugNC8ak2rRSdQZXG3sFrS"},
  {"name":"Wayne Enterprises","owner_id":"zrw7HSesLI0ugNC8ak2rRSdQZXG3sFrS"},
  {"name":"Stark Medical",    "owner_id":"zrw7HSesLI0ugNC8ak2rRSdQZXG3sFrS"}],
 "total":9}

objectstack start 不播种 dev admin,组织里只有套件自己这一个用户,sweep 于是把 9 条 account 全部认领给它。所以在 CI 上,sweep 之前(行是无主的,谁都能读)和 sweep 之后(行归套件自己)都可读——守卫的失败分支在 CI 上不可达,而不只是「大概不会触发」。

对照组是 pnpm dev:同样 demo:reset 之后启动,:49 起服务,:50 的 sweep 一跑,9 条 account 的 owner_id 全变成 dev admin(admin@objectos.ai)的 id,守卫按预期响亮失败——即 #665 的复现态。


Generated by Claude Code


Generated by Claude Code

@os-zhuang
os-zhuang marked this pull request as ready for review August 4, 2026 01:03
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 4, 2026
Merged via the queue into main with commit 05daa5c Aug 4, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd CI plumbing and the verification pipeline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pnpm test:e2e fails against a long-lived local dev server once demo_bootstrap has claimed the seeds

1 participant