Skip to content

feat(plugin-email,platform-objects): sys_email 加 headers_json + 有界 attachments_json —— 自定义 header 与小附件取得队列投递资格 (#5177) - #5211

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-5177-sys-email-headers-attachments
Aug 4, 2026
Merged

feat(plugin-email,platform-objects): sys_email 加 headers_json + 有界 attachments_json —— 自定义 header 与小附件取得队列投递资格 (#5177)#5211
os-zhuang merged 2 commits into
mainfrom
claude/issue-5177-sys-email-headers-attachments

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #5177

为什么这是一处真实的数据丢失面

耐久投递从来不是从内存里的消息投出去的,而是sys_email投出去的:send() 发布的是 { rowId }(#5160),开机清扫重新读行(#5161),两条路径最后都走到 rowToNormalized。所以行装不下的部分,就是行式投递会静默丢掉的部分 —— 自定义 header 和附件正好是这两样。

#5160 当时的处理是诚实的:带这两样的消息一律退回内联投递,至少能整封发出去。代价是最值得做成耐久的邮件恰好被挡在耐久路径之外(带签名的回执、List-Unsubscribe、发票 PDF)。本单把这道门打开。

A. headers_json(终态)

  • sys_email 新增可空列;send() 两种模式都落(它既是投递输入也是审计证据);rowToNormalized() 重建;
  • header 不再是退回内联的理由。

B. attachments_json(一期:有界内联,schema 一次到位)

元素形状:{ filename, contentType?, size, hash, cid?, contentForm, inline?, storageKey? },内容 base64 落在 inline

  • 上限 SYS_EMAIL_ATTACHMENT_LIMIT_BYTES = 256 KiB(整封合计原始字节),从 @objectstack/plugin-email 导出,不做配置项。最坏情况列里 ~350 KB base64,行体积有界;
  • 超限维持现行为:整封内联投出去(不剥附件),列里什么都不写,理由以 info 说明「合计超过 X 字节」。超限不是错误,最坏结果 = 今天的现状,所以是 info 不是 error;
  • cid 进形状 —— HTML 正文里用 cid: 引用的内联图,少了它就是坏图;
  • storageKey 一期无生产者,TSDoc 已注明由 plugin-email: sys_email 无附件/自定义 header 列,带附件的邮件拿不到队列投递的持久化保证(#5160 落地后) #5172 跟踪,防 liveness 扫描误判为死键。将来支持大附件是给已声明字段补生产者,不动 schema、不迁移数据。

一处对 issue 所给形状的补充,请维护者过目

Issue 给的形状里 contentType 未标可选、也没有区分 content 两种形态的键。落地时这两点都需要动,理由都是可实测的:

1. contentType 改为「调用方给了才落」,不默认 application/octet-stream
传输层在没有 contentType 时会按文件名推断(nodemailer:report.pdfapplication/pdf)。若我们补一个默认值,同一封信走队列和走内联会拿到不同的 MIME 类型,直接违反验收里「附件完整还原」。

2. 新增必填 contentForm,取值 stringbuffer
契约是 content: string | Buffer,而 nodemailer 对两者的输出并不等价 —— 实测同一段 UTF-8 内容,附件那一节的 Content-Type 头分别是:

string content →  Content-Type: text/plain; charset=utf-8; name=...
Buffer content →  Content-Type: text/plain; name=...

base64 正文完全一致,但 charset=utf-8 声明没了。把字符串附件还原成 Buffer 会让收件端客户端拿到一个再也认不出编码的 UTF-8 文件 —— 字节对了,内容烂了。所以行里记下发送时用的是哪一支,而不是猜(猜正是本模块拒绝做的那种静默强转)。这一条由 sys-email-payload.wire.test.ts 的 charset 用例直接钉住。

若维护者更希望严格照原形状走(接受上述两处偏差),回退成本很小,请在 review 里说一声。

解码为什么是「严格拒绝」

这里每一种失败模式的后果都是「收件人收到的不是发出去的那封信」,而且从外面完全看不出来:行是 sent,SMTP 回 250。所以列在但内容对不上(JSON 坏、size/hash 与内容不符、缺 contentForm)一律抛错,由 deliverPersistedRow 落成带原因的 failed 行,而不是投一封缺零件的信。严到读时重算 sha256 —— size/hash 不是装饰,它们正是把「列被截断/被改写」变成一次响亮失败的东西。

两列都不存在的老行读取完全安全,默认(内联)模式行为不变,既有断言零改动。

清扫与订阅者路径

两条路径都经 rowToNormalized,因此自动受益;各补了端到端用例(见下)。inline 模式也写这两列,正是因为 #5161 的开机清扫在 inline 模式下同样从行重投 —— 崩溃后残留的行若没带 header/附件,会被再次剥光重发,而这恰是本单要消灭的丢失类。

测试

  • sys-email-payload.test.ts(24 例):编码/解码、原始字节计量(一个中文字是 3 字节而不是 1 个字符)、上限含边界、老行安全、以及 8 种「列在说谎」的拒绝路径;
  • sys-email-payload.wire.test.ts(4 例):真 nodemailer + 进程内假 SMTP(复用 smtp.wire.test.ts 的设施),断言「内联直发」与「经行重建后发」产出的 MIME 逐字节一致、两个附件字节一致、中文文件名完好、text 附件 charset=utf-8 未丢、Content-ID 在;
  • email-plugin.queue-delivery.test.ts:send() → 真 DbQueueAdapter → worker 投递,header 与两种 content 形态的附件完整到达 transport;
  • email-plugin.outbox-sweep.test.ts:队列模式与 inline 模式各一条,搁浅行带 header/附件被完整重投;
  • email-service.queue-delivery.test.ts:小附件/header 现在入队并落列;超限不落列且仍整封内联发出;上限按整封合计而非单个附件;off-contract content 退回内联;inline 模式同样落列;两样都没有则两列都不写。

smtp.wire.test.ts 只做了「假 SMTP server 与解码器搬进 transports/fake-smtp.testkit.ts」的机械抽取,断言零改动 —— 两份手写假服务器一定会漂移,而这套设施的全部价值就在于「传输层与线缆之间没有任何东西」。

pnpm --filter @objectstack/plugin-email --filter @objectstack/platform-objects test
  platform-objects  Test Files 9 passed (9)    Tests 266 passed (266)
  plugin-email      Test Files 15 passed (15)  Tests 231 passed (231)

pnpm --filter @objectstack/plugin-email --filter @objectstack/platform-objects typecheck
  platform-objects: Done      plugin-email: Done

pnpm check:engine-double-contract   OK — 17 pinned, 31 in the DEBT ledger, 1 exempt
pnpm check:durability-log-level     14 durability-critical catch seam(s), all loud or rethrowing
pnpm check:published-files          OK
pnpm check:nul-bytes                OK

边界

🤖 Generated with Claude Code

https://claude.ai/code/session_017MCKJaEomEqg4tvz4SzdNd

…ded attachments (#5177)

Durable delivery works from the sys_email ROW, not the in-memory message
(#5160 publishes `{ rowId }`, #5161 re-reads rows, both end at
`rowToNormalized`), so anything a row could not carry was silently dropped
by a row-based delivery. Custom headers and attachments were exactly that,
and the honest workaround was to refuse them: such a message was pushed
back onto inline delivery so it would at least go out whole — closing the
durable path to precisely the mail most worth making durable.

- `sys_email.headers_json`: custom headers as JSON, written in BOTH delivery
  modes and rebuilt on read. Headers are no longer a reason to refuse the
  queue.
- `sys_email.attachments_json`: attachments as
  `{ filename, contentType?, size, hash, cid?, contentForm, inline?, storageKey? }`,
  content base64 in `inline`, written only when one message's combined RAW
  attachment size is within `SYS_EMAIL_ATTACHMENT_LIMIT_BYTES` (256 KiB;
  ~350 KB of base64 worst case, so the row stays bounded). Over the limit,
  behaviour is unchanged — delivered inline and whole, nothing stored, the
  reason stated at `info`.
- `contentForm` records which arm of `content: string | Buffer` was sent:
  restoring a text attachment as a Buffer round-trips the bytes yet drops
  `charset=utf-8` from its MIME part, which mis-decodes a UTF-8 file at the
  recipient. Proven on a real nodemailer + in-process SMTP wire test.
- `storageKey` is declared with no phase-1 producer (objectstack#5172 adds
  one) so large-attachment support lands as a producer, not a migration.
- Decoding is strict: malformed JSON, a size/hash that disagrees with the
  content, or a missing `contentForm` REJECTS the row (it lands at `failed`
  with the reason) instead of delivering a message with a part missing.
  Rows written before these columns read exactly as before.

The fake SMTP server + wire decoders move from `smtp.wire.test.ts` into
`transports/fake-smtp.testkit.ts` so the round-trip suite proves itself
against the same wire rather than a second hand-written copy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MCKJaEomEqg4tvz4SzdNd
@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 4, 2026 10:22am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/platform-objects, @objectstack/plugin-email.

5 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/automation/flows.mdx (via @objectstack/plugin-email)
  • content/docs/deployment/environment-variables.mdx (via @objectstack/plugin-email)
  • content/docs/plugins/packages.mdx (via @objectstack/platform-objects, @objectstack/plugin-email)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/plugin-email)
  • content/docs/ui/setup-app.mdx (via @objectstack/platform-objects)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

`headers_json` / `attachments_json` 进 `sys-email.object.ts` 时没有重跑 i18n
抽取,四个 bundle 相对 schema 漂移,`pnpm check:i18n` 因此在 "TypeScript Type
Check" 这个 job 里变红(tsc 本身 119/119 全过 —— 红的是同 job 靠后的 i18n 步骤,
不是类型错误)。

按 gate 自己给的方式重生成(`node scripts/check-i18n-bundles.mjs --write`),
并把两个新 key 的 label/help 翻成 zh-CN / ja-JP / es-ES —— `sys_email` 其余字段
在这三个语言里都是译好的,merge 模式只会用源文填坑并提示 "they still need
translating",留英文会在译好的邻居中间留下两行英文。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MCKJaEomEqg4tvz4SzdNd

Copy link
Copy Markdown
Contributor Author

接手补验:CI 红的真因是 i18n bundle 漂移,不是类型错误

原 dev agent 推完这个分支后会话被终止,没留下验证证据。接手后逐条重验,并定位了那条唯一的红。

诊断

红的 job 叫 TypeScript Type Check(run 30895672389 / job 91947847849),但拉全量日志(不只 tail)后,tsc 本身是干净的:

 Tasks:    119 successful, 119 total
Cached:    92 cached, 119 total

失败发生在同一个 job 靠后的一个非 tsc 步骤上 —— pnpm check:i18n

  platform-objects               DRIFTED (4)

check-i18n-bundles: 1 bundle problem(s)
  • platform-objects: 4 bundle(s) drifted from the schema

Regenerate and commit: node scripts/check-i18n-bundles.mjs --write

真因很直接:headers_json / attachments_json 两个新列带着 label + description 进了 sys-email.object.ts,但没有重跑 i18n 抽取,四个 bundle 因此相对 schema 漂移。与本单的实现逻辑无关,是加列必须配套的生成物没跟上。

(这一步在本地复现需要先构建工作区 —— 该门禁跑的是 packages/cli/bin/run.js。未构建时它报的是 command i18n:extract:... not found,容易被误读成配置缺失。)

修法

按门禁自己给出的方式重生成,并把两个新 key 译成 zh-CN / ja-JP / es-ES:merge 模式只会用源文填坑并提示 “they still need translating”,而 sys_email 其余字段在这三个语言里都是译好的,留英文会在译好的邻居中间留下两行英文。

945935ed6 —— 只动 packages/platform-objects/src/apps/translations/ 下的四个生成文件,实现代码零改动。

全量重验(flock 串行 + --max-old-space-size=4096

pnpm --filter @objectstack/plugin-email --filter @objectstack/platform-objects exec vitest run --maxWorkers=2
  platform-objects   Test Files  9 passed (9)     Tests  266 passed (266)
  plugin-email       Test Files  15 passed (15)   Tests  231 passed (231)

pnpm --filter @objectstack/plugin-email --filter @objectstack/platform-objects run typecheck
  packages/platform-objects typecheck: Done
  packages/plugins/plugin-email typecheck: Done

node scripts/check-engine-double-contract.mjs
  OK — 17 pinned, 31 in the DEBT ledger, 1 exempt
  (本包两个假引擎都在 pinned 名单里:email-plugin.outbox-sweep / email-plugin.queue-delivery)

npx eslint <15 个改动文件>        exit 0,零输出

node scripts/check-i18n-bundles.mjs   ← 之前红的那条
  platform-objects               in sync (8 bundle(s))
  ...
  check-i18n-bundles: OK (9 package(s) — all bundles in sync, no undeclared authoring keys).

对分支既有实现的核对结论

逐块核过 issue #5177 的验收条款,未发现不符合规格之处,因此除 i18n 外没有改动实现:headers 两种模式都落且不再是退回内联的理由;attachments 元素形状含 cidstorageKey 扩展点(TSDoc 已注明一期无生产者、由 #5172 跟踪);256 KiB 上限为导出常量、按整封合计原始字节且边界含等号;超限退回内联 + info列里不写任何内容email-service.queue-delivery.test.ts 有直接断言 attachments_jsonundefined);Buffer/string 双形态经 contentForm 各自还原;老行两列皆无读取安全。端到端由真 DbQueueAdapter 与真 nodemailer + 进程内假 SMTP 的逐字节断言兜住。

packages/spec、⛔ content/docs/releases/、⛔ #5172 二期均未触碰;未 force-push,PR 保持 draft,认领与 assignee 未动。

仍需维护者定夺的一点:PR 正文「一处对 issue 所给形状的补充」那一节 —— contentType 改为可选、以及新增必填 contentForm —— 是对 issue 明文形状的两处偏差。两处都有实测理由(后者由 wire 测试的 charset 用例直接钉住),但它们进的是持久化列的元素形状,属于契约面,请在 review 时明确认可或驳回。


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

2 participants