Skip to content

fix(s3): recover multipart uploads after request timeout - #7581

Merged
c121914yu merged 2 commits into
labring:mainfrom
xqvvu:fix/multipart-upload-timeout-retry
Aug 24, 2026
Merged

fix(s3): recover multipart uploads after request timeout#7581
c121914yu merged 2 commits into
labring:mainfrom
xqvvu:fix/multipart-upload-timeout-retry

Conversation

@xqvvu

@xqvvu xqvvu commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • propagate client disconnects from the multipart proxy to storage provider part uploads
  • retry multipart completion after client/network timeouts and 409 completing responses
  • add regression coverage for request-close cancellation and completion recovery

Validation

  • pnpm --dir projects/app exec vitest run -c vitest.config.ts test/service/common/file/utils.test.ts test/service/common/s3/proxy.test.ts test/api/system/file/accessLink.test.ts --coverage.enabled=false
  • pnpm --dir packages/service exec vitest run --config vitest.config.ts test/common/s3/buckets/base.test.ts --coverage.enabled=false
  • pnpm --dir sdk/storage exec vitest run --config vitest.config.ts test/unit/adapters/aws-s3.adapter.test.ts test/unit/adapters/oss.adapter.test.ts test/unit/adapters/cos.adapter.test.ts --coverage.enabled=false
  • pnpm --dir projects/app typecheck
  • pnpm --dir sdk/storage typecheck:test

Existing pro submodule worktree change is intentionally excluded.

@github-actions

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 28.38% 2666 / 9393
🔵 Statements 28.31% 2808 / 9917
🔵 Functions 23.2% 522 / 2250
🔵 Branches 23.29% 1263 / 5422
File CoverageNo changed files found.
Generated in workflow #2491 for commit ec1c01c by the Vitest Coverage Report Action

@github-actions

Copy link
Copy Markdown

Admin Preview Image Ready!

ghcr.io/labring/fastgpt-pr:admin_ec1c01c4745795d14dad65ec8a469b9b3623c611

🕒 Time: 2026-08-24 14:48:20 (UTC+8)

@github-actions

Copy link
Copy Markdown

Build and publish successful - Preview fastgpt Image:

ghcr.io/labring/fastgpt-pr:fastgpt_ec1c01c4745795d14dad65ec8a469b9b3623c611

🕒 Time: 2026-08-24 14:48:35 (UTC+8)

@c121914yu c121914yu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这次修复的方向是对的:代理层已经能把客户端断开转换成 AbortSignal,complete 也开始识别网络超时和 409 completing。不过当前实现还有两个会影响需求闭环的问题,建议合并前确认:

  1. complete 重试窗口与服务端 lease 不匹配

    packages/web/common/file/uploader/multipart.ts:102-109 复用了分片的 maxRetry。默认 maxRetry=3、base delay 300ms 时,总等待仅为 300+600+1200=2100ms;服务端 S3_MULTIPART_COMPLETING_LEASE_MS 却是 5 分钟。若已经超时 120 秒的原 complete 请求没有在随后 2.1 秒内完成,客户端仍会失败,并且无法等到 lease reclaim。当前新增测试只覆盖“一次 409 后立即成功”。

    建议将 complete 恢复策略与“单分片最大重试次数”解耦,例如增加状态感知轮询、Retry-After 或独立的 completion retry/timeout 策略,并补充连续 409、原请求晚完成和 lease reclaim 的时间边界测试。

  2. OSS/COS 没有完整取消 provider 请求

    sdk/storage/src/adapters/oss.adapter.ts:195-206cos.adapter.ts:248-268 只把 signal 绑定到请求体 Readable。流 close 后 bindAbortSignalToReadable 会移除监听;当 body 已经发送完、SDK 仍在等待 provider 响应时,客户端断开不会取消底层 HTTP 请求。Buffer/string body 也只支持发起前取消。AWS adapter 使用 SDK 原生 abortSignal,三家行为因此不一致。

    现有 proxy 测试 mock 了 bucket,并未经过真实 adapter;三份 adapter 测试也没有传入 upload abortSignal。建议按厂商能力实现请求级取消,并至少增加“body 已结束、provider 仍 pending 时 abort”的 adapter 测试。

另外,这个 PR 还包含独立的 ChatInput 最小高度修复,以及 AWS endpoint 错误文案、OSS/COS marker/header 类型清理。它们和 multipart timeout 没有直接依赖,建议拆到独立 PR,降低回归面。

本地聚焦验证:App 41、Service 32、Storage 59 个测试全部通过,App/Storage typecheck 和 git diff --check 通过。上述问题属于当前测试没有覆盖的时序和厂商 SDK 边界。

@c121914yu
c121914yu merged commit 1e9e794 into labring:main Aug 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants