Skip to content

fix(upload): 加固上传冲突策略与批量处理 - #49

Merged
Cloudwhile merged 5 commits into
masterfrom
feat/issue-29-upload-conflicts
Jul 28, 2026
Merged

fix(upload): 加固上传冲突策略与批量处理#49
Cloudwhile merged 5 commits into
masterfrom
feat/issue-29-upload-conflicts

Conversation

@Cloudwhile

@Cloudwhile Cloudwhile commented Jul 28, 2026

Copy link
Copy Markdown
Owner

变更内容

  • 在上传会话中持久化原始文件名、冲突目标节点与对象键,完成阶段不再按同名文件重新绑定目标。
  • 为 rename/version 完成事务增加 Serializable 隔离与随机退避重试;overwrite 精确校验并清理实际被替换的对象。
  • 支持并发 rename 最终名称回传、完成中断后的幂等恢复、过期冲突快照替换,以及竞态 skip 的 canceled 终态。
  • 批量上传按规范化文件名识别目录内与批次内冲突;同名组串行、不同名组受控并发,并准确展示跳过与最终重命名结果。
  • 上传会话快照迁移采用 expand/contract 的扩展阶段:本次只增加可空列,新代码双写新旧名称字段,并兼容滚动部署期间旧实例写入的 NULL 快照。

审查修复

  • 迁移移除同步全表回填和立即设置 NOT NULL,避免长时间持锁及旧实例写入失败。
  • findReusable 仅在 requested_file_name 为 NULL 时回退匹配旧 file_name,并继续同时执行过期过滤。
  • 重命名冲突查询复用统一生成的 directoryKey,测试中的重试次数复用生产常量。
  • 前端冲突策略复用同一类型定义,上传分组使用默认最多 4 个 worker 的并发上限。
  • Prisma Client 按仓库脚本重新生成后,快照字段均为明确的 string | null 类型,未使用运行时强转掩盖缺失字段。

公共重试模块说明

上传完成与分享下载都需要对 Prisma Serializable 冲突执行同一套 equal-jitter 退避。为避免在上传模块复制 shares 私有实现,本 PR 将该 helper 提取到 common/database,shares 中只调整导入路径,行为与重试参数未改变;这项跨模块改动直接服务于上传冲突的并发完成路径。

验证

  • Backend:79 个测试套件、651 个测试通过
  • Frontend:28 个测试文件、208 个测试通过
  • Backend / Frontend lint:通过(Frontend 仅有 23 条既有 warning)
  • 根级生产构建:通过
  • Playwright Chromium:5/5 通过
  • git diff --check:通过
  • 3 笔审查修复提交均通过 GPG 完整验签

部署说明

本次 PostgreSQL 迁移只增加可空列。待旧实例全部退出后,应在独立的后续 contract 迁移中分批回填、校验数据,再收紧 requested_file_name 的非空约束;本 PR 不提前执行该收缩步骤。

Closes #29

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

新增文件上传冲突策略,覆盖前端冲突规划、后端冲突目标快照、事务写入、版本/覆盖/重命名/跳过处理,以及完成声明取消和审计。

Changes

上传冲突处理

Layer / File(s) Summary
事务重试基础
backend/src/common/database/serializable-transaction-retry.ts, backend/src/modules/shares/*
事务重试支持调用方指定可重试错误,并统一使用公共实现。
上传会话冲突快照
database/schema.prisma, database/migrations/..., backend/src/database/*, backend/src/modules/files/upload-session-*, backend/src/modules/files/upload-sessions.repository.ts
UploadSession 保存请求文件名及冲突目标节点、对象键,并支持迁移、SQLite 启动补丁、复用校验和取消完成声明。
上传完成冲突写入
backend/src/modules/files/file-upload-completion-write.ts, backend/src/modules/files/file-nodes.repository.ts, backend/src/modules/files/*spec.ts
完成写入支持目标校验、版本创建、覆盖、自动重命名、跳过映射及相关重试。
完成生命周期与审计
backend/src/modules/files/file-upload.service.ts, backend/src/modules/files/file-upload-policy.service.ts, backend/src/modules/files/upload-session-completion.ts
完成流程处理会话重放、跳过竞态、对象清理、版本修剪、取消状态和审计事件。
前端冲突规划与反馈
frontend/src/views/drive/*, frontend/src/features/file/*, frontend/src/lib/drive-api-errors.ts, frontend/src/components/ui/*
前端按规范名规划冲突策略,执行批量上传,并处理重命名、完成和跳过状态。

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

Suggested labels: bug, Priority: High

Poem

兔子抱着文件蹦,
重名来了不慌张。
覆盖、重命名、版本存,
跳过也有清晰章。
上传完成月光亮。

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning 修改中包含 shares 仓库的可序列化事务重试导入路径调整,与上传冲突处理目标无直接关联。 若无跨模块重构需求,请移除 shares 相关导入改动,或在 PR 说明中单独标注其与本次上传冲突工作的关系。
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed 实现了同名冲突检测、skip/overwrite/rename/version 策略、批量统一处理、前端结果展示及版本/审计兼容。
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 标题准确概括了本次 PR 的核心:增强上传冲突策略与批量处理流程。
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (4)
backend/src/modules/files/file-nodes.repository.spec.ts (1)

992-992: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

建议改用重试上限常量替代硬编码 5。

serializableTransactionMaxAttempts 调整后本断言会误报。从 ../../common/database/serializable-transaction-retry 导出常量并引用更稳妥。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/src/modules/files/file-nodes.repository.spec.ts` at line 992,
更新相关测试断言,使用从 serializable-transaction-retry 导出的
serializableTransactionMaxAttempts 常量替代 createVersion 调用次数中的硬编码
5,确保重试上限调整后测试仍保持同步。
backend/src/modules/files/file-upload-completion-write.ts (1)

233-243: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

建议复用 requestedStorageKeys.directoryKey,避免二次推导目录键。

这里用 input.parentNodeId ?? '' 手工推导 directoryKey,而唯一索引与写入路径都使用 createFileNodeStorageKeys 的结果。一旦目录键编码规则变化,兄弟节点查询会与唯一约束脱节,导致重命名解析漏判并退化成重试直至报冲突。

♻️ 建议改动
 async function resolveAvailableUploadName(
   tx: Pick<Prisma.TransactionClient, 'fileNode'>,
   input: {
+    directoryKey: string;
     ownerScopeKey: string;
     parentNodeId: string | null;
     requestedFileName: string;
     spaceScope: FileNodeSpaceScope;
     workspaceId: string;
   },
 ) {
   const siblings = await tx.fileNode.findMany({
     where: {
       archivedAt: null,
-      directoryKey: input.parentNodeId ?? '',
+      directoryKey: input.directoryKey,

调用处(Line 131-137)同步传入 directoryKey: requestedStorageKeys.directoryKey

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/src/modules/files/file-upload-completion-write.ts` around lines 233 -
243, 更新兄弟节点查询中的 directoryKey,复用 createFileNodeStorageKeys 生成的
requestedStorageKeys.directoryKey,不要再通过 input.parentNodeId ?? ''
二次推导;同时确保调用该查询的参数传入 requestedStorageKeys.directoryKey,使查询与唯一索引及写入路径使用同一目录键。
frontend/src/views/drive/upload-conflict-planning.ts (2)

3-7: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

UploadConflictResolutionStrategyactions.tsUploadConflictStrategy 重复定义。

两处联合类型字面量完全相同("overwrite" | "rename" | "skip" | "version"),但分别独立声明。由于 TypeScript 对联合类型是结构化比较,未来若一方新增/修改策略而忘记同步另一方,编译期不会报错,只会在运行时产生行为不一致。建议复用同一类型定义。

♻️ 建议的重构
-export type UploadConflictResolutionStrategy =
-  | "overwrite"
-  | "rename"
-  | "skip"
-  | "version";
+import type { UploadConflictStrategy } from "`@/features/file/actions`";
+
+export type UploadConflictResolutionStrategy = UploadConflictStrategy;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/src/views/drive/upload-conflict-planning.ts` around lines 3 - 7,
Remove the duplicate UploadConflictResolutionStrategy union in
upload-conflict-planning.ts and reuse the existing UploadConflictStrategy type
from actions.ts. Update imports and references as needed so both upload conflict
planning and action handling share one canonical strategy definition.

86-99: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

不同名分组缺少并发上限。

runUploadGroupsgroups 中的每个分组都通过 Promise.all 并行启动,组数即并发数。对于选择大量互不同名文件的批量上传场景(如数百个文件),会瞬间发起等量并发请求,可能触发浏览器每主机连接数限制排队、或对后端造成瞬时压力。建议引入并发上限(如 worker-pool 模式)控制同时进行的分组数。

♻️ 建议的重构
 export async function runUploadGroups<T>(
   groups: readonly (readonly T[])[],
   start: (item: T) => Promise<unknown> | void,
+  concurrency = 4,
 ) {
-  await Promise.all(groups.map(async (group) => {
-    for (const item of group) {
-      try {
-        await start(item);
-      } catch {
-        // A failed item must not prevent the remaining items in its canonical-name group.
-      }
-    }
-  }));
+  let cursor = 0;
+  async function worker() {
+    while (cursor < groups.length) {
+      const group = groups[cursor++];
+      for (const item of group) {
+        try {
+          await start(item);
+        } catch {
+          // A failed item must not prevent the remaining items in its canonical-name group.
+        }
+      }
+    }
+  }
+  await Promise.all(
+    Array.from({ length: Math.min(concurrency, groups.length) }, worker),
+  );
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/src/views/drive/upload-conflict-planning.ts` around lines 86 - 99, 为
runUploadGroups 增加分组级并发上限,避免 Promise.all 一次性启动所有分组;使用 worker-pool
或等效调度方式控制同时处理的分组数量,并确保每个分组内仍按顺序处理项目、单个项目失败不会阻止同组后续项目,且所有分组最终完成。
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/src/modules/files/upload-session-types.ts`:
- Around line 102-105: 修正 PrismaUploadSession 及其来源 row 的类型定义,使
requestedFileName、conflictTargetNodeId 和 conflictTargetObjectKey 与 Prisma
返回值保持明确且正确的强类型,消除这些字段赋值处的 no-unsafe-assignment。优先更新 Prisma schema
或生成类型并重新生成客户端,不要通过 String(...) 等运行时转换掩盖 unknown 或缺失值。

In
`@database/migrations/20260728120000_upload_session_conflict_snapshot/migration.sql`:
- Around line 3-13: 将迁移调整为可滚动发布的扩展/收缩流程:本次仅添加可空的
requested_file_name、conflict_target_node_id 和 conflict_target_object_key
列,移除全表回填及立即设置 requested_file_name 为 NOT NULL 的操作;保留后续由兼容代码双写 requested_file_name
的空间,并将分批回填、校验及约束收紧安排到后续迁移。

---

Nitpick comments:
In `@backend/src/modules/files/file-nodes.repository.spec.ts`:
- Line 992: 更新相关测试断言,使用从 serializable-transaction-retry 导出的
serializableTransactionMaxAttempts 常量替代 createVersion 调用次数中的硬编码
5,确保重试上限调整后测试仍保持同步。

In `@backend/src/modules/files/file-upload-completion-write.ts`:
- Around line 233-243: 更新兄弟节点查询中的 directoryKey,复用 createFileNodeStorageKeys 生成的
requestedStorageKeys.directoryKey,不要再通过 input.parentNodeId ?? ''
二次推导;同时确保调用该查询的参数传入 requestedStorageKeys.directoryKey,使查询与唯一索引及写入路径使用同一目录键。

In `@frontend/src/views/drive/upload-conflict-planning.ts`:
- Around line 3-7: Remove the duplicate UploadConflictResolutionStrategy union
in upload-conflict-planning.ts and reuse the existing UploadConflictStrategy
type from actions.ts. Update imports and references as needed so both upload
conflict planning and action handling share one canonical strategy definition.
- Around line 86-99: 为 runUploadGroups 增加分组级并发上限,避免 Promise.all 一次性启动所有分组;使用
worker-pool 或等效调度方式控制同时处理的分组数量,并确保每个分组内仍按顺序处理项目、单个项目失败不会阻止同组后续项目,且所有分组最终完成。
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 293ef0d7-3ca1-4356-88d1-fec04e5cd689

📥 Commits

Reviewing files that changed from the base of the PR and between 214c842 and dba7d4a.

📒 Files selected for processing (30)
  • backend/src/common/database/serializable-transaction-retry.spec.ts
  • backend/src/common/database/serializable-transaction-retry.ts
  • backend/src/database/prisma.service.spec.ts
  • backend/src/database/prisma.service.ts
  • backend/src/modules/files/file-nodes.repository.spec.ts
  • backend/src/modules/files/file-nodes.repository.ts
  • backend/src/modules/files/file-upload-completion-write.ts
  • backend/src/modules/files/file-upload-completion.service.spec.ts
  • backend/src/modules/files/file-upload-policy.service.spec.ts
  • backend/src/modules/files/file-upload-policy.service.ts
  • backend/src/modules/files/file-upload-test-repository.helper.ts
  • backend/src/modules/files/file-upload-test-sessions.helper.ts
  • backend/src/modules/files/file-upload.service.ts
  • backend/src/modules/files/upload-session-completion.repository.spec.ts
  • backend/src/modules/files/upload-session-completion.ts
  • backend/src/modules/files/upload-session-test-fixtures.ts
  • backend/src/modules/files/upload-session-types.ts
  • backend/src/modules/files/upload-sessions.repository.ts
  • backend/src/modules/shares/share-download-commit.repository.ts
  • backend/src/modules/shares/shares.repository.ts
  • database/migrations/20260728120000_upload_session_conflict_snapshot/migration.sql
  • database/schema.prisma
  • frontend/src/components/ui/upload-conflict-dialog.test.tsx
  • frontend/src/features/file/actions.test.ts
  • frontend/src/features/file/actions.ts
  • frontend/src/lib/drive-api-errors.test.ts
  • frontend/src/lib/drive-api-errors.ts
  • frontend/src/views/drive/upload-conflict-planning.test.ts
  • frontend/src/views/drive/upload-conflict-planning.ts
  • frontend/src/views/drive/use-drive-transfers.ts

Comment thread backend/src/modules/files/upload-session-types.ts Outdated
Comment thread database/migrations/20260728120000_upload_session_conflict_snapshot/migration.sql Outdated

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Cloudwhile
Cloudwhile merged commit a18d7fc into master Jul 28, 2026
3 checks passed
@Cloudwhile
Cloudwhile deleted the feat/issue-29-upload-conflicts branch July 28, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 增加文件上传冲突处理策略

1 participant