Skip to content

Conversation

@dnslin
Copy link

@dnslin dnslin commented Dec 29, 2025

Description / 描述

Implement a new multipart upload API that supports client-side chunked uploads with concurrent upload capability.

实现新的分片上传接口,支持客户端分片并发上传。

API Endpoint: PUT /api/fs/multipart?action=upload|complete

Features / 功能:

  • Uses File-Path header (consistent with /api/fs/put and /api/fs/form)
  • Two-step workflow: upload chunks → complete
  • Auto-initialize session on first chunk upload
  • Support concurrent chunk uploads
  • Idempotent chunk upload (re-uploading same chunk won't fail)
  • Support Overwrite and As-Task headers
  • Session auto-cleanup after expiration (2 hours)

Request Headers:

Header Required Description
File-Path Yes File path (URL encoded)
X-File-Size First chunk Total file size
X-Chunk-Size First chunk Chunk size
X-Chunk-Index Yes Chunk index (0-based)
X-Upload-Id After first Upload session ID
Overwrite No Overwrite existing file (default: true)
As-Task No Async upload on complete (default: false)

Motivation and Context / 背景

The existing upload APIs (/api/fs/put, /api/fs/form) don't support resumable uploads for large files. This new multipart upload API allows clients to:

  • Split large files into chunks and upload them concurrently
  • Resume interrupted uploads by re-uploading missing chunks
  • Better handle unstable network conditions

现有的上传接口不支持大文件的断点续传。新的分片上传接口允许客户端:

  • 将大文件分片并发上传
  • 通过重新上传缺失的分片来恢复中断的上传
  • 更好地处理不稳定的网络环境

Relates to #XXXX (如果有相关 issue 请填写)

How Has This Been Tested? / 测试

  • Manual testing with Python async client
  • Tested concurrent chunk uploads (4 parallel uploads)
  • Tested idempotent chunk upload (re-upload same chunk)
  • Tested session expiration cleanup
  • Tested complete upload with As-Task=true
  • Compiled successfully with go build

使用 Python 异步客户端进行手动测试,测试了并发上传、幂等性、会话过期清理等功能。

Checklist / 检查清单

Copilot AI review requested due to automatic review settings December 29, 2025 08:10
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a new client-side multipart upload API that enables chunked file uploads with concurrent upload support and resumable capabilities. The implementation provides a two-step workflow (upload chunks → complete) with automatic session initialization and cleanup.

Key Changes:

  • New PUT /api/fs/multipart endpoint supporting upload and complete actions
  • Session-based chunk management with 2-hour expiration
  • Idempotent chunk uploads allowing retry of failed chunks
  • Support for concurrent chunk uploads via shared session state

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.

File Description
server/router.go Adds the new /multipart endpoint with upload rate limiting and authentication middleware
server/handles/multipart.go Implements HTTP handlers for chunk upload and completion actions with header-based API design
internal/fs/multipart.go Core multipart session manager with chunk storage, merging, and cleanup logic
internal/model/multipart.go Data structures for multipart sessions, chunk metadata, and API responses

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dnslin and others added 4 commits December 29, 2025 16:24
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
@dnslin
Copy link
Author

dnslin commented Dec 29, 2025

@copilot open a new pull request to apply changes based on the comments in this thread

Copy link
Member

@KirCute KirCute left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

能不能开始上传的时候就构建FileStream并调用驱动的上传,这样做至少有两个好处:

  1. 可以在首个分片的上传请求里携带文件哈希,在驱动触发秒传,如果秒传成功,后续分片就都不用上传了
  2. 可以同时进行用户终端->OpenListOpenList->网盘两个上传过程,这两个过程分别吃服务器的下行和上行带宽,不冲突,可以很大程度上减少总的上传时间

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants