fix(s3): only skip direct redirect for true sub-resource queries#2604
Conversation
The direct-redirect path treated any non-allowlisted query parameter as a sub-resource request and fell back to server-side proxying. S3 clients that append `response-content-disposition` (and other `response-*` overrides) to a presigned GET were therefore always proxied instead of redirected to the upstream direct link. Replace the allowlist with a blocklist of the query keys that actually route a GET to a gofakes3 sub-resource handler (uploadId, uploads, versioning, versions, location, versionId), so plain object downloads carrying response-* overrides are redirected as 302 again. Also align the redirect responses with the native download handler: set Referrer-Policy: no-referrer on both the download and upload redirects to keep signed upstream URLs out of the Referer header, add the no-store Cache-Control to the download redirect, and resolve the client IP via utils.ClientIP so X-Forwarded-For is honored behind a reverse proxy. Relates to OpenListTeam#2598 Co-Authored-By: Claude <noreply@anthropic.com>
|
LGTM overall. Before merge, I’d prefer adding a small table test for hasNonObjectQuery covering response-content-disposition, x-id=GetObject, uploads/uploadId/versioning/versions/location, and versionId null/non-null cases. One thing to double-check: switching from RemoteAddr to utils.ClientIP means this path now trusts forwarded IP headers. If utils.ClientIP is not limited to trusted proxies, clients may spoof IP-bound direct-link generation in non-proxied deployments. |
Add table cases for response-content-disposition/type, x-id, the uploads/uploadId/versioning/versions/location sub-resources, and versionId null/non-null/empty, per review on OpenListTeam#2604. Correct the stale list-type expectation: with an object segment present gofakes3 routes to getObject, so list-type must not block a direct redirect. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Thanks for the review! 1. Table test — Done in f002a6d. Extended One note: I changed the existing 2. IP trust — Good catch, and you're right about the mechanism. A couple of points for context:
If you'd prefer a trusted-proxy gate, I think that's better addressed globally (a shared |
Summary / 摘要
Follow-up fix for #2598 (
feat(s3): support direct transfer redirects).After #2598 shipped, S3
GET Objectrequests that carried standardresponse-*override parameters (most commonlyresponse-content-disposition,which clients add to set the download filename) were incorrectly forced through
the gofakes3 server-side streaming path instead of being redirected to the
upstream direct link.
Root cause:
hasNonObjectQueryused an allow-list — it let through only theSigV4 signing parameters (
x-amz-*,awsaccesskeyid,signature,expires,x-id) and treated every other query parameter as a "non-object" request,disabling the redirect. Any client that appended
response-content-disposition(or any other unanticipated parameter) silently lost direct-download.
This affects every direct-link-capable driver, not a specific one: the
allow-list runs before storage/driver selection, so the regression is generic.
Changes
hasNonObjectQueryfrom an allow-list to a block-list: only thequery keys that make gofakes3 route a request to a sub-resource handler
(
uploadId,uploads,versioning,versions,location, and a non-nullversionId) disable the direct redirect. This list is derived directly fromgofakes3
routing.gorouteBase/routeBucket, soresponse-*and all otherdownload-irrelevant parameters now correctly allow a
302.handles.redirectpath by settingReferrer-Policy: no-referrer(avoid leaking the signed upstream URL viaReferer) and
Cache-Control: max-age=0, no-cache, no-store, must-revalidate(avoid caching short-lived signed links). Also set
Referrer-Policyon theupload redirect branch for consistency.
utils.ClientIP(r)(X-Forwarded-For / X-Real-Ip aware) insteadof a local
RemoteAddr-only helper, so drivers that bind direct links to theclient IP receive the real client IP when running behind a reverse proxy.
No public API, config, or storage format changes. Upload-path logic from #2598
is untouched.
/ 此 PR 包含破坏性变更。
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
/ 此 PR 需要关联仓库同步修改。
Related repository PRs / 关联仓库 PR:
Related Issues / 关联 Issue
Relates to #2598
Testing / 测试
go build ./...gofmtclean on changed fileBuilt from source with S3 enabled on port 5246 behind an openresty reverse
proxy, and exercised three direct-link-capable drivers via signed SigV4
presigned
GETrequests. Each was tested both with and withoutresponse-content-disposition.response-content-disposition)302direct302direct302direct302direct302direct302directBefore the fix, the
response-content-dispositionvariant returned304/200(server-side proxy); after the fix it returns
302to the upstream direct link.Verified the new response headers are present on every
302:Verified
X-Forwarded-Forhandling: requests carryingX-Forwarded-For: 1.2.3.4still redirect correctly, confirming the client-IPchange does not break the link flow.
Verified the block-list against gofakes3
routing.go: sub-resource requests(
?uploads,?uploadId=,?versioning,?versions,?location,?versionId=) still fall back to the gofakes3 server-side path; plain objectdownloads (including
?x-id=GetObjectand?response-content-disposition=...)are redirected.
Checklist / 检查清单
/ 我已阅读 CONTRIBUTING。
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
gofmt,go fmt, orprettierwhere applicable./ 我已按适用情况使用
gofmt、go fmt或prettier格式化变更代码。/ 我已在适用情况下请求相关维护者或代码所有者审查。
AI Disclosure / AI 使用声明
/ 此 PR 包含 AI 辅助内容。
Tools used / 使用工具:
Usage scope / 使用范围:
Code generation / 代码生成
Refactoring / 重构
Documentation / 文档
Tests / 测试
Translation / 翻译
Review assistance / 审查辅助
I have reviewed and validated all AI-assisted content included in this PR.
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。
I have ensured that all AI-assisted commits include
Co-Authored-Byattribution./ 我已确保所有 AI 辅助提交都包含
Co-Authored-By归属信息。I can reproduce all AI-assisted content included in this PR without any AI tools.
/ 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。