Skip to content

fix: Preserve SigV4 range through dfdaemon proxy#1820

Open
YQ-Wang wants to merge 2 commits intodragonflyoss:mainfrom
YQ-Wang:fix/proxy-preserve-sigv4-signed-range
Open

fix: Preserve SigV4 range through dfdaemon proxy#1820
YQ-Wang wants to merge 2 commits intodragonflyoss:mainfrom
YQ-Wang:fix/proxy-preserve-sigv4-signed-range

Conversation

@YQ-Wang
Copy link
Copy Markdown
Contributor

@YQ-Wang YQ-Wang commented Apr 29, 2026

Description

This fixes s5cmd multipart S3 downloads through dfdaemon's HTTP proxy when the request signs the Range header with AWS SigV4.

Changes include:

  • Detect SigV4 requests where Range is included in SignedHeaders, including both Authorization header and presigned URL forms.
  • Preserve the original signed Range header for the source request instead of rewriting it to Dragonfly piece ranges.
  • Include the Range value in task ID input for signed ranged requests, preventing cache collisions between multipart parts for the same object.
  • Use the full object size from Content-Range: bytes N-M/Total during stat handling so non-zero range parts validate correctly.
  • Add focused unit tests for signed range detection, backend range preservation, legacy range rewriting, and Content-Range total parsing.

Related Issue

#1821

Motivation and Context

s5cmd uses multipart ranged GetObject requests for downloads. With AWS SigV4, those requests include range in SignedHeaders. dfdaemon's proxy normally rewrites Range when fetching pieces from the source, which changes the canonical request and causes S3 to reject the request with 403 SignatureDoesNotMatch.

This change keeps the fix narrow: it only preserves Range when the request's authentication material explicitly signs it. Other proxy downloads continue through the existing piece-splitting path.

Tests

  • cargo fmt --all -- --check
  • cargo test -p dragonfly-client-backend --lib http::tests::
  • cargo test -p dragonfly-client --lib proxy::header::
  • cargo test -p dragonfly-client --lib resource::task::
  • cargo clippy -p dragonfly-client-backend --all-targets -- -D warnings
  • Local docker-compose smoke test with patched dfdaemon + real s5cmd multipart download through the proxy against AWS S3. The proxied file matched direct download by md5, and repeat download hit local cache.

Signed-off-by: Yiqing Wang <yiqingwang@roblox.com>
@YQ-Wang YQ-Wang requested review from a team as code owners April 29, 2026 17:53
@YQ-Wang YQ-Wang changed the title Fix SigV4 ranged downloads through dfdaemon proxy Fix: Preserve SigV4 range through dfdaemon proxy Apr 29, 2026
@YQ-Wang YQ-Wang changed the title Fix: Preserve SigV4 range through dfdaemon proxy fix: Preserve SigV4 range through dfdaemon proxy Apr 29, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 87.33333% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.28%. Comparing base (cdb702e) to head (888b2aa).

Files with missing lines Patch % Lines
dragonfly-client/src/resource/task.rs 66.66% 22 Missing ⚠️
dragonfly-client/src/resource/piece.rs 0.00% 17 Missing ⚠️
dragonfly-client/src/grpc/dfdaemon_download.rs 0.00% 9 Missing ⚠️
dragonfly-client-backend/src/http.rs 97.57% 5 Missing ⚠️
dragonfly-client/src/proxy/header.rs 97.36% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1820      +/-   ##
==========================================
+ Coverage   43.58%   44.28%   +0.69%     
==========================================
  Files          92       92              
  Lines       26929    27376     +447     
==========================================
+ Hits        11738    12124     +386     
- Misses      15191    15252      +61     
Files with missing lines Coverage Δ
dragonfly-client/src/proxy/header.rs 90.04% <97.36%> (+4.12%) ⬆️
dragonfly-client-backend/src/http.rs 96.38% <97.57%> (+0.28%) ⬆️
dragonfly-client/src/grpc/dfdaemon_download.rs 4.73% <0.00%> (-0.06%) ⬇️
dragonfly-client/src/resource/piece.rs 37.59% <0.00%> (-1.27%) ⬇️
dragonfly-client/src/resource/task.rs 11.48% <66.66%> (+5.43%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Yiqing Wang <yiqingwang@roblox.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant