New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix S3 v3 Range handling #9082
fix S3 v3 Range handling #9082
Conversation
Just wanted to strengthen the above argument by drawing attention to #9076 (comment). The rfc gives servers the latitude to ignore an invalid range request:
|
Thank you @sjperkins, I'll update the PR description with the quote. Very appreciated! |
1910bed
to
20c58e5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Motivation
As reported in #9076, there is an issue in the way Moto and LocalStack handles HTTP Range requests for S3. AWS is apparently not following the RFC for range requests and is very, very forgiving, as shown by the added test.
As pointed out by @sjperkins who went through the RFC, the server can ignore the range header for some conditions:
Basically, if the range request is badly formatted or invalid, S3 will just treat the request as a non-range request and will return the entire object with a
200
status code.Only certain conditions will trigger an exception and
416
, mainly if the beginning of the request is above the end of the object and the request is a validRange
request (meaning it can be parsed), or if the request is a "suffix" request and it ends with 0.Changes
This fixes the behaviour only for the v3 provider for now, I'll need to open a PR upstream in moto to fix the behaviour there to fix the default current provider. Edit: the PR upstream is already merged, so I'd like to have the v3 behaviour in line.
I might remove all the
skip
for the new provider tests and selectively xfail the tests for the default provider in a follow up PR, to see the extent of the issue with the current provider, and try to have some tests such as this one valid for both.I've launched a run with this fix in the forced v3 S3 branch here, and it's green: https://app.circleci.com/pipelines/github/localstack/localstack/17924/workflows/3026ab82-afaa-4b31-9b45-fbd7afbab1bd
This also fix the behaviour for
UploadPartCopy
which has a very strict handling of theCopySourceRange
parameter.