Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle bytes=0- correctly in objectnode #1286

Merged
merged 1 commit into from
Dec 23, 2021

Conversation

bndbsh
Copy link
Contributor

@bndbsh bndbsh commented Dec 13, 2021

Some clients (e.g. docker registry) send a get to fetch the whole file
by specifying bytes=0- as the range. In this case, ChubaoFS will set
the range to 0-0 and the Content-Length to 1 (upper - lower + 1). It
will then error out when writing the object, since it will write more
than the specified Content-Length.

As a simple fix, we can treat a rangeUpper of 0 the same as max, which
should result in correct behavior.

Special notes for your reviewer: I'm not able to build chubaofs locally (errors while building rocksdb), so I haven't been able to test this change.

Release note:

objectnode: handle `Range: bytes=0-` correctly

Some clients (e.g. docker registry) send a get to fetch the whole file
by specifying `bytes=0-` as the range. In this case, ChubaoFS will set
the range to 0-0 and the `Content-Length` to 1 (upper - lower + 1). It
will then error out when writing the object, since it will write more
than the specified `Content-Length`.

As a simple fix, we can treat a rangeUpper of 0 the same as max, which
should result in correct behavior.

Signed-off-by: Belhorma Bendebiche <amro@bndb.sh>
@bndbsh
Copy link
Contributor Author

bndbsh commented Dec 13, 2021

Here's a curl showcasing the issue:

curl -H "Authorization: <redacted>" http://<redacted>/registry/docker/registry/v2/repositories/csi/_uploads/7b6cd0da-c3d4-4f58-a765-b32f1195b64b/startedat -H "User-Agent: aws-sdk-go/1.15.11 (go1.11.2; linux; amd64)" -H "Range: bytes=0-" -H "Date: Mon, 13 Dec 2021 23:02:12 UTC" -v
* Connected to <redacted> (#0)
> GET /registry/docker/registry/v2/repositories/csi/_uploads/7b6cd0da-c3d4-4f58-a765-b32f1195b64b/startedat HTTP/1.1
> Host: <redacted>
> Accept: */*
> Authorization: <redacted>
> User-Agent: aws-sdk-go/1.15.11 (go1.11.2; linux; amd64)
> Range: bytes=0-
> Date: Mon, 13 Dec 2021 23:02:12 UTC
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 1
< Content-Range: bytes 0-0/20
< Content-Type: application/octet-stream
< ETag: "abca19b27a83646832ef00ce6e5f3083"
< Last-Modified: Mon, 13 Dec 2021 23:02:12 GMT
< Server: ChubaoFS
< x-amz-request-id: 1503ed88eb0e47beb6f999de9faf1b4d
< Date: Mon, 13 Dec 2021 23:05:04 GMT
<
* transfer closed with 1 bytes remaining to read
* Closing connection 0
curl: (18) transfer closed with 1 bytes remaining to read

Specifically:

< Content-Length: 1
< Content-Range: bytes 0-0/20

@shuoranliu
Copy link
Member

@bndbsh Thanks for submitting this pull request! Looks good to me, and we'll have a sanity test.

BTW, about the build issue, have you tried run "make build" under the chubaofs root directory? If there is still error, would you mind opening another issue so we can try to fix it? Thanks in advance!

@bndbsh
Copy link
Contributor Author

bndbsh commented Dec 16, 2021

I think the build issue is because I have rocksdb 6.x and my environment can't build the bundle 5.x version of rocksdb.

@shuoranliu shuoranliu merged commit 972aa4c into cubefs:master Dec 23, 2021
@shuoranliu
Copy link
Member

@bndbsh please take a look at this PR #1290 And would you mind try building the latest master branch? Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants