New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic:rgw: fix chunked-encoding for chunks >1MiB #24363

merged 1 commit into from Oct 17, 2018


None yet
4 participants
Copy link

pdvian commented Oct 2, 2018

Robin H. Johnson Prashant D
rgw: fix chunked-encoding for chunks >1MiB
For HTTP responses sent with chunked-encoding, and greater than 1MiB in
size, the chunk-size field was being printed wrong.

Specifically, the chunk-size field was being sent with a mangled or
missing trailer of '\r\n'.

This bug manifested as HTTP clients being unable to read the response:
Python/boto generates httplib.LineTooLong: got more than 65536 bytes when reading chunk size

The wrong variable was being used to determine the size of the buffer
used for the chunk-size field.

Fix it by using the correct variable, and rename the variables to
clearly reflect their purpose.

Prior to PR#23940, this would only have been seen in some Swift
operations. PR#23940 changed some S3 operations to also use chunked
encoding to get responses sent faster, and made the bug easier to
detect. It was initially reported for a ListBucket call with a high
max-keys argument.

Backport: luminous, mimic
Reference: #23940
Signed-off-by: Robin H. Johnson <>
(cherry picked from commit 3b86448)

@smithfarm smithfarm added this to the mimic milestone Oct 2, 2018

@smithfarm smithfarm requested review from cbodley and yehudasa Oct 2, 2018


cbodley approved these changes Oct 2, 2018


This comment has been minimized.

Copy link

yuriw commented Oct 15, 2018

@yuriw yuriw merged commit 544551f into ceph:mimic Oct 17, 2018

4 checks passed

Docs: build check OK - docs built
Signed-off-by all commits in this PR are signed
Unmodified Submodules submodules for project are unmodified
make check make check succeeded
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment