Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: Fix a bug that multipart upload may exceed the quota. #12010

merged 1 commit into from Jul 13, 2017


None yet
5 participants
Copy link

zhangsw commented Nov 16, 2016

Multipart upload has immutable head. After we have uploaded a part, we invoke write_meta to write the head obj of this part while the head obj actually has been written at the beginning of uploading. The orig_exists is always true and orig_size is the size of the first chunk. But these information is not in quota cache. So we update the quota stats with incorrect data.


Signed-off-by: Zhang Shaowen

@cbodley cbodley added the rgw label Nov 16, 2016

@mattbenjamin mattbenjamin self-assigned this Dec 9, 2016

@yehudasa yehudasa added the bug fix label Mar 23, 2017

rgw: Fix a bug that multipart upload may exceed the quota because of
improperly operation to the cache.


Signed-off-by: Zhang Shaowen <>

@zhangsw zhangsw force-pushed the zhangsw:fix-rgw-multipart-bug branch from 06646d0 to ffe95d3 Apr 13, 2017


This comment has been minimized.

Copy link

mattbenjamin commented Jun 6, 2017

@zhangsw will schedule qa run; @yehudasa also feels that this is not ideal way to identify the final obj


This comment has been minimized.

Copy link

mattbenjamin commented Jun 7, 2017

@zhangsw, @yehudasa one run of the gc-list testcase failed; since this depends on a multipart upload and subsequent gc and gc failed, this might indicate an unexpected failure in the upload step

# TESTCASE 'gc-list', 'gc', 'list', 'get list of objects ready for garbage collection'

# create an object large enough to be split into multiple parts
test_string = 'foo'*10000000

big_key = boto.s3.key.Key(bucket)

# now delete the head

# wait a bit to give the garbage collector time to cycle

(err, out) = rgwadmin(ctx, client, ['gc', 'list'])

assert len(out) > 0

This comment has been minimized.


This comment has been minimized.

Copy link

yehudasa commented Jul 13, 2017

@mattbenjamin @zhangsw this passed qa, so I assume earlier report by matt was a different issue so I'll merge it. Not extermely happy about this solution, but we can deal with it some other time.

@yehudasa yehudasa merged commit df7fbbf into ceph:master Jul 13, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Unmodifed Submodules submodules for project are unmodified
default Build finished.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.