Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: Fix a bug that multipart upload may exceed the quota. #12010

Merged
merged 1 commit into from Jul 13, 2017

Conversation

Projects
None yet
5 participants
@zhangsw
Copy link
Contributor

zhangsw commented Nov 16, 2016

Multipart upload has immutable head. After we have uploaded a part, we invoke write_meta to write the head obj of this part while the head obj actually has been written at the beginning of uploading. The orig_exists is always true and orig_size is the size of the first chunk. But these information is not in quota cache. So we update the quota stats with incorrect data.

Fixes: http://tracker.ceph.com/issues/19602

Signed-off-by: Zhang Shaowen zhangshaowen@cmss.chinamobile.com

@cbodley cbodley added the rgw label Nov 16, 2016

@mattbenjamin mattbenjamin self-assigned this Dec 9, 2016

@yehudasa yehudasa added the bug fix label Mar 23, 2017

rgw: Fix a bug that multipart upload may exceed the quota because of
improperly operation to the cache.

Fixes: http://tracker.ceph.com/issues/19602

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

@zhangsw zhangsw force-pushed the zhangsw:fix-rgw-multipart-bug branch from 06646d0 to ffe95d3 Apr 13, 2017

@mattbenjamin

This comment has been minimized.

Copy link
Contributor

mattbenjamin commented Jun 6, 2017

@zhangsw will schedule qa run; @yehudasa also feels that this is not ideal way to identify the final obj

@mattbenjamin

This comment has been minimized.

Copy link
Contributor

mattbenjamin commented Jun 7, 2017

@zhangsw, @yehudasa one run of the radosgw_admin.py gc-list testcase failed; since this depends on a multipart upload and subsequent gc and gc failed, this might indicate an unexpected failure in the upload step

# TESTCASE 'gc-list', 'gc', 'list', 'get list of objects ready for garbage collection'

# create an object large enough to be split into multiple parts
test_string = 'foo'*10000000

big_key = boto.s3.key.Key(bucket)
big_key.set_contents_from_string(test_string)

# now delete the head
big_key.delete()

# wait a bit to give the garbage collector time to cycle
time.sleep(15)

(err, out) = rgwadmin(ctx, client, ['gc', 'list'])

assert len(out) > 0
@yuriw

This comment has been minimized.

@yehudasa

This comment has been minimized.

Copy link
Member

yehudasa commented Jul 13, 2017

@mattbenjamin @zhangsw this passed qa, so I assume earlier report by matt was a different issue so I'll merge it. Not extermely happy about this solution, but we can deal with it some other time.

@yehudasa yehudasa merged commit df7fbbf into ceph:master Jul 13, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.