Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: rgw: mitigate bucket list with max-entries excessively high #30134

Merged
merged 2 commits into from Oct 8, 2019

Conversation

@smithfarm smithfarm added this to the mimic milestone Sep 4, 2019
@smithfarm smithfarm added the rgw label Sep 4, 2019
@smithfarm smithfarm requested review from ivancich and cbodley Sep 4, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Sep 8, 2019

test this please

@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Sep 9, 2019

Added DNM because (if I understand https://tracker.ceph.com/issues/41655#note-1 correctly) this caused https://tracker.ceph.com/issues/41655 - once the fix for that merges, I'll cherry-pick it into this PR.

ivancich added 2 commits Jul 19, 2019
When listing a bucket with radosgw-admin, the user can specify the
maximum number of entries. That number can be unreasonably large, and
can affect the performance and memory availability. For example:

    radosgw-admin bucket list --bucket mybucket1 --max-entries=10000000

This has the potential for creating large data structures at multiple
levels in the the call stack of the radosgw(-admin) process,
potentially causing the process to run out of memory. This change
limits the maximum number of entries requested in all but the high
level code to help mitigate this issue.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 300429c)

Conflicts:
	src/rgw/rgw_rados.cc
- RGWRados::Bucket::List::list_objects_ordered takes an additional argument
  (unrelated to this change) in master
A recent PR made sure that a bucket listing could not request too many
entries at once. It also did a minimum computation for number of
entries. For ordered listing the minimum was 0, as required to pass
all unit tests. However the minimum for unordered listing was left at
1. In order to make ordered and unordered listing behave the same --
with the exception of ordering -- the minimum for unordered listing is
modified to 0.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(manually cherry picked from 38e02a7)

Conflicts:
     src/srgw/rgw_rados.cc
- automated cherry-pick from master failed strangely: made the changes manually
@smithfarm smithfarm force-pushed the smithfarm:wip-41103-mimic branch from 40c56cc to 934d4db Oct 1, 2019
@smithfarm smithfarm removed the DNM label Oct 1, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Oct 1, 2019

@ivancich Please have another look?

Copy link
Member

ivancich left a comment

Yep, this looks good!

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Oct 3, 2019

@yuriw yuriw merged commit ddfe6fe into ceph:mimic Oct 8, 2019
4 checks passed
4 checks passed
Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
@smithfarm smithfarm deleted the smithfarm:wip-41103-mimic branch Oct 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.