New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: allow buffers to be accounted in arbitrary mempools #15352

Merged
merged 8 commits into from May 31, 2017

Conversation

Projects
None yet
2 participants
@liewegas
Member

liewegas commented May 29, 2017

No description provided.

@liewegas

This comment has been minimized.

Member

liewegas commented May 31, 2017

http://pulpito.ceph.com/sage-2017-05-30_17:22:40-rados-wip-sage-testing---basic-smithi/
but patches were revised a bit, should go through another run

liewegas added some commits May 5, 2017

mempool: kill unittest_? mempools
These were just annoying.  Use some real ones that will be empty
for the unit tests.

Signed-off-by: Sage Weil <sage@redhat.com>
buffer: fix mempool accounting for claimed chars
These are allocated by the caller, outside of the mempool; do not
deallocate via the mempool.

Signed-off-by: Sage Weil <sage@redhat.com>
mempool: allow pool counts to be adjusted explicitly
Signed-off-by: Sage Weil <sage@redhat.com>
buffer: allow buffers to map into arbitrary mempools
- default is buffer_anon
- buffer_data includes *all* buffers across all mempools, and is the
*actual* allocation (includes raw_combined).
- buffer_meta is other buffer-related overhead.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: mark buffer caches buffers as bluestore_data
Signed-off-by: Sage Weil <sage@redhat.com>
osd: (loosely) mark cached map bl buffers as osd_mapbl
Signed-off-by: Sage Weil <sage@redhat.com>
mempool: dump total, too
Signed-off-by: Sage Weil <sage@redhat.com>
buffer: remove buffer_data mempool
We are explicitly accounting for buffers via arbitrary pools, defaulting
to buffer_anon, so we don't need buffer_data anymore.  It leads to
double-counting.

Signed-off-by: Sage Weil <sage@redhat.com>
@markhpc

This comment has been minimized.

Member

markhpc commented May 31, 2017

So far this PR is working well for me. In testing the 4k min_alloc size memory growth I'm seeing the majority of it appears to be coming from bluestore_data.

@liewegas liewegas merged commit cbc5a34 into ceph:master May 31, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@liewegas liewegas deleted the liewegas:wip-buffer-mempools branch May 31, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment