New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: core: os/bluestore: cache autotuning and memory limit #25283

Merged
merged 8 commits into from Dec 10, 2018

Conversation

Projects
None yet
5 participants
@jdurgin
Copy link
Member

jdurgin commented Nov 27, 2018

This is a mimic backport of:

#22009
#22838
#22970
#25069

These make bluestore much simpler to configure, since users no longer need to worry about bluestore's cache size or internal details of what is allocated where.

Instead, the osd_memory_target tunable is all that is needed. Since bluestore cannot use the kernel's page cache, the default osd memory target is 4GB. This has no effect on filestore.

http://tracker.ceph.com/issues/37340

markhpc and others added some commits Apr 26, 2018

common/PriorityCache: First pass at prototype.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit cb46541)

Conflicts:
	src/os/bluestore/BlueStore.cc (trivial)
os/bluestore: Fix potential uint64_t to int conversion issue.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1d51e76)
src/kv: Initial import of a custom RocksDB cache.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9dd6b28)

Conflicts:
	src/kv/RocksDBStore.cc
trivial - g_conf has no operator()
kv/rocksdb_cache/BinnedLRUCache: Don't promote data to the high pri p…
…ool.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 269713d)
kv/RocksDBStore: Handle nullptr if clock cache is chosen.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9202285)

Conflicts:
	src/kv/RocksDBStore.cc

trivial - g_conf operator()
os/bluestore: limit OSD memory usage by tuning the cache size.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1b8a87b)

Conflicts:
	src/common/options.cc
	src/os/bluestore/BlueStore.cc
trivial - no g_conf operator. or option TYPE_SIZE
doc/rados: Add bluestore memory autotuning docs.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9e2d1a5)

Conflicts:
	doc/rados/configuration/bluestore-config-ref.rst

Deployment docs updated in master, pulled in those here too
PendingReleaseNotes: add note about osd memory target and bluestore c…
…ache

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

@jdurgin jdurgin added the bluestore label Nov 27, 2018

@jdurgin jdurgin added this to the mimic milestone Nov 27, 2018

@jdurgin jdurgin requested a review from liewegas Nov 27, 2018

@jdurgin

This comment has been minimized.

Copy link
Member

jdurgin commented Nov 27, 2018

jenkins render docs

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Nov 27, 2018

@ceph-jenkins

This comment has been minimized.

Copy link
Collaborator

ceph-jenkins commented Nov 27, 2018

Doc render available at http://docs.ceph.com/ceph-prs/25283/

@jdurgin jdurgin merged commit 3c4b142 into ceph:mimic Dec 10, 2018

4 of 5 checks passed

Docs: build check Docs: failed with errors
Details
Docs: render build OK - docs rendered
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details

@jdurgin jdurgin deleted the jdurgin:wip-osd-memory-target-mimic branch Dec 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment