Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
os/bluestore: default cache size of 3gb #15976
Yeah. The idea is that bluestore OSDs are newly deployed or converted OSDs, so it is a "safe" opportunity to have a new default. And that 5gb is more in line with what is deployed in the real world. Also, performance improves pretty dramatically going from 1gb -> 2 or 3gb.
FWIW, I think we need to fix the memory leak Igor found and see how much memory we end up using after that. I'd be in favor of a larger cache size (3GB) for NVMe by default and a smaller cache size (1GB) for HDD.
Especially for configurations where data is on HDD and the DB partition is placed on flash (big enough to hold all onode/extent metadata), I'm not sure how much a large onode cache in bluestore actually matters. The large flash based DB partition is probably more important. FWIW it looks like an 8GB DB partition is enough to hold metadata for ~600K 4K objects with min_alloc set to 64k (default for HDD).
When everything is on NVMe, the onode cache is very important to achieve high IOPS rates. That's where I think we really want to encourage (perhaps demand) that users allocate plenty of RAM for onode cache. This may matter less if DB/WAL partitions are on even faster storage technologies (nvdimm, optane, etc).