doc/rados: edit placement-groups.rst (2 of x) #51991
Conversation
| Setting a Minimum Number of PGs and a Maximum Number of PGs | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| If a minimum is set, then Ceph will not itself reduce (or recommend that you |
| for you based on how much data is stored in the pool (see above, :ref:`pg-autoscaler`). | ||
| If you opt not to specify ``pg_num`` in this command, the cluster uses the PG | ||
| autoscaler to automatically configure the parameter in accordance with the | ||
| amount of data that is stored in the pool (see :ref:`pg-autoscaler` above). |
There was a problem hiding this comment.
Actually, won't it use osd_pool_default_pg_num? As written this assumes that the autoscaler is enabled.
| the additional of the balancer (which is also enabled by default), a | ||
| value of more like 50 PGs per OSD is probably reasonable. The | ||
| challenge (which the autoscaler normally does for you), is to: | ||
| The traditional rule of thumb has held that there should be 100 PGs per OSD. |
There was a problem hiding this comment.
Tradition was 200, retconned to 100 a few years ago. I personally disagree with the party line here, but this isn't about me. Also, is rule of thumb a thing that might not be known to non-native English readers?
Suggest
Without the balancer, approximately 100 PG replicas on each OSD is the suggested target. With the balancer, however, an initial target of 50 PG replicas on each OSD is reasonable.
There was a problem hiding this comment.
Accepted with slight modifications.
| consideration | ||
| - the number of PGs per pool should be proportional to the amount of | ||
| data in the pool | ||
| - there should be 50-100 PGs per pool, taking into account the |
| - the number of PGs per pool should be proportional to the amount of | ||
| data in the pool | ||
| - there should be 50-100 PGs per pool, taking into account the | ||
| replication overhead or erasure-coding fan-out of each PG across OSDs |
|
|
||
| As long as there are one or two orders of magnitude more PGs than OSDs, the | ||
| distribution is likely to be even. For example: 256 PGs for 3 OSDs, 512 PGs for | ||
| 10 OSDs, or 1024 PGs for 10 OSDs. |
There was a problem hiding this comment.
Maybe add a note about how pg_num not being a power of 2 can work against uniform distribution. Don't go into why, if someone really cares they can read chapter 8 of my book, so long as they pre-dose with their favorite headache medication.
There was a problem hiding this comment.
This is covered later on this same page (and will be edited in the "3 of x" PR in this series).
| 10 OSDs, or 1024 PGs for 10 OSDs. | ||
|
|
||
| However, uneven data distribution can emerge due to factors other than the | ||
| ratio of OSDs to PGs. For example, since CRUSH does not take into account the |
|
|
||
| However, uneven data distribution can emerge due to factors other than the | ||
| ratio of OSDs to PGs. For example, since CRUSH does not take into account the | ||
| size of the objects, the presence of a few very large objects can create an |
There was a problem hiding this comment.
size of RADOS objects
There was a problem hiding this comment.
Accepted in all instances.
| at all times and even more during recovery. Sharing this overhead by | ||
| clustering objects within a placement group is one of the main reasons | ||
| they exist. | ||
| Every PG in the cluster imposes additional memory, network, and CPU demands |
| clustering objects within a placement group is one of the main reasons | ||
| they exist. | ||
| Every PG in the cluster imposes additional memory, network, and CPU demands | ||
| upon OSDs and MONs. These needs must be met at all times and are still more |
There was a problem hiding this comment.
and are increased during recovery.
Edit doc/rados/operations/placement-groups.rst. https://tracker.ceph.com/issues/58485 Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
zdover23
force-pushed
the
wip-doc-2023-06-09-rados-operations-placement-groups-2-of-x
branch
from
04a439d
to
8bdd271
Compare
Edit doc/rados/operations/placement-groups.rst.
https://tracker.ceph.com/issues/58485
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windows