Placement Group Management #560
Comments
|
This is interesting @hunter, it seems like it has two separate parts:
http://ceph.com/pgcalc/ can get a bit complicated as it depends on a fair amount of cluster "forecasting" knowledge to be known by the user, but I do like your thinking about trying to scope that down to a slim set of information that we'd need to collect from the user via the TPRs. |
|
I think ceph needs to support shrinking pgs. Without this it would be hard to make things dynamic. |
|
I don't think we should be exposing PG's to the end user. I think we need to figure out how to do the "right thing." One of the tenants of Rook is that we manage the SDS system so clients of storage don't need to be experts. |
|
The difficulty in hiding the PGs is that we need some way of knowing the amount of data to be stored inside a pool. Exposing a % may be an option, although may confuse things since an end user won't know about the system pools ( Alternatively for smaller clusters, it is possible to use standard sizes (although not as optimal) - http://docs.ceph.com/docs/master/rados/operations/placement-groups/ |
|
Ceph appears to be recommending the following on my Luminous cluster:
Prehaps it'd be worth using the data from |
|
Interesting and very relevant: |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation. |
|
It looks like ceph intends to add the option to autoscale placement groups, so rook may not need to attend to this. Rook could calculate a good starting, or just wait for the autoscaling to increase the default 100. Don't see this in 13.2.2, so maybe a later version. |
|
Need to confirm that the mgr module for placement mgmt solves this |
do we need a generic way to enable or disable mgr modues? e.g. dashboard, prometheus, diskprediction, crash, influx, pg_autoscaler, etc. |
|
I had to enable the OSDs balancer in mgr manually... That's a super useful thing to have |
|
Although this stuff is heavily dependent on ceph features currently used... |
|
👍 on adding a CRD property to enable mgr modules, as long as specifying some set of modules to enable does not make rook disable any modules not explicitly called out (since we tend to add new modules with each release). As for converting straw to straw2, I think that's something that rook could do (as an opinionated operator), but it's also something that ceph-mgr could do for the benefit of all users. I'll add a ticket for ceph to do this on its own... |
|
Ha, ceph already has a CLI command to do this: |
|
Yup, but as I said it's heavily dependent on tunables version, which is dependent on versions of client kernels.. Which you can see the current ones, but you don't know what user will want to connect |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation. |
@travisn - I assume the above is still relevant? Should we open this item? (it's not clear to me why it's 'done' under 1.1) |
|
Agreed, reopening... the bot just closed it and marked it as done |
|
@LenzGr this might create conflicts between what the dashboard things and what rook things are enabled modules |
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now enable via the CRD any manager module, to do this simply do
the following:
```
mgr:
modules:
- name: pg_autoscaler
```
Closes: rook#560
Signed-off-by: Sébastien Han <seb@redhat.com>
Following from some of the discussion in #554 (and #558), it may be useful to add PG configuration to the Pool TPR. That would give more control over the global default which are not a great fit for some of the default pools (the PG calc has some recommendations - http://ceph.com/pgcalc/)
One of the benefits of the dynamic nature of Rook is that some of the PG configuration could be automated. Since the PGs should change as more OSDs are added, we can get some of that info from the rook cluster and adjust if needed (assuming the PGs don't scale down).
The calculator uses the formula of
(( Target PGs per OSD ) x ( OSD # ) x ( %Data )) / ( Size )so perhaps the estimated%Datacould be specified in the Pool TPR to let Rook manage the PG Allocation across all the pools.This also raises the question, when FS or RGW services are launched, should they pre-create the Pool TPRs to allow control over the smaller/less utilised pools.
The text was updated successfully, but these errors were encountered: