ceph: reduce the number of provisioner pods in small K8s cluster #6437
Conversation
| @@ -34,6 +34,12 @@ data: | |||
| # Supported values from 0 to 5. 0 for general useful logs, 5 for trace level verbosity. | |||
| # CSI_LOG_LEVEL: "0" | |||
|
|
|||
| # Set the number of replicas of rbdplugin-provisioner | |||
| # CSI_RBD_PLUGIN_PROVISIONER_REPLICAS: "2" | |||
There was a problem hiding this comment.
can we just check the number of worker nodes in code and set replica to 1?
With this approach, a user can set a higher value and it creates more replicas
There was a problem hiding this comment.
It's a nice idea.
I have one question. Do you recommend not to the number of replicas larger than 2? If so, I just modify this PR as you suggested. Otherwise, it's also nice to make it configurable and the default value becomes 1 or 2 depending on the number of nodes.
There was a problem hiding this comment.
Yes earlier the count was 3. as it consumes more resources and provisioner will always be in HA, what we thought was 2 is good enough
There was a problem hiding this comment.
OK, I'll fix it.
satoru-takeuchi
force-pushed
the
ceph-the-number-of-provisioner-pods
branch
from
1479965
to
433d9f0
Compare
satoru-takeuchi
changed the title
pkg/operator/ceph/csi/spec.go
Outdated
| defaultProvisionerReplicas, err) | ||
| } | ||
| n := uint8(len(nodes.Items)) | ||
| if n < defaultProvisionerReplicas { |
There was a problem hiding this comment.
To make it more obvious that this is really for the single node case, how about this?
if len(nodes.Items) == 1 {
tp.ProvisionerReplicas = 1
}
pkg/operator/ceph/csi/spec.go
Outdated
| if err != nil { | ||
| logger.Errorf("failed to get nodes. Defaulting the number of replicas of provisioner pods to %d. %v", | ||
| defaultProvisionerReplicas, err) | ||
| } |
There was a problem hiding this comment.
If there was an error, nodes would be nil, so we need to put what's next in an else block, right?
The default number of provisioner pods are 2. When there is only one node, for example the CI environment, one of these becomes Pending state. Although it's harmless, it's better to reduce this value to 1. Closes: rook#6294 Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
satoru-takeuchi
force-pushed
the
ceph-the-number-of-provisioner-pods
branch
from
433d9f0
to
e22bedb
Compare
|
@satoru-takeuchi The CI issue was intermittent, I'll go ahead and merge so we can get it included in the release today. Thanks! |
ceph: reduce the number of provisioner pods in small K8s cluster (bp #6437)
|
@travisn which rook-ceph version has this fix ? am using 1.4.3 and am facing the same problem |
|
@rajivml It's v1.4.6 or later. In addition, v1.5.0 also includes this fix. |
Description of your changes:
The default number of provisioner pods are 2. When there is only one node, for example, the CI environment, one of these becomes Pending state. Although it's harmless, it's better to reduce this value to 1.
Which issue is resolved by this Pull Request:
Resolves #6294
Checklist:
make codegen) has been run to update object specifications, if necessary.[test ceph]