Skip to content

Commit

Permalink
Don't consider stderr CLI output to be an error
Browse files Browse the repository at this point in the history
Between RabbitMQ v3.8.14 and v3.8.15, the command
`rabbitmq-queues rebalance quorum` outputs warn messages like
`10:59:50.674 [warn]  Node :"rabbit@r1-server-0.r1-nodes.default" only
contains 1 queues, do nothing` to stderr.

Although this CLI command exits with 0, the operator considers this as a
failure, retrying to execute that command. The operator logs the
following:
`2021-03-19T08:39:56.305Z	ERROR
controller-runtime.manager.controller.rabbitmqcluster	Reconciler error
{"reconciler group": "rabbitmq.com", "reconciler kind":
  "RabbitmqCluster", "name": "r1", "namespace": "default", "error":
    "failed to run queue rebalance on pod r1-server-0:`

In general when CLI commands output to stderr, it doesn't mean that
there was an error. So, in this commit we only consider non-zero exit
codes as error.
  • Loading branch information
ansd committed Mar 22, 2021
1 parent 7117161 commit b6d9557
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 9 deletions.
2 changes: 1 addition & 1 deletion config/crd/bases/rabbitmq.com_rabbitmqclusters.yaml
Expand Up @@ -3499,7 +3499,7 @@ spec:
additionalPlugins:
description: 'List of plugins to enable in addition to essential plugins: rabbitmq_management, rabbitmq_prometheus, and rabbitmq_peer_discovery_k8s.'
items:
description: kubebuilder validating tags 'Pattern' and 'MaxLength' must be specified on string type. Alias type 'string' as 'Plugin' to specify schema validation on items of the list 'AdditionalPlugins'
description: A Plugin to enable on the RabbitmqCluster.
maxLength: 100
pattern: ^\w+$
type: string
Expand Down
10 changes: 2 additions & 8 deletions controllers/pod_executor.go
Expand Up @@ -3,7 +3,7 @@ package controllers
import (
"bufio"
"bytes"
"fmt"

corev1 "k8s.io/api/core/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/kubernetes/scheme"
Expand Down Expand Up @@ -48,12 +48,6 @@ func (p *rabbitmqPodExecutor) Exec(clientset *kubernetes.Clientset, clusterConfi
Stdin: nil,
Tty: false,
})
if err != nil {
return stdOut.String(), stdErr.String(), err
}
if stdErr.Len() > 0 {
return stdOut.String(), stdErr.String(), fmt.Errorf("%v", stdErr)
}

return stdOut.String(), "", nil
return stdOut.String(), stdErr.String(), err
}

0 comments on commit b6d9557

Please sign in to comment.