Problem when running IOChaos #2305
Description
Bug Report
What version of Kubernetes are you using?
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0+4c3480d", GitCommit:"4c3480dcd4299c3b3e9a75e28d643177600e7d72", GitTreeState:"clean", BuildDate:"2021-07-09T00:02:08Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/amd64"}
What version of Chaos Mesh are you using?
Controller manager Version: version.Info{GitVersion:"v2.0.1", GitCommit:"2989e66ed8c1a9815cba025664a72ed59422c73d", BuildDate:"2021-08-26T10:04:06Z", GoVersion:"go1.15.11", Compiler:"gc", Platform:"linux/amd64"}
What did you do?
chaos-mesh was installed by executing the following command:
helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-testing --set chaosDaemon.runtime=crio --set chaosDaemon.socketPath=/var/run/crio/crio.sock
Trying to do an I/O experiment:
Here is the experiment yaml:
kind: IOChaos
apiVersion: chaos-mesh.org/v1alpha1
metadata:
name: chaosiomqtest1
namespace: cp4i
annotations:
experiment.chaos-mesh.org/pause: 'true'
spec:
selector:
namespaces:
- cp4i
labelSelectors:
statefulset.kubernetes.io/pod-name: mq-ddd-qm-dev-ibm-mq-0
mode: one
action: latency
delay: 100ms
percent: 100
volumePath: /opt/mqm/bin/
duration: 5m
What did you expect to see?
I/O latency running without error.
What did you see instead?
Got this error:
Failed to apply chaos: cannot find daemonIP on node 10.183.69.209 in related Endpoints {{Endpoints v1} {chaos-daemon chaos-testing /api/v1/namespaces/chaos-testing/endpoints/chaos-daemon c33f3b35-e841-458b-841e-e6edec3f7ad5 2108161 0 2021-09-13 08:17:55 +0000 UTC map[app.kubernetes.io/component:chaos-daemon app.kubernetes.io/instance:chaos-mesh app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:chaos-mesh app.kubernetes.io/part-of:chaos-mesh app.kubernetes.io/version:v2.0.1 helm.sh/chart:chaos-mesh-v2.0.1 service.kubernetes.io/headless:] map[endpoints.kubernetes.io/last-change-trigger-time:2021-09-13T08:17:55Z] [] [] [{kube-controller-manager Update v1 2021-09-13 08:17:55 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 109 101 116 97 100 97 116 97 34 58 123 34 102 58 97 110 110 111 116 97 116 105 111 110 115 34 58 123 34 46 34 58 123 125 44 34 102 58 101 110 100 112 111 105 110 116 115 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 108 97 115 116 45 99 104 97 110 103 101 45 116 114 105 103 103 101 114 45 116 105 109 101 34 58 123 125 125 44 34 102 58 108 97 98 101 108 115 34 58 123 34 46 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 105 110 115 116 97 110 99 101 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 109 97 110 97 103 101 100 45 98 121 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 110 97 109 101 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 112 97 114 116 45 111 102 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 118 101 114 115 105 111 110 34 58 123 125 44 34 102 58 104 101 108 109 46 115 104 47 99 104 97 114 116 34 58 123 125 44 34 102 58 115 101 114 118 105 99 101 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 104 101 97 100 108 101 115 115 34 58 123 125 125 125 125],}}]} []}
Output of chaosctl
2021-09-14T15:05:01.964Z DEBUG controller-runtime.manager.events Warning {"object": {"kind":"PodIOChaos","namespace":"cp4i","name":"mq-ddd-qm-dev-ibm-mq-0","uid":"cf1ce43d-0dcb-4cff-9e4c-8aeabfcae68a","apiVersion":"chaos-mesh.org/v1alpha1","resourceVersion":"3482590"}, "reason": "Failed", "message": "cannot find daemonIP on node 10.183.69.209 in related Endpoints {{Endpoints v1} {chaos-daemon chaos-testing /api/v1/namespaces/chaos-testing/endpoints/chaos-daemon c33f3b35-e841-458b-841e-e6edec3f7ad5 2108161 0 2021-09-13 08:17:55 +0000 UTC map[app.kubernetes.io/component:chaos-daemon app.kubernetes.io/instance:chaos-mesh app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:chaos-mesh app.kubernetes.io/part-of:chaos-mesh app.kubernetes.io/version:v2.0.1 helm.sh/chart:chaos-mesh-v2.0.1 service.kubernetes.io/headless:] map[endpoints.kubernetes.io/last-change-trigger-time:2021-09-13T08:17:55Z] [] [] [{kube-controller-manager Update v1 2021-09-13 08:17:55 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 109 101 116 97 100 97 116 97 34 58 123 34 102 58 97 110 110 111 116 97 116 105 111 110 115 34 58 123 34 46 34 58 123 125 44 34 102 58 101 110 100 112 111 105 110 116 115 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 108 97 115 116 45 99 104 97 110 103 101 45 116 114 105 103 103 101 114 45 116 105 109 101 34 58 123 125 125 44 34 102 58 108 97 98 101 108 115 34 58 123 34 46 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 105 110 115 116 97 110 99 101 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 109 97 110 97 103 101 100 45 98 121 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 110 97 109 101 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 112 97 114 116 45 111 102 34 58 123 125 44 34 102 58 97 112 112 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 118 101 114 115 105 111 110 34 58 123 125 44 34 102 58 104 101 108 109 46 115 104 47 99 104 97 114 116 34 58 123 125 44 34 102 58 115 101 114 118 105 99 101 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 104 101 97 100 108 101 115 115 34 58 123 125 125 125 125],}}]} []}"}