NetworkChaos would fail with device: eth0@if12, including @ifXX in the device

[STRRL]

Bug Report

NetworkChaos works with device: eth0, not work with device: eth0@if12

I am not sure if it would be a bug or an enhancement.

We could add a validation webhook, or trim the things after @.

What version of Kubernetes are you using?

N/A

What version of Chaos Mesh are you using?

It would affect all the Chaos Mesh versions.

What did you do?

• minikube start --cni flannel
• k create deployment --image nginx:latest --replicas 3 nginx
• apply this NetworkChaos:

kind: NetworkChaos
apiVersion: chaos-mesh.org/v1alpha1
metadata:
  namespace: default
  name: loss
spec:
  selector:
    namespaces:
      - default
    labelSelectors:
      app: nginx   
  mode: all
  action: loss
  duration: 1m
  device: "eth0@if7"
  loss:
    loss: '50'
    correlation: '0'
  direction: to

What did you expect to see?

NetworkChaos injected succeed, or creation failure with some error or warning from validation webhook.

What did you see instead?

NetworkChaos created and failed to inject, I could only forcely delete it.

logs

2022-05-10T03:31:16.237Z        INFO    chaos-daemon.daemon-server      chaosdaemon/server.go:187       flush ipset     {"namespacedName": "default/nginx-7c658794b9-bdzzj", "request": "container_id:\"docker://e982ed98e8f48c07272ba7726cec9db459fae9a5a5366c505a5b76bca83ef2f8\"  enterNS:true"}
2022-05-10T03:31:16.238Z        INFO    chaos-daemon.daemon-server      chaosdaemon/server.go:187       Set iptables chains     {"namespacedName": "default/nginx-7c658794b9-bdzzj", "request": "container_id:\"docker://e982ed98e8f48c07272ba7726cec9db459fae9a5a5366c505a5b76bca83ef2f8\"  enterNS:true"}
2022-05-10T03:31:16.238Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:177      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -N CHAOS-INPUT"}
2022-05-10T03:31:16.239Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:217      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -F CHAOS-INPUT"}
2022-05-10T03:31:16.241Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:185      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -S INPUT"}
2022-05-10T03:31:16.242Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:177      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -N CHAOS-OUTPUT"}
2022-05-10T03:31:16.243Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:217      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -F CHAOS-OUTPUT"}
2022-05-10T03:31:16.244Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/iptables_server.go:185      build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- iptables -w -S OUTPUT"}
2022-05-10T03:31:16.245Z        INFO    chaos-daemon.daemon-server      chaosdaemon/server.go:187       handling tc request     {"namespacedName": "default/nginx-7c658794b9-bdzzj", "tcs": "tcs:{netem:{loss:50}  device:\"eth0@if7\"}  container_id:\"docker://e982ed98e8f48c07272ba7726cec9db459fae9a5a5366c505a5b76bca83ef2f8\"  enterNS:true"}
2022-05-10T03:31:16.246Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/tc_server.go:113    build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- ip -j addr show"}
2022-05-10T03:31:16.247Z        INFO    chaos-daemon.daemon-server      pb/chaosdaemon.pb.go:3710       get interfaces from ip command  {"namespacedName": "default/nginx-7c658794b9-bdzzj", "ifaces": ["lo", "tunl0", "eth0"]}
2022-05-10T03:31:16.247Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/tc_server.go:119    build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- tc qdisc del dev lo root"}
2022-05-10T03:31:16.249Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/tc_server.go:119    build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- tc qdisc del dev tunl0 root"}
2022-05-10T03:31:16.250Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/tc_server.go:119    build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- tc qdisc del dev eth0 root"}
2022-05-10T03:31:16.251Z        INFO    chaos-daemon.daemon-server      chaosdaemon/tc_server.go:166    add tc  {"namespacedName": "default/nginx-7c658794b9-bdzzj", "tc": "netem:{loss:50}  device:\"eth0@if7\""}
2022-05-10T03:31:16.252Z        INFO    chaos-daemon.daemon-server      chaosdaemon/tc_server.go:204    adding netem    {"namespacedName": "default/nginx-7c658794b9-bdzzj", "device": "eth0@if7", "parent": "root", "handle": "handle 1:"}
2022-05-10T03:31:16.252Z        INFO    chaos-daemon.daemon-server.background-process-manager.process-builder   chaosdaemon/tc_server.go:331    build command   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "command": "/usr/local/bin/nsexec -n /proc/4442/ns/net -- tc qdisc add dev eth0@if7 root handle 1: netem loss 50.000000"}
2022-05-10T03:31:16.254Z        ERROR   chaos-daemon.daemon-server      pb/chaosdaemon.pb.go:3710       error while adding tc   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "error": "error code: exit status 1, msg: Cannot find device \"eth0@if7\"\n", "errorVerbose": "error code: exit status 1, msg: Cannot find device \"eth0@if7\"\n\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/util.EncodeOutputToError\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/util/util.go:118\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*tcClient).addNetem\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:390\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*tcClient).addTc\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:331\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*DaemonServer).setGlobalTcs\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:204\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*DaemonServer).SetTcs\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:166\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler.func1\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3710\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.MetadataExtractor.func1\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/server.go:187\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/chaos-mesh/chaos-mesh/pkg/grpc.TimeoutServerInterceptor\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/grpc/utils.go:205\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3712\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1571"}
github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler.func1
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3710
github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.MetadataExtractor.func1
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/server.go:187
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/chaos-mesh/chaos-mesh/pkg/grpc.TimeoutServerInterceptor
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/grpc/utils.go:205
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34
github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3712
google.golang.org/grpc.(*Server).processUnaryRPC
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297
google.golang.org/grpc.(*Server).handleStream
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626
google.golang.org/grpc.(*Server).serveStreams.func1.2
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941
2022-05-10T03:31:16.255Z        ERROR   chaos-daemon.daemon-server      chaosdaemon/server.go:187       error while setting global tc   {"namespacedName": "default/nginx-7c658794b9-bdzzj", "error": "error code: exit status 1, msg: Cannot find device \"eth0@if7\"\n", "errorVerbose": "error code: exit status 1, msg: Cannot find device \"eth0@if7\"\n\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/util.EncodeOutputToError\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/util/util.go:118\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*tcClient).addNetem\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:390\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*tcClient).addTc\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:331\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*DaemonServer).setGlobalTcs\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:204\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.(*DaemonServer).SetTcs\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/tc_server.go:166\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler.func1\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3710\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.MetadataExtractor.func1\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/server.go:187\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/chaos-mesh/chaos-mesh/pkg/grpc.TimeoutServerInterceptor\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/grpc/utils.go:205\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1\n\t/tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34\ngithub.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler\n\t/home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3712\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1571"}
github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.MetadataExtractor.func1
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/server.go:187
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/chaos-mesh/chaos-mesh/pkg/grpc.TimeoutServerInterceptor
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/grpc/utils.go:205
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
        /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34
github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_SetTcs_Handler
        /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3712
google.golang.org/grpc.(*Server).processUnaryRPC
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297
google.golang.org/grpc.(*Server).handleStream
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626
google.golang.org/grpc.(*Server).serveStreams.func1.2
        /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941

Output of chaosctl

N/A

[YangKeao]

It's expected, eth0@if7 only means the peer of eth0 device is the interface on host whose index is 7, and @if7 is never part of the interface name.

We could add a validation webhook, or trim the things after @.

Good suggestion.

[basit9958]

/assign

[STRRL]

Hi @basit9958 , before you start coding, do you mind sharing what would you do to us? :)

[Moynur]

Hey is this open to be picked up

[basit9958]

@Moynur I am not working on this. You can pick this up

[basit9958]

@STRRL I believe we can check if Device contains @ and trim it here https://github.com/chaos-mesh/chaos-mesh/blob/master/api/v1alpha1/zz_generated.chaosmesh.go#L1376