Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

istioctl proxy-status POD shows diffs by default #21709

Closed
howardjohn opened this issue Mar 2, 2020 · 2 comments
Closed

istioctl proxy-status POD shows diffs by default #21709

howardjohn opened this issue Mar 2, 2020 · 2 comments
Assignees
Milestone

Comments

@howardjohn
Copy link
Member

@howardjohn howardjohn commented Mar 2, 2020

I just installed an empty 1.5 cluster, added a sleep deployment, and ran ~/istio/istio-1.5.0-beta.5/bin/istioctl proxy-status SLEEP_POD.

I got this diff:

Clusters Match
--- Pilot Listeners
+++ Envoy Listeners
@@ -1,10 +1,11 @@
 {
    "dynamicListeners": [
       {
+         "name": "0.0.0.0_15010",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "0.0.0.0_15010",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -179,14 +180,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "0.0.0.0_15014",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "0.0.0.0_15014",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -361,14 +363,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "0.0.0.0_80",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "0.0.0.0_80",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -543,14 +546,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "0.0.0.0_8080",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "0.0.0.0_8080",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -725,14 +729,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "0.0.0.0_9090",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "0.0.0.0_9090",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -907,14 +912,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.0.10_53",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.0.10_53",
                "address": {
                   "socketAddress": {
                      "address": "10.0.0.10",
@@ -961,14 +967,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.0.1_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.0.1_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.0.1",
@@ -1015,14 +1022,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.11.116_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.11.116_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.11.116",
@@ -1206,14 +1214,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.11.163_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.11.163_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.11.163",
@@ -1260,14 +1269,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.12.183_5473",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.12.183_5473",
                "address": {
                   "socketAddress": {
                      "address": "10.0.12.183",
@@ -1451,14 +1461,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.15.188_15012",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.15.188_15012",
                "address": {
                   "socketAddress": {
                      "address": "10.0.15.188",
@@ -1505,14 +1516,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.15.188_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.15.188_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.15.188",
@@ -1559,14 +1571,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15020",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15020",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -1750,14 +1763,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15029",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15029",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -1941,14 +1955,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15030",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15030",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -2132,14 +2147,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15031",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15031",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -2323,14 +2339,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15032",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15032",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -2514,14 +2531,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_15443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_15443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -2568,14 +2586,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.2.83_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.2.83_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.2.83",
@@ -2622,14 +2641,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.7.133_15011",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.7.133_15011",
                "address": {
                   "socketAddress": {
                      "address": "10.0.7.133",
@@ -2676,14 +2696,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.7.133_15012",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.7.133_15012",
                "address": {
                   "socketAddress": {
                      "address": "10.0.7.133",
@@ -2730,14 +2751,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.7.133_443",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.7.133_443",
                "address": {
                   "socketAddress": {
                      "address": "10.0.7.133",
@@ -2784,14 +2806,15 @@
                   "bindToPort": false
                },
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.7.29_44134",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.7.29_44134",
                "address": {
                   "socketAddress": {
                      "address": "10.0.7.29",
@@ -2975,14 +2998,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.0.7.98_80",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.0.7.98_80",
                "address": {
                   "socketAddress": {
                      "address": "10.0.7.98",
@@ -3166,14 +3190,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.128.15.214_10250",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.128.15.214_10250",
                "address": {
                   "socketAddress": {
                      "address": "10.128.15.214",
@@ -3274,14 +3299,15 @@
                   }
                ],
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.128.15.227_10250",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.128.15.227_10250",
                "address": {
                   "socketAddress": {
                      "address": "10.128.15.227",
@@ -3382,14 +3408,15 @@
                   }
                ],
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.128.15.228_10250",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.128.15.228_10250",
                "address": {
                   "socketAddress": {
                      "address": "10.128.15.228",
@@ -3490,14 +3517,15 @@
                   }
                ],
                "trafficDirection": "OUTBOUND"
             }
          }
       },
       {
+         "name": "10.28.1.74_15020",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.28.1.74_15020",
                "address": {
                   "socketAddress": {
                      "address": "10.28.1.74",
@@ -3552,14 +3580,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "INBOUND"
             }
          }
       },
       {
+         "name": "10.28.1.74_80",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "10.28.1.74_80",
                "address": {
                   "socketAddress": {
                      "address": "10.28.1.74",
@@ -3699,14 +3728,15 @@
                "listenerFiltersTimeout": "0.100s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "INBOUND"
             }
          }
       },
       {
+         "name": "virtualInbound",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "virtualInbound",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",
@@ -4088,14 +4118,15 @@
                "listenerFiltersTimeout": "1s",
                "continueOnListenerFiltersTimeout": true,
                "trafficDirection": "INBOUND"
             }
          }
       },
       {
+         "name": "virtualOutbound",
          "activeState": {
             "listener": {
                "@type": "type.googleapis.com/envoy.api.v2.Listener",
                "name": "virtualOutbound",
                "address": {
                   "socketAddress": {
                      "address": "0.0.0.0",

Routes Don't Match (RDS last loaded at Mon, 02 Mar 2020 08:35:36 PST)
--- Pilot Routes
+++ Envoy Routes
@@ -1,14 +1,31 @@
 {
    "dynamicRouteConfigs": [
       {
          "routeConfig": {
             "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
             "name": "15010",
             "virtualHosts": [
+               {
+                  "name": "allow_any",
+                  "domains": [
+                     "*"
+                  ],
+                  "routes": [
+                     {
+                        "match": {
+                           "prefix": "/"
+                        },
+                        "route": {
+                           "cluster": "PassthroughCluster",
+                           "timeout": "0s"
+                        }
+                     }
+                  ]
+               },
                {
                   "name": "istio-pilot.istio-system.svc.cluster.local:15010",
                   "domains": [
                      "istio-pilot.istio-system.svc.cluster.local",
                      "istio-pilot.istio-system.svc.cluster.local:15010",
                      "istio-pilot.istio-system",
                      "istio-pilot.istio-system:15010",
@@ -44,15 +61,24 @@
                            "maxGrpcTimeout": "0s"
                         },
                         "decorator": {
                            "operation": "istio-pilot.istio-system.svc.cluster.local:15010/*"
                         }
                      }
                   ]
-               },
+               }
+            ],
+            "validateClusters": false
+         }
+      },
+      {
+         "routeConfig": {
+            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
+            "name": "15014",
+            "virtualHosts": [
                {
                   "name": "allow_any",
                   "domains": [
                      "*"
                   ],
                   "routes": [
                      {
@@ -61,24 +87,15 @@
                         },
                         "route": {
                            "cluster": "PassthroughCluster",
                            "timeout": "0s"
                         }
                      }
                   ]
-               }
-            ],
-            "validateClusters": false
-         }
-      },
-      {
-         "routeConfig": {
-            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
-            "name": "15014",
-            "virtualHosts": [
+               },
                {
                   "name": "istio-pilot.istio-system.svc.cluster.local:15014",
                   "domains": [
                      "istio-pilot.istio-system.svc.cluster.local",
                      "istio-pilot.istio-system.svc.cluster.local:15014",
                      "istio-pilot.istio-system",
                      "istio-pilot.istio-system:15014",
@@ -114,15 +131,24 @@
                            "maxGrpcTimeout": "0s"
                         },
                         "decorator": {
                            "operation": "istio-pilot.istio-system.svc.cluster.local:15014/*"
                         }
                      }
                   ]
-               },
+               }
+            ],
+            "validateClusters": false
+         }
+      },
+      {
+         "routeConfig": {
+            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
+            "name": "80",
+            "virtualHosts": [
                {
                   "name": "allow_any",
                   "domains": [
                      "*"
                   ],
                   "routes": [
                      {
@@ -131,24 +157,15 @@
                         },
                         "route": {
                            "cluster": "PassthroughCluster",
                            "timeout": "0s"
                         }
                      }
                   ]
-               }
-            ],
-            "validateClusters": false
-         }
-      },
-      {
-         "routeConfig": {
-            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
-            "name": "80",
-            "virtualHosts": [
+               },
                {
                   "name": "default-http-backend.kube-system.svc.cluster.local:80",
                   "domains": [
                      "default-http-backend.kube-system.svc.cluster.local",
                      "default-http-backend.kube-system.svc.cluster.local:80",
                      "default-http-backend.kube-system",
                      "default-http-backend.kube-system:80",
@@ -362,15 +379,24 @@
                            "maxGrpcTimeout": "0s"
                         },
                         "decorator": {
                            "operation": "sleep.without-istio.svc.cluster.local:80/*"
                         }
                      }
                   ]
-               },
+               }
+            ],
+            "validateClusters": false
+         }
+      },
+      {
+         "routeConfig": {
+            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
+            "name": "8080",
+            "virtualHosts": [
                {
                   "name": "allow_any",
                   "domains": [
                      "*"
                   ],
                   "routes": [
                      {
@@ -379,24 +405,15 @@
                         },
                         "route": {
                            "cluster": "PassthroughCluster",
                            "timeout": "0s"
                         }
                      }
                   ]
-               }
-            ],
-            "validateClusters": false
-         }
-      },
-      {
-         "routeConfig": {
-            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
-            "name": "8080",
-            "virtualHosts": [
+               },
                {
                   "name": "istio-pilot.istio-system.svc.cluster.local:8080",
                   "domains": [
                      "istio-pilot.istio-system.svc.cluster.local",
                      "istio-pilot.istio-system.svc.cluster.local:8080",
                      "istio-pilot.istio-system",
                      "istio-pilot.istio-system:8080",
@@ -432,15 +449,24 @@
                            "maxGrpcTimeout": "0s"
                         },
                         "decorator": {
                            "operation": "istio-pilot.istio-system.svc.cluster.local:8080/*"
                         }
                      }
                   ]
-               },
+               }
+            ],
+            "validateClusters": false
+         }
+      },
+      {
+         "routeConfig": {
+            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
+            "name": "9090",
+            "virtualHosts": [
                {
                   "name": "allow_any",
                   "domains": [
                      "*"
                   ],
                   "routes": [
                      {
@@ -449,24 +475,15 @@
                         },
                         "route": {
                            "cluster": "PassthroughCluster",
                            "timeout": "0s"
                         }
                      }
                   ]
-               }
-            ],
-            "validateClusters": false
-         }
-      },
-      {
-         "routeConfig": {
-            "@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
-            "name": "9090",
-            "virtualHosts": [
+               },
                {
                   "name": "prometheus.istio-system.svc.cluster.local:9090",
                   "domains": [
                      "prometheus.istio-system.svc.cluster.local",
                      "prometheus.istio-system.svc.cluster.local:9090",
                      "prometheus.istio-system",
                      "prometheus.istio-system:9090",
@@ -502,31 +519,14 @@
                            "maxGrpcTimeout": "0s"
                         },
                         "decorator": {
                            "operation": "prometheus.istio-system.svc.cluster.local:9090/*"
                         }
                      }
                   ]
-               },
-               {
-                  "name": "allow_any",
-                  "domains": [
-                     "*"
-                  ],
-                  "routes": [
-                     {
-                        "match": {
-                           "prefix": "/"
-                        },
-                        "route": {
-                           "cluster": "PassthroughCluster",
-                           "timeout": "0s"
-                        }
-                     }
-                  ]
                }
             ],
             "validateClusters": false
          }
       },
       {
          "routeConfig": {

Seems like there should be no diff in the expected case?

Note sure if this is due to changes on pilot, envoy, or istioctl. cc @esnible

@esnible

This comment has been minimized.

Copy link
Contributor

@esnible esnible commented Mar 6, 2020

@howardjohn , I hadn't realized that it does this.

There is currently no integration test of this so we missed it when we went to the new Envoy.

The diff is real. For listeners, Envoy is really creating dynamicListeners[*].name from dynamicListeners[*].activeState.listener.name. I didn't research the problem with the routes but I expect it will be similar.

I see three ways forward:

  • Modify Pilot discovery to push this redundant info to Envoy, fixing this tool
  • Tell this tool about the fields that Envoy creates on its own and suppress them from output
  • Deprecate this tool

I suspect 2 is the best. Thoughts?

@howardjohn

This comment has been minimized.

Copy link
Member Author

@howardjohn howardjohn commented Mar 6, 2020

Hm yeah it might be doing some other weird stuff with v3 api as well, as I have seen it replace things like host with hiddenDeprecated_host or similar.
I wouldn't be surprised if (1) doesn't actually work, but I could be wrong.

The ordering is different too, right? Is the diff order sensitive?

(2) seems reasonable for the name thing assuming its not too weird, I am not sure what the code looks like

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants
You can’t perform that action at this time.