Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test: 1/10K: AggregatedClusterSuccess_SwitchBetweenLeafAndAggregate #6615

Open
easwars opened this issue Sep 11, 2023 · 0 comments
Open

Flaky test: 1/10K: AggregatedClusterSuccess_SwitchBetweenLeafAndAggregate #6615

easwars opened this issue Sep 11, 2023 · 0 comments
Assignees
@easwars
Labels
P2 Type: Testing

Comments

@easwars
Copy link
Contributor

easwars commented Sep 11, 2023 •
edited 

--- FAIL: Test (6.62s)
    --- FAIL: Test/AggregatedClusterSuccess_SwitchBetweenLeafAndAggregate (5.06s)
        tlogger.go:116: INFO server.go:132 [xds-e2e] Created new snapshot cache...  (t=+75.701µs)
        tlogger.go:116: INFO server.go:667 [core] [Server #303] Server created  (t=+335.405µs)
        tlogger.go:116: INFO server.go:158 [xds-e2e] Registered Aggregated Discovery Service (ADS)...  (t=+383.406µs)
        tlogger.go:116: INFO server.go:177 [xds-e2e] xDS management server serving at: 127.0.0.1:42559...  (t=+423.106µs)
        tlogger.go:116: INFO client_new.go:80 [xds] [xds-client 0xc0007b0460] Created client to xDS management server: 127.0.0.1:42559-insecure-xds_v3  (t=+2.091831ms)
        tlogger.go:116: INFO clientconn.go:318 [core] [Channel #304] Channel created  (t=+2.223833ms)
        tlogger.go:116: INFO clientconn.go:1825 [core] [Channel #304] original dial target is: "whatever:///test.service"  (t=+2.299634ms)
        tlogger.go:116: INFO clientconn.go:1832 [core] [Channel #304] parsed dial target is: {URL:{Scheme:whatever Opaque: User: Host: Path:/test.service RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}  (t=+2.357535ms)
        tlogger.go:116: INFO clientconn.go:1987 [core] [Channel #304] Channel authority set to "test.service"  (t=+2.392336ms)
        tlogger.go:116: INFO resolver_conn_wrapper.go:246 [core] [Channel #304] Resolver state updated: {
              "Addresses": null,
              "Endpoints": [],
              "ServiceConfig": {
                "Config": {
                  "Config": null,
                  "LB": null,
                  "Methods": {}
                },
                "Err": null
              },
              "Attributes": {
                "\u003c%!p(xdsclient.clientKeyType=grpc.xds.internal.client.Client)\u003e": "\u003c0xc0007ac6f0\u003e"
              }
            } (service config updated)  (t=+2.561138ms)
        tlogger.go:116: INFO server.go:855 [core] [Server #303 ListenSocket #305] ListenSocket created  (t=+2.646239ms)
        tlogger.go:116: INFO balancer_conn_wrappers.go:180 [core] [Channel #304] Channel switches to new LB policy "cds_experimental"  (t=+2.723441ms)
        tlogger.go:116: INFO cdsbalancer.go:99 [xds] [cds-lb 0xc0002f7080] Created  (t=+2.765441ms)
        tlogger.go:116: INFO cdsbalancer.go:110 [xds] [cds-lb 0xc0002f7080] xDS credentials in use: false  (t=+2.794542ms)
        tlogger.go:116: INFO cdsbalancer.go:499 [xds] [cds-lb 0xc0002f7080] Received balancer config update: {
              "LoadBalancingConfig": null,
              "Cluster": "cluster1"
            }  (t=+2.849643ms)
        tlogger.go:116: INFO cluster_handler.go:190 [xds] [cds-lb 0xc0002f7080] CDS watch started on cluster1  (t=+2.897743ms)
        tlogger.go:116: INFO clientconn.go:318 [core] [Channel #306] Channel created  (t=+2.979844ms)
        tlogger.go:116: INFO clientconn.go:1825 [core] [Channel #306] original dial target is: "127.0.0.1:42559"  (t=+3.024245ms)
        tlogger.go:116: INFO clientconn.go:1830 [core] [Channel #306] dial target "127.0.0.1:42559" parse failed: parse "127.0.0.1:42559": first path segment in URL cannot contain colon  (t=+3.074246ms)
        tlogger.go:116: INFO clientconn.go:1846 [core] [Channel #306] fallback to scheme "passthrough"  (t=+3.105446ms)
        tlogger.go:116: INFO clientconn.go:1854 [core] [Channel #306] parsed dial target is: {URL:{Scheme:passthrough Opaque: User: Host: Path:/127.0.0.1:42559 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}  (t=+3.154247ms)
        tlogger.go:116: INFO clientconn.go:1987 [core] [Channel #306] Channel authority set to "127.0.0.1:42559"  (t=+3.189648ms)
        tlogger.go:116: INFO resolver_conn_wrapper.go:246 [core] [Channel #306] Resolver state updated: {
              "Addresses": [
                {
                  "Addr": "127.0.0.1:42559",
                  "ServerName": "",
                  "Attributes": null,
                  "BalancerAttributes": null,
                  "Metadata": null
                }
              ],
              "Endpoints": [
                {
                  "Addresses": [
                    {
                      "Addr": "127.0.0.1:42559",
                      "ServerName": "",
                      "Attributes": null,
                      "BalancerAttributes": null,
                      "Metadata": null
                    }
                  ],
                  "Attributes": null
                }
              ],
              "ServiceConfig": null,
              "Attributes": null
            } (resolver returned new addresses)  (t=+3.32355ms)
        tlogger.go:116: INFO server.go:220 [xds-e2e] Created new resource snapshot...  (t=+3.432051ms)
        tlogger.go:116: INFO server.go:226 [xds-e2e] Updated snapshot cache with resource snapshot...  (t=+3.468852ms)
        tlogger.go:116: INFO balancer_conn_wrappers.go:180 [core] [Channel #306] Channel switches to new LB policy "pick_first"  (t=+3.564453ms)
        tlogger.go:116: INFO clientconn.go:956 [core] [Channel #306 SubChannel #307] Subchannel created  (t=+3.632754ms)
        tlogger.go:116: INFO clientconn.go:592 [core] [Channel #306] Channel Connectivity change to CONNECTING  (t=+3.673155ms)
        tlogger.go:116: INFO clientconn.go:1338 [core] [Channel #306 SubChannel #307] Subchannel Connectivity change to CONNECTING  (t=+3.731456ms)
        tlogger.go:116: INFO transport.go:238 [xds] [xds-client 0xc0007b0460] [127.0.0.1:42559] Created transport to server "127.0.0.1:42559"  (t=+3.827257ms)
        tlogger.go:116: INFO clientconn.go:1453 [core] [Channel #306 SubChannel #307] Subchannel picks a new address "127.0.0.1:42559" to connect  (t=+4.118862ms)
        tlogger.go:116: INFO clientconn.go:1338 [core] [Channel #306 SubChannel #307] Subchannel Connectivity change to READY  (t=+4.948374ms)
        tlogger.go:116: INFO clientconn.go:592 [core] [Channel #306] Channel Connectivity change to READY  (t=+5.043875ms)
        tlogger.go:116: INFO transport.go:347 [xds] [xds-client 0xc0007b0460] [127.0.0.1:42559] ADS stream created  (t=+5.468282ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] and known map[]. Diff [cluster1]  (t=+5.772786ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] version "" with version "1"  (t=+5.820487ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] and known map[cluster1:{}]. Diff []  (t=+6.503697ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 1 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "1"  (t=+6.557098ms)
        tlogger.go:116: INFO cdsbalancer.go:290 [xds] [cds-lb 0xc0002f7080] Received Cluster resource contains content: [
              {
                "ClusterType": 0,
                "ClusterName": "cluster1",
                "EDSServiceName": "service1",
                "LRSServerConfig": 0,
                "SecurityCfg": null,
                "MaxRequests": null,
                "DNSHostName": "",
                "PrioritizedClusterNames": null,
                "LBPolicy": [
                  {
                    "xds_wrr_locality_experimental": {
                      "childPolicy": [
                        {
                          "round_robin": {}
                        }
                      ]
                    }
                  }
                ],
                "OutlierDetection": null,
                "Raw": {
                  "type_url": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
                  "value": "CghjbHVzdGVyMRoOCgIaABIIc2VydmljZTEQAw=="
                }
              }
            ], security config: null  (t=+6.783201ms)
        tlogger.go:116: INFO clusterresolver.go:86 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Created  (t=+6.851102ms)
        tlogger.go:116: INFO cdsbalancer.go:316 [xds] [cds-lb 0xc0002f7080] Created child policy 0xc00079b1c0 of type cluster_resolver_experimental  (t=+6.909303ms)
        tlogger.go:116: INFO server.go:220 [xds-e2e] Created new resource snapshot...  (t=+8.366425ms)
        tlogger.go:116: INFO simple.go:238 [xds-e2e] respond open watch 1 type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] with new version "2"  (t=+8.417926ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] version "1" with version "2"  (t=+8.457526ms)
        tlogger.go:116: INFO server.go:226 [xds-e2e] Updated snapshot cache with resource snapshot...  (t=+8.497127ms)
        tlogger.go:116: INFO cluster_handler.go:190 [xds] [cds-lb 0xc0002f7080] CDS watch started on cluster1-eds  (t=+8.880633ms)
        tlogger.go:116: INFO cluster_handler.go:190 [xds] [cds-lb 0xc0002f7080] CDS watch started on cluster1-dns  (t=+8.936333ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] and known map[cluster1:{}]. Diff []  (t=+9.255438ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 2 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "2"  (t=+9.308039ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1 cluster1-eds] and known map[cluster1:{}]. Diff [cluster1-eds]  (t=+9.37244ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1 cluster1-eds] version "2" with version "2"  (t=+9.407841ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] and known map[cluster1:{} cluster1-eds:{}]. Diff [cluster1-dns]  (t=+9.949349ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] version "2" with version "2"  (t=+10.006949ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] and known map[cluster1:{} cluster1-dns:{} cluster1-eds:{}]. Diff []  (t=+10.671759ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 3 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "2"  (t=+10.72226ms)
        tlogger.go:116: INFO clusterresolver.go:190 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Received new balancer config: {
              "discoveryMechanisms": [
                {
                  "cluster": "cluster1",
                  "edsServiceName": "service1",
                  "outlierDetection": {}
                }
              ],
              "xdsLbPolicy": [
                {
                  "xds_wrr_locality_experimental": {
                    "childPolicy": [
                      {
                        "round_robin": {}
                      }
                    ]
                  }
                }
              ]
            }  (t=+11.002764ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] and known map[]. Diff [service1]  (t=+11.261568ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] version "" with version "2"  (t=+11.308369ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] and known map[service1:{}]. Diff []  (t=+11.630274ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 4 for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "2"  (t=+11.677974ms)
        tlogger.go:116: INFO cdsbalancer.go:290 [xds] [cds-lb 0xc0002f7080] Received Cluster resource contains content: [
              {
                "ClusterType": 0,
                "ClusterName": "cluster1-eds",
                "EDSServiceName": "service1",
                "LRSServerConfig": 0,
                "SecurityCfg": null,
                "MaxRequests": null,
                "DNSHostName": "",
                "PrioritizedClusterNames": null,
                "LBPolicy": [
                  {
                    "xds_wrr_locality_experimental": {
                      "childPolicy": [
                        {
                          "round_robin": {}
                        }
                      ]
                    }
                  }
                ],
                "OutlierDetection": null,
                "Raw": {
                  "type_url": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
                  "value": "CgxjbHVzdGVyMS1lZHMaDgoCGgASCHNlcnZpY2UxEAM="
                }
              },
              {
                "ClusterType": 1,
                "ClusterName": "cluster1-dns",
                "EDSServiceName": "",
                "LRSServerConfig": 0,
                "SecurityCfg": null,
                "MaxRequests": null,
                "DNSHostName": "dns_host:8080",
                "PrioritizedClusterNames": null,
                "LBPolicy": [
                  {
                    "xds_wrr_locality_experimental": {
                      "childPolicy": [
                        {
                          "round_robin": {}
                        }
                      ]
                    }
                  }
                ],
                "OutlierDetection": null,
                "Raw": {
                  "type_url": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
                  "value": "CgxjbHVzdGVyMS1kbnOKAhcSFRITChEKDwoNEghkbnNfaG9zdBiQPxAC"
                }
              }
            ], security config: null  (t=+11.976479ms)
        tlogger.go:116: INFO server.go:220 [xds-e2e] Created new resource snapshot...  (t=+14.07051ms)
        tlogger.go:116: INFO simple.go:238 [xds-e2e] respond open watch 3 type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] with new version "3"  (t=+14.127011ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] version "2" with version "3"  (t=+14.167812ms)
        tlogger.go:116: INFO simple.go:238 [xds-e2e] respond open watch 4 type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] with new version "3"  (t=+14.219812ms)
        tlogger.go:116: INFO simple.go:428 [xds-e2e] respond type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] version "2" with version "3"  (t=+14.405115ms)
        tlogger.go:116: INFO server.go:226 [xds-e2e] Updated snapshot cache with resource snapshot...  (t=+14.450616ms)
        tlogger.go:116: INFO cluster_handler.go:193 [xds] [cds-lb 0xc0002f7080] CDS watch canceled on cluster1-eds  (t=+14.831022ms)
        tlogger.go:116: INFO cluster_handler.go:193 [xds] [cds-lb 0xc0002f7080] CDS watch canceled on cluster1-dns  (t=+14.872822ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] and known map[cluster1:{} cluster1-dns:{} cluster1-eds:{}]. Diff []  (t=+15.187227ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 5 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1-eds cluster1-dns cluster1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "3"  (t=+15.242228ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] and known map[service1:{}]. Diff []  (t=+15.304229ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 6 for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[service1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "3"  (t=+15.346529ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1 cluster1-dns] and known map[cluster1:{} cluster1-dns:{} cluster1-eds:{}]. Diff []  (t=+15.446631ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 7 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1 cluster1-dns] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "3"  (t=+15.489631ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] and known map[cluster1:{} cluster1-dns:{} cluster1-eds:{}]. Diff []  (t=+15.570033ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 8 for type.googleapis.com/envoy.config.cluster.v3.Cluster[cluster1] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "3"  (t=+15.609033ms)
        tlogger.go:116: WARNING cdsbalancer.go:285 [xds] [cds-lb 0xc0002f7080] Watch error from xds-client 0xc0007ac6f0: resource name "cluster1-dns" of type Cluster not found in received response  (t=+15.779636ms)
        tlogger.go:116: INFO clusterresolver.go:190 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Received new balancer config: {
              "discoveryMechanisms": [
                {
                  "cluster": "cluster1-eds",
                  "edsServiceName": "service1",
                  "outlierDetection": {}
                },
                {
                  "cluster": "cluster1-dns",
                  "type": "LOGICAL_DNS",
                  "dnsHostname": "dns_host:8080",
                  "outlierDetection": {}
                }
              ],
              "xdsLbPolicy": [
                {
                  "xds_wrr_locality_experimental": {
                    "childPolicy": [
                      {
                        "round_robin": {}
                      }
                    ]
                  }
                }
              ]
            }  (t=+15.905638ms)
        tlogger.go:116: WARNING clusterresolver.go:273 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Received error: resource name "cluster1-dns" of type Cluster not found in received response  (t=+15.988539ms)
        tlogger.go:116: INFO dns_resolver.go:296 [dns] dns: A record lookup error: lookup dns_host on 127.0.0.53:53: dial udp 127.0.0.53:53: operation was canceled  (t=+16.398345ms)
        tlogger.go:116: INFO clientconn.go:592 [core] [Channel #304] Channel Connectivity change to TRANSIENT_FAILURE  (t=+16.468746ms)
        tlogger.go:116: INFO balancer.go:76 [xds] [priority-lb 0xc0002d8a80] Created  (t=+16.531247ms)
        tlogger.go:116: INFO clusterresolver.go:248 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Built child policy config: {}  (t=+16.584948ms)
        tlogger.go:116: INFO clusterresolver.go:248 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Built child policy config: {}  (t=+16.648049ms)
        tlogger.go:116: INFO simple.go:359 [xds-e2e] nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7" requested type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[] and known map[service1:{}]. Diff []  (t=+16.808151ms)
        tlogger.go:116: INFO simple.go:380 [xds-e2e] open watch 9 for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment[] from nodeID "a8512e34-5f04-43e6-a768-1c8080b87fb7", version "3"  (t=+16.852852ms)
        cluster_handler_test.go:427: timeout when waiting for child policy to receive its configuration
        tlogger.go:116: INFO clientconn.go:592 [core] [Channel #304] Channel Connectivity change to SHUTDOWN  (t=+5.003732295s)
        tlogger.go:116: INFO balancer_conn_wrappers.go:191 [core] [Channel #304] ccBalancerWrapper: closing  (t=+5.003812996s)
        tlogger.go:116: INFO cluster_handler.go:193 [xds] [cds-lb 0xc0002f7080] CDS watch canceled on cluster1  (t=+5.003889997s)
        tlogger.go:116: INFO clusterresolver.go:337 [xds] [xds-cluster-resolver-lb 0xc0003f2d00] Shutdown  (t=+5.0040893s)
        tlogger.go:116: INFO cdsbalancer.go:439 [xds] [cds-lb 0xc0002f7080] Shutdown  (t=+5.0041387s)
        tlogger.go:116: INFO resolver_conn_wrapper.go:126 [core] [Channel #304] Closing the name resolver  (t=+5.004182601s)
        tlogger.go:116: INFO clientconn.go:318 [core] [Channel #304] Channel deleted  (t=+5.004246602s)
        tlogger.go:116: WARNING authority.go:417 [xds] [xds-client 0xc0007b0460] [127.0.0.1:42559] Watchers not notified since ADS stream failed after having received at least one response: rpc error: code = Canceled desc = context canceled  (t=+5.004371804s)
        tlogger.go:116: WARNING transport.go:497 [xds] [xds-client 0xc0007b0460] [127.0.0.1:42559] ADS stream closed: rpc error: code = Canceled desc = context canceled  (t=+5.00441[210](https://github.com/grpc/grpc-go/actions/runs/6138087235/job/16654609153?pr=6575#step:8:211)4s)
        tlogger.go:116: INFO clientconn.go:592 [core] [Channel #306] Channel Connectivity change to SHUTDOWN  (t=+5.004459305s)
        tlogger.go:116: INFO balancer_conn_wrappers.go:191 [core] [Channel #306] ccBalancerWrapper: closing  (t=+5.004503005s)
        tlogger.go:116: INFO resolver_conn_wrapper.go:126 [core] [Channel #306] Closing the name resolver  (t=+5.004561306s)
        tlogger.go:116: INFO clientconn.go:1338 [core] [Channel #306 SubChannel #307] Subchannel Connectivity change to SHUTDOWN  (t=+5.004617707s)
        tlogger.go:116: INFO clientconn.go:1693 [core] [Channel #306 SubChannel #307] Subchannel deleted  (t=+5.004667908s)
        tlogger.go:116: INFO clientconn.go:318 [core] [Channel #306] Channel deleted  (t=+5.00470[230](https://github.com/grpc/grpc-go/actions/runs/6138087235/job/16654609153?pr=6575#step:8:231)8s)
        tlogger.go:116: INFO clientimpl.go:88 [xds] [xds-client 0xc0007b0460] Shutdown  (t=+5.004756009s)
        tlogger.go:116: INFO server.go:806 [core] [Server #303 ListenSocket #305] ListenSocket deleted  (t=+5.0049[268](https://github.com/grpc/grpc-go/actions/runs/6138087235/job/16654609153?pr=6575#step:8:269)11s)
FAIL
FAIL	google.golang.org/grpc/xds/internal/balancer/cdsbalancer	8.949s
@easwars easwars self-assigned this Sep 11, 2023
@easwars easwars changed the title Flaky test: AggregatedClusterSuccess_SwitchBetweenLeafAndAggregate Flaky test: 1/10K: AggregatedClusterSuccess_SwitchBetweenLeafAndAggregate Oct 6, 2023
@easwars easwars added the P2 label Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@easwars easwars

Labels
P2 Type: Testing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@easwars @ginayeh