Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestRouter integration flake #8502

Closed
smarterclayton opened this issue Apr 13, 2016 · 14 comments
Closed

TestRouter integration flake #8502

smarterclayton opened this issue Apr 13, 2016 · 14 comments
Assignees
Labels
component/routing kind/test-flake Categorizes issue or PR as related to test flakes. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/P2

Comments

@smarterclayton
Copy link
Contributor

failed  TestRouter
I0412 23:06:19.694953   23932 plugins.go:41] Registered credential provider "google-jwt-key"
I0412 23:06:19.695044   23932 plugins.go:41] Registered credential provider "google-dockercfg"
I0412 23:06:19.695054   23932 plugins.go:41] Registered credential provider "google-dockercfg-url"
I0412 23:06:19.695064   23932 plugins.go:41] Registered credential provider "google-container-registry"
I0412 23:06:19.695084   23932 plugins.go:62] Registered admission plugin "ExternalIPRanger"
I0412 23:06:19.695097   23932 plugins.go:62] Registered admission plugin "NamespaceLifecycle"
I0412 23:06:19.695552   23932 plugins.go:62] Registered admission plugin "SecurityContextDeny"
I0412 23:06:19.695574   23932 plugins.go:62] Registered admission plugin "ResourceQuota"
I0412 23:06:19.695584   23932 plugins.go:62] Registered admission plugin "PersistentVolumeLabel"
I0412 23:06:19.695598   23932 plugins.go:62] Registered admission plugin "NamespaceExists"
I0412 23:06:19.695608   23932 plugins.go:62] Registered admission plugin "NamespaceAutoProvision"
I0412 23:06:19.695622   23932 plugins.go:62] Registered admission plugin "LimitRanger"
I0412 23:06:19.696190   23932 plugins.go:62] Registered admission plugin "InitialResources"
I0412 23:06:19.696207   23932 plugins.go:62] Registered admission plugin "DenyEscalatingExec"
I0412 23:06:19.696215   23932 plugins.go:62] Registered admission plugin "DenyExecOnPrivileged"
I0412 23:06:19.696223   23932 plugins.go:62] Registered admission plugin "AlwaysDeny"
I0412 23:06:19.696232   23932 plugins.go:62] Registered admission plugin "AlwaysPullImages"
I0412 23:06:19.696241   23932 plugins.go:62] Registered admission plugin "AlwaysAdmit"
I0412 23:06:19.696257   23932 plugins.go:62] Registered admission plugin "SecurityContextConstraint"
I0412 23:06:19.696268   23932 plugins.go:62] Registered admission plugin "SCCExecRestrictions"
I0412 23:06:19.696288   23932 plugins.go:62] Registered admission plugin "PodNodeConstraints"
I0412 23:06:19.696303   23932 plugins.go:62] Registered admission plugin "RunOnceDuration"
I0412 23:06:19.696314   23932 plugins.go:62] Registered admission plugin "OriginResourceQuota"
I0412 23:06:19.696325   23932 plugins.go:62] Registered admission plugin "ClusterResourceOverride"
I0412 23:06:19.696339   23932 plugins.go:62] Registered admission plugin "ProjectRequestLimit"
I0412 23:06:19.696350   23932 plugins.go:62] Registered admission plugin "OriginPodNodeEnvironment"
I0412 23:06:19.696362   23932 plugins.go:62] Registered admission plugin "OriginNamespaceLifecycle"
I0412 23:06:19.696378   23932 plugins.go:62] Registered admission plugin "BuildByStrategy"
I0412 23:06:19.696390   23932 plugins.go:62] Registered admission plugin "BuildOverrides"
I0412 23:06:19.696402   23932 plugins.go:62] Registered admission plugin "BuildDefaults"
Started, serving at 172.18.12.210:8080
Started, serving at 172.18.12.210:8888
Started, serving at 172.18.12.210:8889
Started, serving TLS at 172.18.12.210:8443
I0413 03:06:21.326210       1 router.go:117] Creating a new template router, writing to /var/lib/containers/router
I0413 03:06:21.329259       1 router.go:159] Template router will coalesce reloads within 1 seconds of each other
I0413 03:06:21.329271       1 router.go:164] Reading persisted state
I0413 03:06:21.329786       1 router.go:168] Committing state
I0413 03:06:21.329953       1 router.go:161] Router is including routes in all namespaces
I0413 03:06:21.330005       1 controller.go:41] Running router controller
I0413 03:06:21.330013       1 reaper.go:17] Launching reaper
I0413 03:06:21.330642       1 router.go:225] Writing the router state
I0413 03:06:21.332353       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.332489       1 plugin.go:132] Processing 1 Endpoints for Name: example (ADDED)
I0413 03:06:21.332548       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:21.332630       1 plugin.go:144] Modifying endpoints for rotorouter/example
I0413 03:06:21.332941       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.333075       1 controller.go:80] Processing Route: example -> example
I0413 03:06:21.333087       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.333093       1 controller.go:82]            Event: ADDED
I0413 03:06:21.333130       1 unique_host.go:157] Route rotorouter/example claims www.example-unsecure.com
I0413 03:06:21.333139       1 status.go:162] has last touch <nil> for rotorouter/example
I0413 03:06:21.333171       1 status.go:252] admit: admitting route by updating status: example (true): www.example-unsecure.com
I0413 03:06:21.334004       1 plugin.go:172] Modifying routes for rotorouter/example
I0413 03:06:21.334322       1 router.go:230] Writing the router config
I0413 03:06:21.335525       1 router.go:235] Reloading the router
I0413 03:06:21.606725       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:21.606782       1 reaper.go:24] Signal received: child exited
I0413 03:06:21.633761       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.633879       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.633971       1 controller.go:80] Processing Route: example -> example
I0413 03:06:21.633981       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.633987       1 controller.go:82]            Event: DELETED
I0413 03:06:21.634005       1 unique_host.go:173] Deleting routes for rotorouter/example
I0413 03:06:21.634026       1 plugin.go:178] Deleting routes for rotorouter/example
I0413 03:06:21.634058       1 plugin.go:132] Processing 0 Endpoints for Name: example (MODIFIED)
I0413 03:06:21.634070       1 plugin.go:144] Modifying endpoints for rotorouter/example
I0413 03:06:21.634485       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.634607       1 controller.go:80] Processing Route: example-path -> example-path
I0413 03:06:21.634616       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.634621       1 controller.go:82]            Event: ADDED
I0413 03:06:21.634634       1 unique_host.go:157] Route rotorouter/example-path claims www.example-unsecure.com
I0413 03:06:21.634643       1 status.go:162] has last touch <nil> for rotorouter/example-path
I0413 03:06:21.634668       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.634681       1 status.go:252] admit: admitting route by updating status: example-path (true): www.example-unsecure.com
I0413 03:06:21.635307       1 plugin.go:168] Creating new frontend for key: rotorouter/example-path
I0413 03:06:21.635324       1 plugin.go:172] Modifying routes for rotorouter/example-path
I0413 03:06:21.635350       1 plugin.go:132] Processing 1 Endpoints for Name: example-path (ADDED)
I0413 03:06:21.635359       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:21.635395       1 plugin.go:144] Modifying endpoints for rotorouter/example-path
I0413 03:06:21.735783       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.735980       1 plugin.go:132] Processing 0 Endpoints for Name: example-path (MODIFIED)
I0413 03:06:21.735993       1 plugin.go:144] Modifying endpoints for rotorouter/example-path
I0413 03:06:21.736029       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.736165       1 controller.go:80] Processing Route: example-path -> example-path
I0413 03:06:21.736175       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.736180       1 controller.go:82]            Event: DELETED
I0413 03:06:21.736193       1 unique_host.go:173] Deleting routes for rotorouter/example-path
I0413 03:06:21.736203       1 plugin.go:178] Deleting routes for rotorouter/example-path
I0413 03:06:21.736461       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.736552       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.736596       1 plugin.go:132] Processing 2 Endpoints for Name: example-preferred-port (ADDED)
I0413 03:06:21.736607       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8889, Protocol:"TCP"}}}
I0413 03:06:21.736638       1 plugin.go:135]   Subset 1 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:21.736658       1 plugin.go:144] Modifying endpoints for rotorouter/example-preferred-port
I0413 03:06:21.736687       1 controller.go:80] Processing Route: example-preferred-port -> example-preferred-port
I0413 03:06:21.736695       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.736701       1 controller.go:82]            Event: ADDED
I0413 03:06:21.736714       1 unique_host.go:157] Route rotorouter/example-preferred-port claims www.example-unsecure.com
I0413 03:06:21.736724       1 status.go:162] has last touch <nil> for rotorouter/example-preferred-port
I0413 03:06:21.736763       1 status.go:252] admit: admitting route by updating status: example-preferred-port (true): www.example-unsecure.com
I0413 03:06:21.737498       1 plugin.go:172] Modifying routes for rotorouter/example-preferred-port
I0413 03:06:21.837846       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.837871       1 controller.go:80] Processing Route: example-preferred-port -> example-preferred-port
I0413 03:06:21.837880       1 controller.go:81]            Alias: www.example-unsecure.com
I0413 03:06:21.837885       1 controller.go:82]            Event: DELETED
I0413 03:06:21.837899       1 unique_host.go:173] Deleting routes for rotorouter/example-preferred-port
I0413 03:06:21.837909       1 plugin.go:178] Deleting routes for rotorouter/example-preferred-port
I0413 03:06:21.837938       1 plugin.go:132] Processing 0 Endpoints for Name: example-preferred-port (MODIFIED)
I0413 03:06:21.837948       1 plugin.go:144] Modifying endpoints for rotorouter/example-preferred-port
I0413 03:06:21.837846       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.838392       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:21.838512       1 plugin.go:132] Processing 1 Endpoints for Name: example-edge (ADDED)
I0413 03:06:21.838523       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:21.838569       1 plugin.go:144] Modifying endpoints for rotorouter/example-edge
I0413 03:06:21.838969       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:21.839082       1 controller.go:80] Processing Route: example-edge -> example-edge
I0413 03:06:21.839098       1 controller.go:81]            Alias: www.example.com
I0413 03:06:21.839103       1 controller.go:82]            Event: ADDED
I0413 03:06:21.839117       1 unique_host.go:157] Route rotorouter/example-edge claims www.example.com
I0413 03:06:21.839126       1 status.go:162] has last touch <nil> for rotorouter/example-edge
I0413 03:06:21.839166       1 status.go:252] admit: admitting route by updating status: example-edge (true): www.example.com
I0413 03:06:21.839822       1 plugin.go:172] Modifying routes for rotorouter/example-edge
I0413 03:06:22.329450       1 router.go:225] Writing the router state
I0413 03:06:22.333939       1 router.go:230] Writing the router config
I0413 03:06:22.337227       1 router.go:235] Reloading the router
I0413 03:06:22.371656       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:22.371741       1 reaper.go:24] Signal received: child exited
I0413 03:06:22.446667       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:22.446872       1 plugin.go:132] Processing 0 Endpoints for Name: example-edge (MODIFIED)
I0413 03:06:22.446893       1 plugin.go:144] Modifying endpoints for rotorouter/example-edge
I0413 03:06:22.447336       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:22.447495       1 controller.go:80] Processing Route: example-edge -> example-edge
I0413 03:06:22.447508       1 controller.go:81]            Alias: www.example.com
I0413 03:06:22.447514       1 controller.go:82]            Event: DELETED
I0413 03:06:22.447530       1 unique_host.go:173] Deleting routes for rotorouter/example-edge
I0413 03:06:22.447541       1 plugin.go:178] Deleting routes for rotorouter/example-edge
I0413 03:06:22.447966       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:22.448251       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:22.448400       1 plugin.go:132] Processing 1 Endpoints for Name: example-edge-path (ADDED)
I0413 03:06:22.448415       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:22.448469       1 plugin.go:144] Modifying endpoints for rotorouter/example-edge-path
I0413 03:06:22.448514       1 controller.go:80] Processing Route: example-edge-path -> example-edge-path
I0413 03:06:22.448525       1 controller.go:81]            Alias: www.example.com
I0413 03:06:22.448531       1 controller.go:82]            Event: ADDED
I0413 03:06:22.448540       1 unique_host.go:157] Route rotorouter/example-edge-path claims www.example.com
I0413 03:06:22.448546       1 status.go:162] has last touch <nil> for rotorouter/example-edge-path
I0413 03:06:22.448573       1 status.go:252] admit: admitting route by updating status: example-edge-path (true): www.example.com
I0413 03:06:22.449029       1 plugin.go:172] Modifying routes for rotorouter/example-edge-path
I0413 03:06:22.555036       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:22.555226       1 plugin.go:132] Processing 0 Endpoints for Name: example-edge-path (MODIFIED)
I0413 03:06:22.555242       1 plugin.go:144] Modifying endpoints for rotorouter/example-edge-path
I0413 03:06:22.555490       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:22.555903       1 plugin.go:132] Processing 1 Endpoints for Name: example-reencrypt (ADDED)
I0413 03:06:22.555917       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8443, Protocol:"TCP"}}}
I0413 03:06:22.555952       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt
I0413 03:06:22.556002       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:22.556119       1 controller.go:80] Processing Route: example-edge-path -> example-edge-path
I0413 03:06:22.556132       1 controller.go:81]            Alias: www.example.com
I0413 03:06:22.556138       1 controller.go:82]            Event: DELETED
I0413 03:06:22.556150       1 unique_host.go:173] Deleting routes for rotorouter/example-edge-path
I0413 03:06:22.556160       1 plugin.go:178] Deleting routes for rotorouter/example-edge-path
I0413 03:06:22.556193       1 certmanager.go:163] attempted to delete file /var/lib/containers/router/certs/rotorouter_example-edge-path.pem but it does not exist
I0413 03:06:22.556716       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:22.556863       1 controller.go:80] Processing Route: example-reencrypt -> example-reencrypt
I0413 03:06:22.556876       1 controller.go:81]            Alias: www.example.com
I0413 03:06:22.556881       1 controller.go:82]            Event: ADDED
I0413 03:06:22.556895       1 unique_host.go:157] Route rotorouter/example-reencrypt claims www.example.com
I0413 03:06:22.556905       1 status.go:162] has last touch <nil> for rotorouter/example-reencrypt
I0413 03:06:22.556945       1 status.go:252] admit: admitting route by updating status: example-reencrypt (true): www.example.com
I0413 03:06:22.557389       1 plugin.go:172] Modifying routes for rotorouter/example-reencrypt
I0413 03:06:23.329430       1 router.go:225] Writing the router state
I0413 03:06:23.329910       1 router.go:230] Writing the router config
I0413 03:06:23.333391       1 router.go:235] Reloading the router
I0413 03:06:23.358070       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.358263       1 controller.go:80] Processing Route: example-reencrypt -> example-reencrypt
I0413 03:06:23.358273       1 controller.go:81]            Alias: www.example.com
I0413 03:06:23.358289       1 controller.go:82]            Event: DELETED
I0413 03:06:23.358304       1 unique_host.go:173] Deleting routes for rotorouter/example-reencrypt
I0413 03:06:23.358315       1 plugin.go:178] Deleting routes for rotorouter/example-reencrypt
I0413 03:06:23.358625       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.358794       1 plugin.go:132] Processing 0 Endpoints for Name: example-reencrypt (MODIFIED)
I0413 03:06:23.358808       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt
I0413 03:06:23.364146       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.364273       1 controller.go:80] Processing Route: example-reencrypt-destcacert -> example-reencrypt-destcacert
I0413 03:06:23.364282       1 controller.go:81]            Alias: www.example.com
I0413 03:06:23.364287       1 controller.go:82]            Event: ADDED
I0413 03:06:23.364301       1 unique_host.go:157] Route rotorouter/example-reencrypt-destcacert claims www.example.com
I0413 03:06:23.364311       1 status.go:162] has last touch <nil> for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.364343       1 status.go:252] admit: admitting route by updating status: example-reencrypt-destcacert (true): www.example.com
I0413 03:06:23.364785       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.366958       1 plugin.go:168] Creating new frontend for key: rotorouter/example-reencrypt-destcacert
I0413 03:06:23.366983       1 plugin.go:172] Modifying routes for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.367010       1 plugin.go:132] Processing 1 Endpoints for Name: example-reencrypt-destcacert (ADDED)
I0413 03:06:23.367019       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8443, Protocol:"TCP"}}}
I0413 03:06:23.367051       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.370679       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:23.370719       1 reaper.go:24] Signal received: child exited
I0413 03:06:23.471221       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.471361       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.471406       1 plugin.go:132] Processing 0 Endpoints for Name: example-reencrypt-destcacert (MODIFIED)
I0413 03:06:23.471418       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.471451       1 controller.go:80] Processing Route: example-reencrypt-destcacert -> example-reencrypt-destcacert
I0413 03:06:23.471458       1 controller.go:81]            Alias: www.example.com
I0413 03:06:23.471464       1 controller.go:82]            Event: DELETED
I0413 03:06:23.471476       1 unique_host.go:173] Deleting routes for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.471485       1 plugin.go:178] Deleting routes for rotorouter/example-reencrypt-destcacert
I0413 03:06:23.471518       1 certmanager.go:163] attempted to delete file /var/lib/containers/router/cacerts/rotorouter_example-reencrypt-destcacert.pem but it does not exist
I0413 03:06:23.471926       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.472050       1 plugin.go:132] Processing 1 Endpoints for Name: example-reencrypt-path (ADDED)
I0413 03:06:23.472061       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8443, Protocol:"TCP"}}}
I0413 03:06:23.472093       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt-path
I0413 03:06:23.472467       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.472573       1 controller.go:80] Processing Route: example-reencrypt-path -> example-reencrypt-path
I0413 03:06:23.472585       1 controller.go:81]            Alias: www.example.com
I0413 03:06:23.472591       1 controller.go:82]            Event: ADDED
I0413 03:06:23.472604       1 unique_host.go:157] Route rotorouter/example-reencrypt-path claims www.example.com
I0413 03:06:23.472612       1 status.go:162] has last touch <nil> for rotorouter/example-reencrypt-path
I0413 03:06:23.472645       1 status.go:252] admit: admitting route by updating status: example-reencrypt-path (true): www.example.com
I0413 03:06:23.473315       1 plugin.go:172] Modifying routes for rotorouter/example-reencrypt-path
I0413 03:06:23.579951       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.580123       1 plugin.go:132] Processing 0 Endpoints for Name: example-reencrypt-path (MODIFIED)
I0413 03:06:23.580136       1 plugin.go:144] Modifying endpoints for rotorouter/example-reencrypt-path
I0413 03:06:23.580252       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.580378       1 controller.go:80] Processing Route: example-reencrypt-path -> example-reencrypt-path
I0413 03:06:23.580394       1 controller.go:81]            Alias: www.example.com
I0413 03:06:23.580400       1 controller.go:82]            Event: DELETED
I0413 03:06:23.580413       1 unique_host.go:173] Deleting routes for rotorouter/example-reencrypt-path
I0413 03:06:23.580423       1 plugin.go:178] Deleting routes for rotorouter/example-reencrypt-path
I0413 03:06:23.580453       1 certmanager.go:163] attempted to delete file /var/lib/containers/router/certs/rotorouter_example-reencrypt-path.pem but it does not exist
I0413 03:06:23.580475       1 certmanager.go:163] attempted to delete file /var/lib/containers/router/cacerts/rotorouter_example-reencrypt-path.pem but it does not exist
I0413 03:06:23.580912       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:23.580984       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:23.581040       1 plugin.go:132] Processing 1 Endpoints for Name: example-passthrough (ADDED)
I0413 03:06:23.581050       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8443, Protocol:"TCP"}}}
I0413 03:06:23.581088       1 plugin.go:144] Modifying endpoints for rotorouter/example-passthrough
I0413 03:06:23.581117       1 controller.go:80] Processing Route: example-passthrough -> example-passthrough
I0413 03:06:23.581124       1 controller.go:81]            Alias: www.example-passthrough.com
I0413 03:06:23.581130       1 controller.go:82]            Event: ADDED
I0413 03:06:23.581143       1 unique_host.go:157] Route rotorouter/example-passthrough claims www.example-passthrough.com
I0413 03:06:23.581153       1 status.go:162] has last touch <nil> for rotorouter/example-passthrough
I0413 03:06:23.581185       1 status.go:252] admit: admitting route by updating status: example-passthrough (true): www.example-passthrough.com
I0413 03:06:23.581859       1 plugin.go:172] Modifying routes for rotorouter/example-passthrough
I0413 03:06:24.329453       1 router.go:225] Writing the router state
I0413 03:06:24.329839       1 router.go:230] Writing the router config
I0413 03:06:24.331199       1 router.go:235] Reloading the router
I0413 03:06:24.361748       1 reaper.go:24] Signal received: child exited
I0413 03:06:24.361748       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:24.391000       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:24.391191       1 plugin.go:132] Processing 0 Endpoints for Name: example-passthrough (MODIFIED)
I0413 03:06:24.391205       1 plugin.go:144] Modifying endpoints for rotorouter/example-passthrough
I0413 03:06:24.391225       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:24.391336       1 controller.go:80] Processing Route: example-passthrough -> example-passthrough
I0413 03:06:24.391343       1 controller.go:81]            Alias: www.example-passthrough.com
I0413 03:06:24.391346       1 controller.go:82]            Event: DELETED
I0413 03:06:24.391355       1 unique_host.go:173] Deleting routes for rotorouter/example-passthrough
I0413 03:06:24.391361       1 plugin.go:178] Deleting routes for rotorouter/example-passthrough
I0413 03:06:24.391844       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:24.391962       1 controller.go:80] Processing Route: websocket-unsecure -> websocket-unsecure
I0413 03:06:24.391974       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:24.391980       1 controller.go:82]            Event: ADDED
I0413 03:06:24.391994       1 unique_host.go:157] Route rotorouter/websocket-unsecure claims 0.0.0.0
I0413 03:06:24.392004       1 status.go:162] has last touch <nil> for rotorouter/websocket-unsecure
I0413 03:06:24.392032       1 status.go:252] admit: admitting route by updating status: websocket-unsecure (true): 0.0.0.0
I0413 03:06:24.392434       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:24.392906       1 plugin.go:168] Creating new frontend for key: rotorouter/websocket-unsecure
I0413 03:06:24.392917       1 plugin.go:172] Modifying routes for rotorouter/websocket-unsecure
I0413 03:06:24.392943       1 plugin.go:132] Processing 1 Endpoints for Name: websocket-unsecure (ADDED)
I0413 03:06:24.392952       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:24.392980       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-unsecure
I0413 03:06:25.329418       1 router.go:225] Writing the router state
I0413 03:06:25.330948       1 router.go:230] Writing the router config
I0413 03:06:25.332304       1 router.go:235] Reloading the router
I0413 03:06:25.362864       1 reaper.go:24] Signal received: child exited
I0413 03:06:25.362935       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:25.393405       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:25.393428       1 controller.go:80] Processing Route: websocket-unsecure -> websocket-unsecure
I0413 03:06:25.393437       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:25.393442       1 controller.go:82]            Event: DELETED
I0413 03:06:25.393457       1 unique_host.go:173] Deleting routes for rotorouter/websocket-unsecure
I0413 03:06:25.393468       1 plugin.go:178] Deleting routes for rotorouter/websocket-unsecure
I0413 03:06:25.393499       1 plugin.go:132] Processing 0 Endpoints for Name: websocket-unsecure (MODIFIED)
I0413 03:06:25.393509       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-unsecure
I0413 03:06:25.393406       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:25.393985       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:25.394117       1 plugin.go:132] Processing 1 Endpoints for Name: websocket-edge (ADDED)
I0413 03:06:25.394135       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8888, Protocol:"TCP"}}}
I0413 03:06:25.394169       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-edge
I0413 03:06:25.394416       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:25.394523       1 controller.go:80] Processing Route: websocket-edge -> websocket-edge
I0413 03:06:25.394538       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:25.394544       1 controller.go:82]            Event: ADDED
I0413 03:06:25.394558       1 unique_host.go:157] Route rotorouter/websocket-edge claims 0.0.0.0
I0413 03:06:25.394567       1 status.go:162] has last touch <nil> for rotorouter/websocket-edge
I0413 03:06:25.394604       1 status.go:252] admit: admitting route by updating status: websocket-edge (true): 0.0.0.0
I0413 03:06:25.395298       1 plugin.go:172] Modifying routes for rotorouter/websocket-edge
I0413 03:06:26.329414       1 router.go:225] Writing the router state
I0413 03:06:26.329918       1 router.go:230] Writing the router config
I0413 03:06:26.331449       1 router.go:235] Reloading the router
I0413 03:06:26.362042       1 router.go:310] Router reloaded:
 - Checking HAProxy /healthz on port 1936 ...
 - HAProxy port 1936 health check ok : 0 retry attempt(s).
I0413 03:06:26.362091       1 reaper.go:24] Signal received: child exited
I0413 03:06:26.402118       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:26.402308       1 plugin.go:132] Processing 0 Endpoints for Name: websocket-edge (MODIFIED)
I0413 03:06:26.402322       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-edge
I0413 03:06:26.402434       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:26.402591       1 controller.go:80] Processing Route: websocket-edge -> websocket-edge
I0413 03:06:26.402601       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:26.402607       1 controller.go:82]            Event: DELETED
I0413 03:06:26.402621       1 unique_host.go:173] Deleting routes for rotorouter/websocket-edge
I0413 03:06:26.402632       1 plugin.go:178] Deleting routes for rotorouter/websocket-edge
I0413 03:06:26.402943       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:26.403061       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:26.403078       1 controller.go:80] Processing Route: websocket-passthrough -> websocket-passthrough
I0413 03:06:26.403087       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:26.403092       1 controller.go:82]            Event: ADDED
I0413 03:06:26.403106       1 unique_host.go:157] Route rotorouter/websocket-passthrough claims 0.0.0.0
I0413 03:06:26.403116       1 status.go:162] has last touch <nil> for rotorouter/websocket-passthrough
I0413 03:06:26.403154       1 status.go:252] admit: admitting route by updating status: websocket-passthrough (true): 0.0.0.0
I0413 03:06:26.403809       1 plugin.go:168] Creating new frontend for key: rotorouter/websocket-passthrough
I0413 03:06:26.403826       1 plugin.go:172] Modifying routes for rotorouter/websocket-passthrough
I0413 03:06:26.403853       1 plugin.go:132] Processing 1 Endpoints for Name: websocket-passthrough (ADDED)
I0413 03:06:26.403867       1 plugin.go:135]   Subset 0 : api.EndpointSubset{Addresses:[]api.EndpointAddress{api.EndpointAddress{IP:"172.18.12.210", TargetRef:(*api.ObjectReference)(nil)}}, NotReadyAddresses:[]api.EndpointAddress(nil), Ports:[]api.EndpointPort{api.EndpointPort{Name:"", Port:8443, Protocol:"TCP"}}}
I0413 03:06:26.403902       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-passthrough
I0413 03:06:26.511219       1 reflector.go:366] pkg/router/controller/factory/factory.go:59: Watch close - *api.Route total 1 items received
I0413 03:06:26.511423       1 controller.go:80] Processing Route: websocket-passthrough -> websocket-passthrough
I0413 03:06:26.511432       1 controller.go:81]            Alias: 0.0.0.0
I0413 03:06:26.511438       1 controller.go:82]            Event: DELETED
I0413 03:06:26.511451       1 unique_host.go:173] Deleting routes for rotorouter/websocket-passthrough
I0413 03:06:26.511461       1 plugin.go:178] Deleting routes for rotorouter/websocket-passthrough
I0413 03:06:26.511684       1 reflector.go:366] pkg/router/controller/factory/factory.go:66: Watch close - *api.Endpoints total 1 items received
I0413 03:06:26.511825       1 plugin.go:132] Processing 0 Endpoints for Name: websocket-passthrough (MODIFIED)
I0413 03:06:26.511837       1 plugin.go:144] Modifying endpoints for rotorouter/websocket-passthrough
Stopping listener at 172.18.12.210:8080
Stopping listener at 172.18.12.210:8888
Stopping listener at 172.18.12.210:8889
Stopping listener at 172.18.12.210:8443
Server message: accept tcp 172.18.12.210:8888: use of closed network connectionServer message: accept tcp 172.18.12.210:8080: use of closed network connectionStopping listener at 172.18.12.210:8080
Stopping listener at 172.18.12.210:8080
Stopping listener at 172.18.12.210:8888
Stopping listener at 172.18.12.210:8888
Stopping listener at 172.18.12.210:8889
Stopping listener at 172.18.12.210:8889
Stopping listener at 172.18.12.210:8443
Stopping listener at 172.18.12.210:8443
Server message: accept tcp 172.18.12.210:8443: use of closed network connectionServer message: accept tcp 172.18.12.210:8889: use of closed network connectionStopping listener at 172.18.12.210:8080
--- FAIL: TestRouter-2 (6.97s)
    router_test.go:351: TC reencrypt failed: Get https://0.0.0.0: dial tcp 0.0.0.0:443: connection reset by peer

from https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_origin/5581/consoleFull#112541649156bf4006e4b05b79524e5923

@smarterclayton smarterclayton added component/routing kind/test-flake Categorizes issue or PR as related to test flakes. labels Apr 13, 2016
@smarterclayton
Copy link
Contributor Author

This may be the go reuse of closed network transactions, i.e. a test side error vs a server side (if you are using pooled connections you can occasionally fail to get a real connection).

@smarterclayton
Copy link
Contributor Author

@knobunc
Copy link
Contributor

knobunc commented Aug 12, 2016

@marun can you take a look at this please?

@marun
Copy link
Contributor

marun commented Aug 12, 2016

Given that this hasn't been seen since May I don't see the value in investigating this further. If it reappears, we can reopen.

@marun marun closed this as completed Aug 12, 2016
@soltysh
Copy link
Member

soltysh commented Nov 25, 2016

Hit it in https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_integration/8777/consoleFull:

--- FAIL: TestRouter (7.72s)
	router_test.go:330: TC non-secure: url 0.0.0.0 alias www.example-unsecure.com protocol http
	router_test.go:330: TC non-secure-path: url 0.0.0.0/test alias www.example-unsecure.com protocol http
	router_test.go:330: TC preferred-port: url 0.0.0.0 alias www.example-unsecure.com protocol http
	router_test.go:330: TC edge termination: url 0.0.0.0 alias www.example.com protocol https
	router_test.go:330: TC edge termination path: url 0.0.0.0/test alias www.example.com protocol https
	router_test.go:330: TC reencrypt: url 0.0.0.0 alias www.example.com protocol https
	router_test.go:332: TC reencrypt failed: Get https://0.0.0.0: dial tcp 0.0.0.0:443: getsockopt: connection reset by peer
	router_test.go:330: TC reencrypt-destcacert: url 0.0.0.0 alias www.example.com protocol https
	router_test.go:330: TC reencrypt path: url 0.0.0.0/test alias www.example.com protocol https
	router_test.go:330: TC passthrough termination: url 0.0.0.0 alias www.example-passthrough.com protocol https
	router_test.go:330: TC websocket unsecure: url 0.0.0.0 alias www.example.com protocol ws
	router_test.go:330: TC ws edge termination: url 0.0.0.0 alias www.example.com protocol wss
	router_test.go:330: TC ws passthrough termination: url 0.0.0.0 alias www.example.com protocol wss
FAIL

Reopening.

@soltysh soltysh reopened this Nov 25, 2016
@soltysh
Copy link
Member

soltysh commented Nov 25, 2016

But I'm lowering the prio to p2, since it's not permanent or anything.

@0xmichalis
Copy link
Contributor

A different failure mode for this test:

--- FAIL: TestRouter (35.99s)
	router_test.go:1229: timed out attempting to send to channel: {"type":"ADDED","object":{"kind":"Endpoints","apiVersion":"v1","metadata":{"name":"example","namespace":"rotorouter","creationTimestamp":null},"subsets":[{"addresses":[{"ip":"172.18.12.182"}],"ports":[{"port":8888}]}]}}
FAIL

https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_integration/9333/consoleFull#-77443714456bf4006e4b05b79524e5923

@0xmichalis
Copy link
Contributor

Seems like it's flaking a lot today, increasing priority.

@0xmichalis
Copy link
Contributor

Seems like the latest failure is tracked in #12236 - resetting priority for this.

@rootfs
Copy link
Member

rootfs commented May 10, 2017

hit this again in https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_integration/1927/consoleFull#165939508756bf4006e4b05b79524e5923

=== RUN   TestRouter
--- FAIL: TestRouter (12.17s)
	router_test.go:355: TC non-secure: url 0.0.0.0 alias unsecure.example.com protocol http
	router_test.go:355: TC non-secure-path: url 0.0.0.0/test alias unsecure.example.com protocol http
	router_test.go:355: TC preferred-port: url 0.0.0.0 alias pref.port.unsecure.example.com protocol http
	router_test.go:355: TC edge termination: url 0.0.0.0 alias edge.example.com protocol https
	router_test.go:355: TC edge termination path: url 0.0.0.0/test alias edge.example.com protocol https
	router_test.go:355: TC reencrypt: url 0.0.0.0 alias reencrypt.example.com protocol https
	router_test.go:355: TC reencrypt-InsecureEdgePolicy: url 0.0.0.0 alias allow.reencrypt.example.com protocol http
	router_test.go:355: TC reencrypt-destcacert: url 0.0.0.0 alias destcert.reencrypt.example.com protocol https
	router_test.go:357: TC reencrypt-destcacert failed: Get https://0.0.0.0: dial tcp 0.0.0.0:443: getsockopt: connection reset by peer
	router_test.go:363: You may need to add an entry to /etc/hosts so that the hostname of the router (destcert.reencrypt.example.com) resolves its the IP address, (0.0.0.0).
	router_test.go:355: TC reencrypt path: url 0.0.0.0/test alias reencrypt.example.com protocol https
	router_test.go:355: TC passthrough termination: url 0.0.0.0 alias passthrough.example.com protocol https
	router_test.go:355: TC websocket unsecure: url 0.0.0.0 alias ws.unsecure.example.com protocol ws
	router_test.go:355: TC ws edge termination: url 0.0.0.0 alias ws.edge.example.com protocol wss
	router_test.go:355: TC ws passthrough termination: url 0.0.0.0 alias ws.passthrough.example.com protocol wss
FAIL

@0xmichalis
Copy link
Contributor

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci-robot openshift-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 15, 2018
@openshift-bot
Copy link
Contributor

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci-robot openshift-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Mar 18, 2018
@openshift-bot
Copy link
Contributor

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/routing kind/test-flake Categorizes issue or PR as related to test flakes. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/P2
Projects
None yet
Development

No branches or pull requests

9 participants