Brief window of routing traffic to container before it's ready

[nokjuh]

• Service Fabric publishes container instance endpoints immediately as the instance starts, but before the application in the container is ready to handle traffic. This will cause Traefik to start routing requests there, even though they will fail. Traefik health check will then notice that the instance is not available, and stop routing traffic there until the instance is available.
• This causes about 5 to 30 second period when some requests may go to the new instance before it's ready, and will fail.
• This probably only causes issues with new connections, since existing connections will have sticky session cookie pointing to another instance.
• Fixing this would require Service Fabric to publish endpoints only when the instance is ready. This info is available to Service Fabric through Docker health checks (in use in our containers) and a Readiness Check (not currently enabled in our containers). However, neither of those will currently prevent the endpoints from being published. Microsoft received our feedback on Azure Connection Programs Yammer about the Readiness Check not working as expected, but did not promise anything.
• Alternatively Traefik could change its behavior to not assume new endpoints are healthy, if not as default then at least as an option. There is an open feature request for this, opened in 2019: traefik/traefik#4544