Fix goroutine leak in consulcatalog when consul is down

[yutopp]

What does this PR do?

Fix goroutine leaks in consulcatalog provider. Use sync.Once to avoid blocking when sending errors to the channel which already used and an owner function of it is already exited.

Below is a number of goroutines by metrics data.

# Before patch
> date && curl -s http://localhost:10080/metrics | grep go_goroutines             
Fri Mar 16 19:32:28 JST 2018
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 28

> sleep 120 && date && curl -s http://localhost:10080/metrics | grep go_goroutines
Fri Mar 16 19:34:29 JST 2018
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 54

# After patch
> date && curl -s http://localhost:10080/metrics | grep go_goroutines             
Fri Mar 16 18:49:38 JST 2018
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 21

> sleep 120 && date && curl -s http://localhost:10080/metrics | grep go_goroutines
Fri Mar 16 18:51:39 JST 2018
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 21

(traefik command is ./dist/traefik --consulcatalog --entryPoints=Name:http Address::18080 --web.address=:10080 --web.metrics --web.metrics.prometheus --web.metrics.prometheus.entrypoint=traefik, and consul is not started)

Motivation

Assume there are 2 goroutines are spawned and they have the same error channel. If one goroutine sends an error to the channel, the caller returns by getting an error from the channel. Then the other goroutine will stuck when also trying to send an error to the channel because an owner of the error channel is already exited.

Additional Notes

[juliens]

Why not add 1 buffer on this chan instead of the sync.Once?

[yutopp]

Adding buffers will reduce maintainability because a number of capacity is depended on the number of gorountines which spawned at other functions. By using sync.Once, it will no longer need to worry about thinking the number of the buffer and blocking of the channel.

[juliens]

ok for me.

We need this bug fix in 1.5, can you do this PR on branch v1.5 ?

[yutopp]

Changed base branch to v1.5 and patched the fix to it.

[juliens]

LGTM

[ldez]

LGTM

[mmatur]

LGTM