Watcher becomes unresponsive but no events

[gaberoffman]

I found that my watchers on prefixes were becoming unresponsive but not emitting "disconnected" or "error" or any event for that matter. It's very hard to reproduce but in my testing, I now have one container updating the value of a key on etcd every twenty minutes. I created six containers that each watch with a prefix of that key. Over the course of a few hours, one or two of those containers' watchers will stop seeing the put events.

Have you seen this behavior happen and/or are there suggestions for how to mitigate this or get better logging on why this might be happening?

[connor4312]

It's possible that etcd is closing the watch stream with a graceful end, I notice that we don't watch for that event. Glancing through etcd's code I don't see anywhere this is done intentionally, but there might be some hook in there or some network proxy that's fiddling with something. I'll add the end in 0.2.10, let me know if you hit issues with that.