Watchtower behaviour when container update breaks? #1262
-
|
Watchtower is intended to be used for unattended updates of running containers. And it performs great on that job. Yet it happened several times to me that a newly released container image - pulled and activated by Watchtower - did not come up correctly. And as it seems, Watchtower is proud to have done it's job but some service in my network is down. Not just that I have to identify that fact myself, I also have to analyze the root cause and often the solution is to go back to the last known-good container image. At least until yet another one gets released. Would it be possible to put such a workflow into Watchtower? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
We've been getting this feature request from time to time, and at this point in time our stance is that we're not looking to add it into Watchtower. There are Prometheus metrics endpoints both in Watchtower itself and in Docker, so getting enough metrics out to write a simple alert rule for Prometheus and Alertmanager warning you if something goes down should be fairly straight-forward. |
Beta Was this translation helpful? Give feedback.
-
|
Well, my suggestion was not to alert if a container goes down. It was more to take the action of restoring the previous version, together with firing off an alarm that the upgrade was not performed as planned. It is a huge difference if my containers get replaced in the middle of the night and Watchtower is able to cover for (simple) problems rather than having to wake me up for such a (simple) problem. This works out quite easily if containers provide a health check (see https://docs.docker.com/engine/reference/builder/#healthcheck), as it would show up in the container's health status, generating a health status event. To be more concise, the algorithm would look like this:
|
Beta Was this translation helpful? Give feedback.
We've been getting this feature request from time to time, and at this point in time our stance is that we're not looking to add it into Watchtower. There are Prometheus metrics endpoints both in Watchtower itself and in Docker, so getting enough metrics out to write a simple alert rule for Prometheus and Alertmanager warning you if something goes down should be fairly straight-forward.