Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: linked/depends-on container restarting #1103

Merged
merged 3 commits into from
Jan 11, 2022
Merged

fix: linked/depends-on container restarting #1103

merged 3 commits into from
Jan 11, 2022

Conversation

piksel
Copy link
Member

@piksel piksel commented Oct 24, 2021

Restarting of dependency containers has been something that has seemed broken in watchtower for some time, but there hasn't been any easy way to reproduce/verify it.
So, the first part of this PR is an e2e test that actually tests whether the "implicit restarts" actually work:
https://asciinema.org/a/444312 (spoiler: they don't)

Unfortunately, I don't think running this test in CI is viable (or even possible), but for now it serves it's purpose and can be run locally when needed.

The second part is actually fixing the problem (mutating the actual array struct instead of a copy):
https://asciinema.org/a/444307


The test script creates two images which run a small webserver that only returns the image version number, and pushes them to a local docker registry (which can be created using scripts/du-cli.sh registry start). Then it creates two containers, with one of them being dependent on the other (through depends-on label or --linked) using their corresponding images.
Next, it creates another revision of the "parent" containers image and pushes it to the local registry and runs watchtower.
After watchtower has ran, it queries the two containers to see if the start time has changed (meaning that the container was at least restarted), and if the version number returned from the service inside the containers have changed (meaning that the container was updated).

Test script usage:

./scripts/dependency-test.sh <linked | label> [...watchtower args]

This will run watchtower by building it from the source. To instead run a released image in docker, set the environment variable WATCHTOWER_TAG to latest or latest-dev.
Example:

WATCHTOWER_TAG=latest ./scripts/dependency-test.sh linked

The first argument is whether the containers should be linked using --link or --label. All other arguments will be passed to the watchtower executable/container.

also adds the non-obvious slash prefix to link names if missing
@piksel piksel requested a review from simskij October 24, 2021 12:49
@codecov
Copy link

codecov bot commented Oct 24, 2021

Codecov Report

Merging #1103 (f921a70) into main (4a66a69) will increase coverage by 0.47%.
The diff coverage is 91.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1103      +/-   ##
==========================================
+ Coverage   61.96%   62.43%   +0.47%     
==========================================
  Files          22       22              
  Lines        1438     1443       +5     
==========================================
+ Hits          891      901      +10     
+ Misses        463      458       -5     
  Partials       84       84              
Impacted Files Coverage Δ
internal/actions/update.go 69.10% <91.66%> (+5.54%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a66a69...f921a70. Read the comment docs.

@jeff-h
Copy link

jeff-h commented Oct 25, 2021

It would be fantastic if dependent container restarting was reliable! :)

Just to clarify though, is this fix so that:

  • watchtower looks at docker-compose's depends_on (ie watchtower would need to look at all other containers and find any that depend on a restarting one)
  • or is it for watchtower's com.centurylinklabs.watchtower.depends-on label, which as I understand it should be specified on the container being restarted?

Or either/both?

@piksel
Copy link
Member Author

piksel commented Oct 25, 2021

It's for --link and watchtowers depends-on label. There is no added support for docker compose.

@piksel piksel changed the title Fix linked/depends-on container restarting fix: linked/depends-on container restarting Jan 11, 2022
@piksel piksel merged commit 084249c into main Jan 11, 2022
@piksel piksel deleted the fix/container-deps branch January 11, 2022 16:15
@simskij simskij mentioned this pull request Jan 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Containers using another container as the network fail when network container is updated
2 participants