Could not do a head request

[MrGibbage]

Two days ago I started getting an error message on one of my containers:

time="2023-01-21T13:33:59Z" level=warning msg="Could not do a head request for "meisnate12/plex-meta-manager:develop", falling back to regular pull." container=/plex-meta-manager image="meisnate12/plex-meta-manager:develop"
time="2023-01-21T13:33:59Z" level=warning msg="Reason: registry responded to head request with "404 Not Found", auth: "not present"" container=/plex-meta-manager image="meisnate12/plex-meta-manager:develop"

And if this helps, here is the email subject line:
Watchtower updates on 0d8200f24652

This container has been running fine (for months) and then this just started happening. Any ideas on what I should do? Ignore it? Is there somehting I should fix?

[piksel]

This is due to a new format of the manifest that is produced with the new version of docker buildx. The only consequence is that the docker API rate limiting will be applied to every check for an image update, so as long as you don't use a really short interval for watchtower it should be fine. In that case you can set the WATCHTOWER_WARN_ON_HEAD_FAILURE=never environment variable for watchtower, or add the equivalent argument --warn-on-head-failure never (see https://containrrr.dev/watchtower/arguments/#head_failure_warnings).

If you are responsible for creating said image, you can look at the upstream bug for ways to opt-out of the unsupported manifest until it's supported: docker/buildx#1509 (comment)

Known images that uses the unsupported manifest:

• koenkk/zigbee2mqtt:latest
• netdata/netdata:latest
• pihole/pihole:latest Fixed as of Unable to update docker (with watchtower) pi-hole/docker-pi-hole#1289 (comment)
• crowdsecurity/crowdsec:latest
• ttionya/vaultwarden-backup:latest
• vikunja/frontend:latest and vikunja/api:latest

[mateo08c]

Watchtower updates on 28cb6411b884
Could not do a head request for "vikunja/frontend:latest", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"
Could not do a head request for "vikunja/api:latest", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

[ramzh]

Could not do a head request for "nicolargo/glances:latest", falling back to regular pull.
Could not do a head request for "amir20/dozzle:latest", falling back to regular pull.

[TheRealKamisama]

Could not do a head request for "xhofe/alist:latest", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

Same here with xhofe/alist:latest, opened issue there.
alist-org/alist#3183

[marcelrv]

I experience exactly this issue with haugene/transmission-openvpn:latest
Maybe add that one to the list as well

@pkishino do you want me to create a issue in https://github.com/haugene/docker-transmission-openvpn/issues for it? (the template is not really favorable to reporting it...)

Could not do a head request for "haugene/transmission-openvpn:latest", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

[pkishino]

I experience exactly this issue with haugene/transmission-openvpn:latest Maybe add that one to the list as well

@pkishino do you want me to create a issue in https://github.com/haugene/docker-transmission-openvpn/issues for it? (the template is not really favorable to reporting it...)

Could not do a head request for "haugene/transmission-openvpn:latest", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

I don’t see a need for issue on our repo as it seems fixed here #1514 (comment)

[Choque]

@piksel What do you mean by "[...]as long as you don't use a really short interval for watchtower it should be fine[...]" ? I have it set up to run once per night and I get this error every day. Or is there another setting I can tweak which I'm unaware of?

That's from last night:

Reason: registry responded to head request with "404 Not Found", auth: "not present"
Could not do a head request for "ghcr.io/parkervcp/games:source", falling back to regular pull.
Reason: registry responded to head request with "429 Too Many Requests", auth: "not present"
Could not do a head request for "ghcr.io/paperless-ngx/tika:latest", falling back to regular pull.
Reason: registry responded to head request with "429 Too Many Requests", auth: "not present"
Could not do a head request for "ghcr.io/pterodactyl/panel:latest", falling back to regular pull.
Reason: registry responded to head request with "429 Too Many Requests", auth: "not present"
Found new grafana/grafana:latest image (a8cd27e6fc5a)
Stopping /grafana (22692444e2bb) with SIGTERM
Creating /grafana
Removing image 83f377cc32a0

[piksel]

It's not an error, it's a warning.

falling back to regular pull

It still checks if the container is updated, but it uses docker pull, which uses up the rate limit imposed by docker hub. If you have a really small interval, let's say 5 minutes, this will lead you to reach the limit after a while and prevents watchtower from working until the limit is renewed (every 6 hours).
See https://www.docker.com/increase-rate-limits/ for more information about the limits.

is there another setting I can tweak

You can disable the warning using WATCHTOWER_WARN_ON_HEAD_FAILURE=never

[kimpenhaus]

@piksel you have any eta for the next release containing the fix from PR #1533?

Thanks, m.

[mhellmeier]

@piksel : Since it worked previously, IMHO, I don't think disabling warnings is the best and most satisfactory solution.

[piksel]

@mhellmeier Either you keep the warning until the issue is resolved, or you don't care about the warning and disable it. The reason for why it started appearing was that repositories started using a new format that wasn't compatible with watchtower, and hence the warning. It's not something that got broken.

@kimpenhaus No ETA, since I do this on my spare time whenever I get time over, but I did manage to release it today.

[mhellmeier]

Great to hear that you are able to create a new fixed release in a short timeline. Thanks a lot for your effort, @piksel, really appreciate it!