-
Notifications
You must be signed in to change notification settings - Fork 640
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Green: Initial installation fails with "Can't install ghcr.io/home-assistant/green-homeassistant:2023.11.3" (Not Found) #4738
Comments
Trying to pull the image while having this corruption actually leads to the "Not Found" error from Docker. So we probably can't improve error handling much here 😢
|
Further investigation on my corrupted device I've concluded that Docker is not able to recovery from this situation. It seems that Docker isn't really aware of the (partially) downloaded image, and the corrupted layer/fragments are just stuck in the overlay2 storage. Downloading any image which has layers based on those corrupted layers fails. ℹ️ ℹ️ ℹ️ That being said, some technical details: The corruption seems to be a known issue in upstream Docker, see moby/moby#42964. With hints from the above issue, I was able to come up with a script which clears the corrupted layer metadata/cache ids: The script can be used from the Home Assistant OS shell available on port 22222 (see SSH access to the host). With the following command it can be executed on the shell directly: sh <(curl -fsSL https://gist.githubusercontent.com/agners/c5ae781c828dbf76c4cc1e4588c034a5/raw/bcb623cc2fa5da0a6864b9c72e2ac8ccbe9316c3/prune-corrupted-layer-metadata.sh) It will print something like:
ℹ️ After running the script a system reboot is necessary! There is a bugfix for this problem merged upstream, see moby/moby#46703. This is part of Docker 24.0.7, which will be in Home Assistant OS 11.2 (see home-assistant/operating-system#2883). However, this fix does not address already corrupted images/layers! So that means if you are in this state, HAOS 11.2 will not fix your problem. But future updates, even in the face of power outages should no longer lead to such corruptions. While the error appears in Supervisor, this is mostly a Operating System issue. But from Supervisor side not a lot can be done here. Since a mitigation is documented, and a fix to make this issue not occur anymore will be in HAOS 11.2, I close this as resolved. |
Unfortunately didn't work for me, I got: Error: Directory /mnt/data/docker/overlay2 does not exist. |
Describe the issue you are experiencing
Starting a Home Assistant Green, then plugging out power ~3 minutes in, then starting the unit again leads to situation where Home Assistant Green seems not to be able to recover from:
What type of installation are you running?
Home Assistant OS
Which operating system are you running on?
Home Assistant Operating System
Steps to reproduce the issue
Anything in the Supervisor logs that might be useful for us?
System Health information
N/A
Supervisor diagnostics
No response
Additional information
Kernel continuously reports this:
Manual pull fails likewise:
Pulling after the command still fails.
Also, the image is not yet listed in the Docker storage:
The text was updated successfully, but these errors were encountered: