[WIP]Delete corrupt exist lower layer when pull image #42978
+64
−6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Related ISSUE #35747 |
zvier
changed the title
zvier
changed the title
zvier
changed the title
zvier
changed the title
zvier
force-pushed
the
fix_corrupt_layer
branch
from
1157ad9
to
7f77f97
Compare
|
@thaJeztah PTAL |
zvier
changed the title
thaJeztah
added
area/storage/overlay
kind/bugfix
cpuguy83
requested changes
Nov 18, 2021
| @@ -398,10 +398,24 @@ func (ls *layerStore) Get(l ChainID) (Layer, error) { | |||
| if layer == nil { | |||
| return nil, ErrLayerDoesNotExist | |||
| } | |||
| if !ls.driver.Exists(layer.cacheID) { | |||
| if err := ls.deleteCorruptLayer(layer); err != nil { | |||
There was a problem hiding this comment.
Even though the layer itself is corrupt, deleting it at this level doesn't really seem right.
For drivers like btrfs this especially wouldn't work (granted it seems like it would be weirder to get into this situation with btrfs).
There was a problem hiding this comment.
It seems like if we want to recover from this we need a way to repair the layer, but this seems non-trivial from a more generic standpoint.
zvier
changed the title
zvier
force-pushed
the
fix_corrupt_layer
branch
3 times, most recently
from
fa073f7
to
87bb081
Compare
zvier
force-pushed
the
fix_corrupt_layer
branch
3 times, most recently
from
9b269f2
to
acbac61
Compare
zvier
force-pushed
the
fix_corrupt_layer
branch
5 times, most recently
from
ae9540d
to
87a2fbb
Compare
If lower layers corrupt with disk full, host or daemon crash or other reasons, may let the layer link and lower file empty in the disk, which can lead new image pull also get an invalid lower file, typically image lower file's content ends with an colon. Despit the image pull success, but can not inspect with error message "Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument". So, when pull image check exist layer, can add layer valid check logic. If cause a corrupt layer, delete and pull register it again. Signed-off-by: Jeff Zvier <zvier20@gmail.com>
zvier
force-pushed
the
fix_corrupt_layer
branch
from
87a2fbb
to
f2d4cbb
Compare
|
Hi, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If layer corrupt with disk full, host or daemon crash or some other reasons,
may let the layer link and lower file empty in the disk, which can lead new
image pull also get lower file with invalid content, typically image lower file's
content ends with an colon character. Despit the image pull success, but can not inspect
with error message "Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument".
So, when pull image check exist layer, can add layer valid check logic. If
cause a corrupt layer, delete it and pull it again.
Signed-off-by: Jeff Zvier zvier20@gmail.com