container image pruning needs to handle client layering #4185
Comments
|
Is deployment 0 the booted one? Can you paste the output of |
|
At the moment yes, 0 is the booted one. Both deployments in the same state.
|
|
New error after downgrading rpm-ostree, not sure if it's a good clue.
Notice: I wonder if that extra |
|
Hmm, seems like it may have something to do with layered packages, but I'm not reproducing this in a basic test. Hmmm wait...I wonder if this is a bug from the new GC logic. We do expect this error after downgrading though, because we need e2aee76 to handle pruning right. |
That's the canonical rendering of |
|
Does |
|
sadly no |
|
This issue relates to #3874 in that we really really should support the daemon starting while in a partially broken state, so one can use upgrade and other commands to try to fix things. |
|
Hmm...I think we could get this if we somehow pruned the layer incorrectly. What was the output of that command? I wonder if we somehow incorrectly pruned a layer. Does it work if you do ? |
cgwalters
added a commit
to cgwalters/ostree-rs-ext
that referenced
this issue
Dec 5, 2022
To aid debugging coreos/rpm-ostree#4185
cgwalters
changed the title
|
OK I edited the title+description here with my current theory |
|
With the debug patches: |
cgwalters
added a commit
to cgwalters/ostree-rs-ext
that referenced
this issue
Dec 5, 2022
Closes: coreos/rpm-ostree#4185 Basically, with package layering or client side commits, rpm-ostree generates synthetic refs under `rpmostree/base` which point to the base commits. But this interacts badly with our image pruning logic. I'm happy about the dependency inversion here, but this is just a band-aid until we have time to think about a more proper fix. Basically there's two dynamic "layers" on top of core ostree going on here - rpm-ostree and container images and we need to figure out a general fix.
|
OK this fixes it for me ostreedev/ostree-rs-ext#432 |
|
OK in order to unwedge your current system let's try out this hacky patch for rpm-ostree: With this, the daemon will start for me and I can |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Updated description of bug from @cgwalters
The logic we landed in https://github.com/ostreedev/ostree-rs-ext/blob/1584320a1c9ab806e0346733db2725b3466430ec/lib/src/container/store.rs#L978 only considers the base ostree deployment roots. But - when rpm-ostree client side layering is involved, it generates e.g.
rpmostree/base/0- and the ostree pruning logic ignores those!So basically rebasing or upgrading when we're using the container flow may end up pruning an image.
I think the fix may be in the ostree-ext side: we should iterate all refs except the image ones, and use those as strong roots.
Original report:
After rebasing to a custom layered image with:
rpm-ostreed fails to start.
rpm-ostree version:
To reproduce:
Rebase to a custom image and reboot a few times? I am not sure yet what triggers it. System looks ok after a single reboot but at some point it gets to this state.
The text was updated successfully, but these errors were encountered: