stable on-disk format

[alban]

When users create pods with rkt version n, the next version n+1 must be able to run them.

I don't know if we need to implement anything for this. Can you think of anything we miss for that?

• add tests to check that a pod created on a previous version can be started. A functional test could download rkt-v0.7.0.tar.gz from internet and prepare the pod with rkt-0.7.0 prepare .... Then, it would run it with rkt-current run-prepared ...
• add documentation about that stable on-disk format. doc: stable on-disk format #1271
• proper error message when a pod is invalid "rkt list" error after upgrading to 0.7.0 #1140. Fix in Use last-ditch manifest parser for error reporting #1266 / Use last-ditch manifest parser for error reporting #1471, requires appc/spec release

[alban]

/cc @philips : do you have an idea already how to proceed with this?

[eyakubovich]

I think it's critical for images/CAS to be really stable. Stability of prepared and exited containers is desirable but it also needs to have the flexibility to evolve. If I upgrade rkt, I'd hate to have to purge all my downloaded images but re-running prepare is not bad. However rkt run-prepared should be able to error out with appropriate message.

The situation with GC is more complex. If I upgrade rkt, I still need to be able to GC old exited pods. When GC was just rm -rf $POD_DIR, things were easy. Now that we have gc stage1 entry point and overlayfs unmounting, things are tricky.

[alban]

Good point about the CAS. Should we introduce a format version for things like /var/lib/rkt/cas/db in order to detect that it has been written by a future format?

At the moment, GC belongs to stage1. And we will have a stage1 based on lkvm soon. So we might have some pods started with the traditional stage1 and some pods started with the lkvm stage1. When using rkt gc, do we use the stage1/gc used to prepare/run the pod, or do we use the stage1/gc as specified in the option rkt gc --stage1-image=? It seems we need to do the former, so stage1/gc knows the correct method to GC the pod.

What should we do with valid images according to an old appc/spec but now invalid due to the spec change? See #1140.

[iaguis]

We already have a database schema migration (#706) and we used it successfully for #1181 (thanks @sgotti!).

So to ensure stability for the CAS we need to make sure we don't remove any of the other directories (/var/lib/rkt/cas/{blob,imagelocks,imageManifest,tree,treestorelocks}) or make any destructive changes to them.

For garbage-collecting older pods, we need to be careful and have the right fallbacks. For example, we try to unmount the overlay fs but if it's already unmounted we continue the process.

For things like #1140 (failed parsing because of a spec update) we discussed having a permissive Pod/Image manifest parser that we'd call if the normal parser fails. In this way, we can get enough infomation to give a good error message ("appName", "podUUID", "acVersion"...). This parser should probably live in appc/spec.

[krnowak]

@iaguis, @alban - a last-ditch parser to review for you - appc/spec#476

[alban]

Documentation was added via #1271. The rest is pending on appc/spec. Moving to next milestone.

[alban]

It seems the only remaining task is #1471 (last-ditch manifest parser).

[jonboulle]

#1471 landed so I think we can tentatively call this closed. Is the doc up to date?

[alban]

Is the doc up to date?

Yes, I don't think it changed recently. The documentation is in on-disk-format.md.

Closing.