[Feature request]: Consider impact of https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable

[cgwalters]

Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Suggestion

Hi 👋

In ostree upstream we have an effort to build sophisticated bridging to/from the OCI (Docker) container ecosystem. There's a whole bunch of parts to this, and (like a lot of ostree parts that I work on) tends to be oriented towards the "bootable host" part. But - all of the code here is explicitly designed to also work in a flatpak-style use case. Specifically for example, the logic in https://docs.rs/ostree-ext/latest/ostree_ext/container/index.html could definitely be used by flatpak I think.

One big benefit to all of this is that via https://github.com/containers/containers-image-proxy-rs we get a robust bridge into the containers Go code with battle-tested support for things like interacting with various registries, mirroring, signing, etc. Signing is a big one - with this approach flatpaks could be e.g. signed via Sigstore/cosign too!

All of the libostree core logic today will continue to be maintained for years to come (we have tons of users outside of flatpak too; a ton in the embedded space), so no immediate change is required.

However, it'd be good to think about how flatpak could make use of some of this new functionality.

A note: most of this new code is in Rust.

[ramcq]

@owtaylor Has anyone over in Fedora Workstation land thought about how this ostree OCI stuff interacts with the flatpak OCI stuff?

[owtaylor]

This really hasn't been coordinated as much as would be good - I haven't had the bandwidth personally to review, provide feedback, etc and I think Colin has been pretty focused on getting something working for needs of his immediate users.

As far as I know, the basic status is:

• The details of how it is done for Colin's work is not the same as how Flatpak handles it - they aren't compatible formats. Without having looked at how ostree is handling things in detail, I suspect there would be no inherent problem with switching Flatpak over to the ostree "native" approach, but there would be considerable work to make the transition smooth.
• The ostree "native" approach doesn't have any provision for delta compression. The solution proposed in container: support splitting inputs ostreedev/ostree-rs-ext#69 are not going to be as good as the current bsdiff based solution for the Flatpak use case.
• There is no solution to the "index" problem on the ostree side - we'd still have to do something like what we're doing now with an external JSON index.

[cgwalters]

The details of how it is done for Colin's work is not the same as how Flatpak handles it - they aren't compatible formats. Without having looked at how ostree is handling things in detail, I suspect there would be no inherent problem with switching Flatpak over to the ostree "native" approach, but there would be considerable work to make the transition smooth.

Right. The most obvious difference here is that the flatpak OCI images aren't executable with podman run, and that gets into a whole thing about dynamic versus static linking of app versus base image. Tangentially, but now that I look I hadn't realized before how much metadata flatpak is exporting for the images. (The AppStream XML-in-JSON is kind of funny...)

But, note that it'd be possible to switch over actually to just using e.g. containers-image-proxy-rs (or, I guess more realistically: directly speaking to skopeo experimental-image-proxy via C - it's a simple "JSON over socketpair() with pipe() fd passing for data") which would bring in all the support for image signatures, mirroring etc. without changing any of the image formats.

The ostree "native" approach doesn't have any provision for delta compression. The solution proposed in ostreedev/ostree-rs-ext#69 are not going to be as good as the current bsdiff based solution for the Flatpak use case.

Right, but that doesn't apply for the OCI path today, does it?

There is no solution to the "index" problem on the ostree side - we'd still have to do something like what we're doing now with an external JSON index.

Yes, application discovery and indexing/search is its own whole other world. There's also parallel work going on (AFAIK) around OLM.

[ramcq]

Right. The most obvious difference here is that the flatpak OCI images aren't executable with podman run, and that gets into a whole thing about dynamic versus static linking of app versus base image.

Is there some kind of jedi mind trick we can apply here so that a Flatpak runtime actually becomes a layer with some metadata? Like rather than the OCI Flatpaks being "rooted" at /app or /usr they are actually rooted at / (ie the usr and app folder is included inside the image) so that podman is able to layer them, but Flatpak has some side-signal and sees that this layer (and its dependents) is actually a runtime and skips those layers when importing?

I guess it doesn't quite work with extensions which lack an easy parallel in OCI-land, because each Flatpak application or runtime can mount them at arbitrary paths - they don't have presumed paths like /app and /usr. But it would certainly open up the field to a lot of tooling to produce Flatpaks more easily. Like BuildStream produces the fd.o SDK as an OCI image, I really like this direction of travel for CI flows that an SDK can be an OCI image, that you could build/compose/etc things with Docker/podman tooling, and the difference between that and a Flatpak is metadata.