New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

β™²πŸ“¦ Implement rpm-ostree rojig #1081

Open
cgwalters opened this Issue Nov 3, 2017 · 23 comments

Comments

Projects
None yet
3 participants
@cgwalters
Member

cgwalters commented Nov 3, 2017

Update 2018-02-22: This effort has been renamed to "rojig" to clarify the distinction from the original "jigdo" project.


Fetching content via both ostree and libdnf (RPM) ends up mixing the tradeoffs of both, requires release engineering to manage both, and makes it harder to mirror content. Not to mention the fact that there's the whole OCI/Docker thing which also works in a completely different way, and admins need to manage/mirror that too.

Now while the "obvious" thing would be to try to align with OCI in some way, the complete lack of wire deltas there is very problematic for uses outside of server clusters, and given that we already have lots of extensive linkage to RPM via libdnf, it makes the most sense to move that direction. Another reason for rojig is to address the package layering cadence issue.

Hence, let's experiment with doing ostree-in-RPM, basically the "compose" process injects additional data (SELinux labels for example) in an "ostree image" RPM, like fedora-atomic-host-27.8-1.x86_64.rpm. That "ostree image" RPM will have extra data, including information of the "rojig set" of the other RPMs that are needed to download - basically we only download changed RPMs, like a traditional yum system. But SELinux labeling, depsolving, etc. are still done server side, and we still have a reliable OSTree commit checksum.

We're adopting the idea of Jigdo, hence the terminology. Though to be clear, much of the design and all of the implementation is fundamentally different, so we call this "rojig".

First PR: #1103

Current status: Proof of concept code works: we can take a set of RPMs + extra data (postprocessing), generating an OSTree commit (with reliable checksum), then convert that back into a "rojig set", which is the original RPMs plus a new "rojig RPM". Then a client side can take the "rojig set", just transferring RPM on the wire, reassembling bit for bit the original OSTree commit.

External discussion:

Followup issues:

External work trackers:


Implementation: server side

When doing a compose tree, import the packages, and after import check out the final tree and SELinux relabel the imports so that they're reliably updated (currently depends on some unified core 🌐 work). Then, we can scan the pkg trees for objects, and find net-new objects that need to go in the oirpm.

I implemented that pretty quickly and hit on the issue that we have the initramfs 3 times in the tree (due to selinux labels), so I then implemented code to detect that; while we have 3 copies on disk, we can easily avoid that on the wire.

The big fundamental issue is SELinux - without that this would be way easier. The basic problem there is that we need to carry the SELinux labels in the oirpm, because the packages aren't labeled. So we need to carry a mapping from (pkg, file) β†’ xattrs in the oirpm, and (on the client side) apply those when unpacking.

Implementation: client side

  • Find OIRPM name (ostree origin: rpmostree-rojig:fedora-atomic-host)
  • Look in repo for new NVR of OIRPM with that name (question: do we want to somehow pin to origin repo?)
  • Download OIRPM, which contains list of packages to import (note this is a subset of the actual rpmdb due to e.g. emacs-filesystem and rootfiles etc., though we could just import all of them)
  • Import each referenced package, using xattrs from OIRPM map
  • Import net-new objects from OIRPM (commit,dirtree,dirmeta)
  • All of the above replaces ostree pull - after this we proceed with the rest of the logic ideally unchanged

Task list

  • Determine a story for GPG; pull the key from the "origin rpm-md repo"
  • Client side UI design
  • Client side implementation

@cgwalters cgwalters changed the title from Experiment with ostree-in-RPM to Experiment with ostree-in-RPM (AKA rpm-ostree jigdo) Nov 3, 2017

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 3, 2017

First, let's use the term rojigRPM for the special "ostree image RPM". I think we can have this be one command that replaces compose, except note it doesn't take --repo since it doesn't write to an ostree repo (at least not a persistent one):

rpm-ostree compose jigdo --oirpm fedora-atomic-host-27.3-1 fedora-atomic/fedora-atomic.json

But for supportability reasons, I think we need both rpmostree-jigdo and "traditional" ostree repos (can I call that traditional yet?). Also implementation wise, I think it will be easier to reason about things if this is implemented as a magical postprocessing step given an ostree commit. Hence:

$ rpm-ostree compose tree --repo=repo-build fedora-atomic/fedora-atomic.json
fedora/27/x86_64/atomic-host ➭ 54c9c878ffdbf96a6773d920f8b6dc89751c1ad1b8297a6efa3f41ca5ede0142
$ rpm-ostree compose commit2jigdo  --repo=repo-build  --oirpm fedora-atomic-host-27.3-1 54c9c878ffdbf96a6773d920f8b6dc89751c1ad1b8297a6efa3f41ca5ede0142
Wrote: fedora-atomic-host-27.3-1.x86_64.rpm

or so?

And then we have a:

$ rpm-ostree compose jigdo2commit --reposdir /etc/yum.repos.d --releasever 27 --repo=repo-build fedora-atomic-host-27-3.1

Which inverts things back, for ease of testing.

@dustymabe

This comment has been minimized.

Collaborator

dustymabe commented Nov 5, 2017

is the plan for us to support both traditional ostree and OIRPM distribution methods forever? I know at least one thing that doesn't work for us in Fedora is that we don't keep around old rpms so deploying older comits would probably not be possible (or even the 'latest' released commit that was released two weeks ago and a newer version of an rpm within that commit has hit stable). Fedora is not the only use case and it's possible things could change there.

I see OIRPM as an augmentation to traditional repository. i.e. the client could choose which one they want to use. The traditional repo would still be the source of truth, but the rpms could be used to distribute things via RPM channels if some people couldn't mirror around ostrees within a big organization for some reason. How do you see it? Augmenting a traditional ostree server or completely replacing it?

Also, how would we handle deltas? Maybe they are no longer needed if the rpms from the previous jigdo are cached and you only need to download the drpms of the new ones.

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 5, 2017

Doesn't currently keep older RPMs on the mirrors - yes, today, but it certainly seems to me we can fix that, or just use S3/Cloudfront/whatever. Or create a repo subset - the things interesting to pkglayer on AH I suspect is a lot smaller than the total.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Nov 6, 2017

lib/postprocess: Create OstreeSePolicy more directly
We already called the `workaround_selinux_cross_labeling_recurse()`
in the postprocessing path, there's no need to do it again during
commit.

Just making this change as I was going to do some SePolicy stuff
for the [jigdo work](projectatomic#1081)
and stumbled again into the ugly mess that is the cross-labeling
hack.

rh-atomic-bot added a commit that referenced this issue Nov 8, 2017

lib/postprocess: Create OstreeSePolicy more directly
We already called the `workaround_selinux_cross_labeling_recurse()`
in the postprocessing path, there's no need to do it again during
commit.

Just making this change as I was going to do some SePolicy stuff
for the [jigdo work](#1081)
and stumbled again into the ugly mess that is the cross-labeling
hack.

Closes: #1082
Approved by: jlebon

@cgwalters cgwalters changed the title from Experiment with ostree-in-RPM (AKA rpm-ostree jigdo) to β™²πŸ“¦ Experiment with ostree-in-RPM (AKA rpm-ostree jigdo) Nov 13, 2017

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 13, 2017

Just to briefly respond regarding deltas: My thoughts are that we have another OIRPM type which has ostree deltas, and not deltaRPM, because...deltaRPM has a lot of flaws, or conversely ostree deltas are way better than deltaRPM.

IOW we'd have

fedora-atomic-host-27.42-x86_64.rpm and fedora-atomic-host-delta-27.41_27.42.x86_64.rpm or so.

Briefly the big flaws of deltaRPM are:

  • It doesn't work with --nodocs as we use for Atomic Host and the container base images
  • It always uses bsdiff even when that's not effective or an rsync-style rollsum would work just fine, and bsdiff is very CPU intensive
  • Related to the above, it does a bsdiff over the entire RPM rather than per-file, and then is used to reconstruct the final RPM bit-for-bit on disk because that's what's GPG signed...the ostree approach of checksummed objects is a lot more efficient
@jlebon

This comment has been minimized.

Member

jlebon commented Nov 14, 2017

Having carefully reread over the jigdo docs, I have a somewhat better grasp of the design here and it definitely sounds interesting. How large do you expect the OIRPM to be here?

Also, what are your thoughts about how signature verification will work? We don't want to download all the RPMs ahead of time -- so it seems like we have to download the OIRPM first, unpack it, check the signature, and only then proceed to download new RPMs? Or alternatively use RPM signatures? Either way, it sounds like we'd have to download the whole OIRPM before checking that it has a valid signature, which is unfortunate. Or are you approaching this a different way?

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 14, 2017

The baseline OIRPM for Fedora Atomic Host is currently:
-rw-r--r--. 1 root root 65M Nov 14 21:48 /root/rpmbuild/RPMS/x86_64/fedora-atomic-host-27.42-1.fc26.x86_64.rpm
which is mostly the initramfs, with the rest the rpmdb, and then random "%post" data like all the XML files update-mime-database generates.

As far as signatures ✍ I think it's going to be annoying to people doing "mirror all RPMs with gpgcheck=1" unless we do RPM signatures. But just like static deltas we'd embed the ostree commit meta (hm I need to implement that).

We'd unpack the RPM and check the ostree signature too (if enabled).

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Nov 16, 2017

compose: Add --ex-unified-core
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).

And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet.  For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)

Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](projectatomic#1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Nov 16, 2017

compose: Add --ex-unified-core
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).

And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet.  For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)

Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](projectatomic#1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Nov 17, 2017

compose: Add --ex-unified-core
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).

And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet.  For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)

Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](projectatomic#1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.

rh-atomic-bot added a commit that referenced this issue Nov 17, 2017

compose: Add --ex-unified-core
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).

And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet.  For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)

Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](#1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.

Closes: #940
Approved by: jlebon
@jlebon

This comment has been minimized.

Member

jlebon commented Nov 27, 2017

Doesn't currently keep older RPMs on the mirrors - yes, today, but it certainly seems to me we can fix that, or just use S3/Cloudfront/whatever.

I'm still trying to wrap my head around this. IIUC, this means that in order to support deploying older commits, the content provider will also have to keep around every RPM referenced in those commits, right? Seems like a pretty big change in RPM management. Or I suppose we could at the very least just keep "delta-OIRPMs" to go from stable to the specific version? But those would still have to carefully managed separately.

Of course, right now on traditional yum/dnf-managed Fedora systems, you only have access to stable and updates snapshots too. Though having access to older versions is definitely one of the nice features of AH.

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 27, 2017

Both CentOS and RHEL keep older RPM versions. It's only Fedora that's a standout. That said clearly we would need to drive versioning of the rpm-md repos more consistently and expose it.

@cgwalters

This comment has been minimized.

Member

cgwalters commented Nov 27, 2017

BTW just from today: https://lwn.net/Articles/740052/

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Nov 30, 2017

Introduce experimental "rpm-ostree jigdo"
Tracking issue: projectatomic#1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Dec 1, 2017

Introduce experimental "rpm-ostree jigdo"
Tracking issue: projectatomic#1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.
@cgwalters

This comment has been minimized.

Member

cgwalters commented Dec 1, 2017

OK so let's start talking about how this looks/feels on the client side. Strawman:

Actually before we dive in here let's make a presupposition: we rework the fedora.repo and fedora-updates.repo to be the same thing, or at least rpm-ostree is aware they're "the same". And we also "repo-bind" the oirpm - in other words, the fedora repo here acts very similar to an ostree remote.

Upgrading

# rpm-ostree status
State: idle, autoexit in 9 seconds
Repos: fedora fedora-cisco-openh264
Transport Security: GPG, https
Booted Image ●: fedora:fedora-atomic-workstation-27.6-1.x86_64 (2017-11-28 23:33:42)
  LayeredPackages: emacs fuse-sshfs git-evtag keepassx krb5-workstation libvirt-client
                   opensc origin-clients pcsc-lite-ccid powerline strace tmux vagrant-libvirt
                   virt-manager xsel ykclient ykpers

# rpm-ostree upgrade
Enabled rpm-md repositories: fedora fedora-cisco-openh264
rpm-md repo 'fedora' (cached); generated: 2017-11-05 05:51:47
Downloading metadata: fedora 18MB...
Updated image: fedora:fedora-atomic-workstation-27.7-1.x86_64 (2017-12-01 10:32:11)
Downloading 5 packages:
  kernel 4.13.15-300.fc27 -> 4.13.16-300.fc27
  kernel-core 4.13.15-300.fc27 -> 4.13.16-300.fc27
  kernel-modules 4.13.15-300.fc27 -> 4.13.16-300.fc27
  kernel-modules-extra 4.13.15-300.fc27 -> 4.13.16-300.fc27
  rpm-ostree 2017.10-2.fc27 -> 2017.10-3.fc27
  rpm-ostree-libs 2017.10-2.fc27 -> 2017.10-3.fc27
...
Success; new update pending: fedora:fedora-atomic-workstation-27.7-1.x86_64
# rpm-ostree status
State: idle, autoexit in 9 seconds
Repos: fedora fedora-cisco-openh264
Transport Security: GPG, https

Pending Image πŸ†•: fedora:fedora-atomic-workstation-27.7-1.x86_64 (2017-12-01 10:32:11)

Booted Image ●: fedora:fedora-atomic-workstation-27.6-1.x86_64 (2017-11-28 23:33:42)
  LayeredPackages: emacs fuse-sshfs git-evtag keepassx krb5-workstation libvirt-client
                   opensc origin-clients pcsc-lite-ccid powerline strace tmux vagrant-libvirt
                   virt-manager xsel ykclient ykpers
#

Well OK I omitted layered pkg updates but eh.

deploy

# rpm-ostree deploy 27.2
Enabled rpm-md repositories: fedora fedora-cisco-openh264
...
Downloading fedora:fedora-atomic-workstation-27.2-1.x86_64
...

rebase

# rpm-ostree rebase fedora-testing:fedora-atomic-workstation
Enabled rpm-md repositories: fedora fedora-testing fedora-cisco-openh264

Adding the repo here automatically enables it; this is like yum --enablerepo=updates-testing distro-sync.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Dec 1, 2017

Introduce experimental "rpm-ostree jigdo"
Tracking issue: projectatomic#1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Dec 1, 2017

Introduce experimental "rpm-ostree jigdo"
Tracking issue: projectatomic#1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.

rh-atomic-bot added a commit that referenced this issue Dec 4, 2017

Introduce experimental "rpm-ostree jigdo"
Tracking issue: #1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.

Closes: #1103
Approved by: jlebon
@cgwalters

This comment has been minimized.

Member

cgwalters commented Dec 4, 2017

Let me try to answer this:

How do you see it? Augmenting a traditional ostree server or completely replacing it?

For Fedora/CentOS/RHEL: my instinct here is that the answer has to be "completely replacing" - otherwise we're not really gaining the intended benefit of reducing releng/management overhead right?. For other organizations...I don't know?

My instinct here is that the people interested in rpm-ostree are going to be happiest with jigdo but I can't say for sure. I would like rpm-ostree to work in "pure ostree" mode (I heard someone compain that rpm-ostree fails on start if there's no rpmdb). And the people who are using libostree today are likely happy enough with it...the audience there is a lot of Debian and OpenEmbedded. It seems not unlikely that some of those OpenEmbedded users will also want jigdo though?

That said I don't see us deleting any of the existing functionality; or stated more positively, we're just adding an alternative in the code for now. The whole test suite continues to pass. But it is possible that some new feature would only be supported in jigdo mode perhaps?

@jlebon

This comment has been minimized.

Member

jlebon commented Dec 4, 2017

Actually before we dive in here let's make a presupposition: we rework the fedora.repo and fedora-updates.repo to be the same thing, or at least rpm-ostree is aware they're "the same". And we also "repo-bind" the oirpm - in other words, the fedora repo here acts very similar to an ostree remote.

That seems kinda cool, but feels a bit too heuristicky/mysterious. It seems conceptually simpler to just stick this information in the origin, no? E.g. have a [jigdo] group in the origin under which we keep the list of repos we have enabled. I guess we'd have to keep the fedora-atomic-workstation magic string there too, right? (I say magic string because I don't know what to call it, maybe... the "jigdo origin" ?). Or were you thinking of storing that somewhere else?

In that case, rebasing to a jigdo origin that's not in the currently enabled repos would just use --enablerepo, which is something people should already be familiar with.

# rpm-ostree deploy 27.2
Enabled rpm-md repositories: fedora fedora-cisco-openh264
...
Downloading fedora:fedora-atomic-workstation-27.2-1.x86_64
...

How does this work? Is it just doing a pure pkg search for ${jigdo_origin}-${version} ? Seems like we want to make sure we use some new Provides notation here to be more explicit? If we also have a provides that embeds the commit SHA, we could keep the deploy $sha syntax as well.

# rpm-ostree rebase fedora-testing:fedora-atomic-workstation
Enabled rpm-md repositories: fedora fedora-testing fedora-cisco-openh264

It would be cool if we supported seamlessly moving back and forth between "ostree-based" and "jigdo-based" remotes. If we don't overload what remotes means, then keeping the remote:branch for ostree-based remotes makes that easy and natural. Though we would need a new syntax for moving from ostree to jigdo... maybe just something like:

rpm-ostree rebase --jigdo fedora-atomic-workstation

?

@cgwalters

This comment has been minimized.

Member

cgwalters commented Dec 4, 2017

(I say magic string because I don't know what to call it, maybe... the "jigdo origin" ?)

How about jigdo:fedora-testing:fedora-atomic-workstation I.e. we tack on the jigdo: prefix for jigdo mode, and the middle value becomes the rpm-md repo?

Basically I really like the idea of "repo pinning" the jigdo origin. (Not implemented in jigdo2commit right now but it'd be trivial to add to the libdnf query right?)

Seems like we want to make sure we use some new Provides notation here to be more explicit? If we also have a provides that embeds the commit SHA, we could keep the deploy $sha syntax as well.

Ooh those are cool ideas πŸ‘. In fact I may just go implement them now!

cgwalters added a commit to cgwalters/ostree that referenced this issue Feb 8, 2018

core: Add API (and standard concept for) content checksum
There are a few cases for knowing whether a commit has identical
content to another commit.  Some people want to do a "promotion workflow",
where the content of a commit on a tesitng branch is then "promoted"
to a production branch with `ostree commit --tree=ref`.

Another use case I just hit in rpm-ostree deals with
[jigdo](projectatomic/rpm-ostree#1081) where we're
importing RPMs on both the client and server, and will be using the
content checksum, since the client/server cases inject different metadata
into the commit object.

Closes: ostreedev#131

cgwalters added a commit to cgwalters/ostree that referenced this issue Feb 8, 2018

core: Add API (and standard concept for) content checksum
There are a few cases for knowing whether a commit has identical
content to another commit.  Some people want to do a "promotion workflow",
where the content of a commit on a tesitng branch is then "promoted"
to a production branch with `ostree commit --tree=ref`.

Another use case I just hit in rpm-ostree deals with
[jigdo](projectatomic/rpm-ostree#1081) where we're
importing RPMs on both the client and server, and will be using the
content checksum, since the client/server cases inject different metadata
into the commit object.

Closes: ostreedev#131

cgwalters added a commit to cgwalters/ostree that referenced this issue Feb 8, 2018

core: Add API (and standard concept for) content checksum
There are a few cases for knowing whether a commit has identical
content to another commit.  Some people want to do a "promotion workflow",
where the content of a commit on a tesitng branch is then "promoted"
to a production branch with `ostree commit --tree=ref`.

Another use case I just hit in rpm-ostree deals with
[jigdo](projectatomic/rpm-ostree#1081) where we're
importing RPMs on both the client and server, and will be using the
content checksum, since the client/server cases inject different metadata
into the commit object.

Closes: ostreedev#1315

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Feb 11, 2018

WIP: (RFC) status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes:
projectatomic#1081 (comment)

Basically, we can use the `NEVRA` of the jigdoRPM instead of displaying
`Version`. Also, let's be opinionated here and entirely drop the `Commit`
checksum by default. I believe the Cockpit guys were right here - versions are
for humans. The fact that we have a checksum is powerful; and we still show it
with `status -v`. The way I think of it is: the checksum shows we're really an
image system. But we don't need to show it by default.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Feb 11, 2018

WIP: (RFC) status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo β™²πŸ“¦:
projectatomic#1081 (comment)

Basically, we can use the `NEVRA` of the jigdoRPM instead of displaying
`Version`. Also, let's be opinionated here and entirely drop the `Commit`
checksum by default. I believe the Cockpit guys were right here - versions are
for humans. The fact that we have a checksum is powerful; and we still show it
with `status -v`. The way I think of it is: the checksum shows we're really an
image system. But we don't need to show it by default.

rh-atomic-bot added a commit to ostreedev/ostree that referenced this issue Feb 12, 2018

core: Add API (and standard concept for) content checksum
There are a few cases for knowing whether a commit has identical
content to another commit.  Some people want to do a "promotion workflow",
where the content of a commit on a tesitng branch is then "promoted"
to a production branch with `ostree commit --tree=ref`.

Another use case I just hit in rpm-ostree deals with
[jigdo](projectatomic/rpm-ostree#1081) where we're
importing RPMs on both the client and server, and will be using the
content checksum, since the client/server cases inject different metadata
into the commit object.

Closes: #1315

Closes: #1449
Approved by: jlebon

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Feb 14, 2018

WIP: (RFC) status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo β™²πŸ“¦:
projectatomic#1081 (comment)

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Feb 15, 2018

status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo β™²πŸ“¦:
projectatomic#1081 (comment)

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Feb 16, 2018

status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo β™²πŸ“¦:
projectatomic#1081 (comment)

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

rh-atomic-bot added a commit that referenced this issue Feb 16, 2018

status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo β™²πŸ“¦:
#1081 (comment)

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

Closes: #1240
Approved by: jlebon

@cgwalters cgwalters changed the title from β™²πŸ“¦ Implement rpm-ostree jigdo to β™²πŸ“¦ Implement rpm-ostree rojig Feb 22, 2018

jlebon added a commit to jlebon/libdnf that referenced this issue Feb 22, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

rh-atomic-bot added a commit to rpm-software-management/libdnf that referenced this issue Feb 23, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

Closes: #420
Approved by: cgwalters

rh-atomic-bot added a commit to rpm-software-management/libdnf that referenced this issue Feb 23, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

Closes: #420
Approved by: cgwalters

jlebon added a commit to projectatomic/libdnf that referenced this issue Feb 28, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

jlebon added a commit to projectatomic/libdnf that referenced this issue Mar 1, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

rh-atomic-bot added a commit to rpm-software-management/libdnf that referenced this issue Mar 27, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

Closes: #420
Approved by: <try>

j-mracek added a commit to rpm-software-management/libdnf that referenced this issue Mar 27, 2018

Add APIs to disable fetching filelists
As part of [rpm-ostree jigdo β™²πŸ“¦](projectatomic/rpm-ostree#1081)
I'd like to make it cheaper to fetch metadata.  For Fedora currently
the filelists metadata is enormous.  In jigdo mode, we don't need it,
so let's add APIs to avoid fetching it.

cgwalters added a commit to cgwalters/rpm-ostree that referenced this issue Sep 10, 2018

compose: Support "pure rojig" mode
With this, rojig becomes significantly closer to a peer of
ostree mode.  Previously, providing a `--repo` argument
was required.  Now, if an OSTree repository is not specified,
then the compose process assumes that *just* rojig output
is desired.

One of the nicer parts of rpm-ostree I think is the
"change detection" bit.  That's now implemented for the rojig
side by performing a libdnf query of the input rpm-md repo
to find the last rojig rpm, and since we now have the inputhash
as a `Provides`, we can quickly and efficiently implement
change detection there.

The biggest gap left in my opinion is that rpm-md repositories
don't have any *native* concept of versioning.

projectatomic#1081
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment