Initial port to libhif #81
Conversation
|
@james-antill You had concerns about stability AIUI. Option 1: Keep yum aroundOne option we could pursue is to keep the yum code around too, and have this be a build time option. That wouldn't be too hard, but if we did that, it would be correspondingly harder to actually make use of the richer APIs, such as the "Rework compose caching" commit. Option 2: git branchThe question with this would be: where are we going to ship it? I guess that's a per-stream discussion. I'm in favor of branching though, if it turns out the libhif port is successful, doing security-and-critical-stability updates only for a branch would not be a substantial burden I'd say. Other options? |
|
Stability isn't really the right word ... and this is public, right? afaik hawkey and friends aren't planned as supported libraries for 7.x, just plans for tech preview. So AFAIK you can't ship anything that uses them in the normal repos. Then hif needs to be shipped, but that can't possibly be used in 7.x as it's even more incompatible with yum than with dnf (in theory someone can still ship it, but don't know who that'd be or where). Then there's the fact that hawkey and friends won't even be GA shipped by default in Fedora for another 6 months, and aren't going to be compatible with yum/yumdb (dnf has, at least, a different path), and hif won't ever be used elsewhere to install a system (that I know of). Assuming we don't depend on it in the client, can have non-public code on the build server (customers won't be able to do builds though), and don't mind that the result doesn't work with yum (we should probably remove all of yum from the image, then) ... it's just a bit more risky on the testing side to move to it now, I think. As I said though, it's the std. problem of world moves forward and the old rhel stays compatible that everyone hits. And we'll need to hit it at some point, but from my experience after you hit it you long for the times before. I don't think option 2 is viable, even though I think using hif is probably the worst option I can't see it blowing up to the point you'd want to scrap the branch ... so that means branching the other way and freezing almost all updates to el7 ostree/rpm-ostree/etc. which seems like a lot to ask, at this point. Having the code segregated so you can build as either yum-compose or hif-compose, and only lose any package layering type future work, in the former, seems viable to me. All the other code that needs queries only can just talk to librpm directly, as it does now ... or do you think the package layering would leak out to the rest of the code? (or have other uses in mind that might?) |
Channels and timingAt a high level, I'd certainly say things are chaotic. But the ecosystem has been changing. An entirely new channel in RHEL Extras was created for Docker on - unstable, fast moving components on a RHEL base, but also supported by Red Hat (unlike EPEL). hawkey/libsolv might fit in that channel, though that's open for discussion. Atomic itself is a new channel. We've invested in having an upstream in CentOS for example that allows a place to test new technology like Docker and Atomic on a stable base. Primary concernsWe're agreed that
I'm not sure where to have this discussion, but things like the yumdb issue span dnf and libhif (and how things like yumdb/history intersect with rpm-ostree's model is a whole other discussion). There are definitely bugs in the libraries. See e.g. rpm-software-management/librepo#38 rpm-ostree is a cleaner slateAtomic/rpm-ostree has the luxury of not supporting upgrades from an existing traditional install (though of course a new unique feature is you can cleanly parallel-install/dual boot). Also the way the system works means that compatibility with things like the yum history are less of a concern (though we could still use an operation history, just filed #85 ) The downside of being a clean slate of course is Atomic has far fewer features, really just two useful commands on the client end. That goes to the desire for package layering. Compose server onlyI do think the fact that this patch set so far is only changing the compose server side (which is an implementation detail) is a good way to improve the libraries without really exposing them to clients yet. |
|
[Some out of band discussion] Let's have this be a branch for a month or so on the side until we've sorted out more release engineering efforts and current bugs. |
|
Rebased on master. |
cgwalters
force-pushed
the
libhif
branch
2 times, most recently
from
bf1c1ab
to
a909fd8
Compare
Some influence from both systemd and packagekit-glib2.
There are a lot of advantages to this. See the linked issue for more details, but briefly: - Lays the groundwork for package layering - Better caching (exit compose after depsolve if no changes) - Better error handling - Potential for unprivileged package downloads - Potential to better containerize installs TODO: - langs handling - progress output Closes: coreos#53
This requires open-coding the 3 steps that hif_context_run() was doing. Which is fine, because ultimately we want to have more control over that.
A major (if not the biggest) point of this endeavor is to support atomic/idempotent operation.
With yum, we would have had to make a custom Python app to cleanly separate the fetch metadata/depsolve phases from installation. Now that libhif/hawkey gives us that, make use of it by exiting after depsolve if the previous compose has the same package set, and the treefile is the same. This saves a fairly substantial amount of time and I/O, and makes it much more palatable to simply run the compose tool on demand in response to say repo regeneration notifications. A further important note; --cachedir is no longer used; we store the inputhash in the OSTree commit metadata itself.
For the obvious reasons, this helps avoid redownloading packages continuously.
And drop our internal copy.
This is a convenient way for external tools to integrate rpm-ostree better into automation. With the caching, it's now cheaper to just rerun rpm-ostree if you know an input yum repository changed, or just on a timer. Then one can use this flag to determine whether or not to launch any further processes after rpm-ostree, such as disk image generation, test suites, etc.
We hard depend on 0.2.0 (which is current git master).
This branch is a functional basis, and I'd like to land it and use it as a future base. If desired I may squash the first 4 commits, but I thought it'd be useful to show the incremental steps I made.