RFE: extract changelog to file, include as %doc automatically and put stub in actual binary and source RPMs.

[mattdm]

In these days of version control, I think we've outlived the usefulness of having full changelogs in RPM files. Yet, it's still useful to have them in the specfile for workflows and build systems which haven't adapted to using version control changelogs.

My suggestion is:

1. Changelog entries in specfile remain as they are.
2. At build time, the changelog is extracted to a file named name-version-release.rpm-changelog (so for example, rpm-4.13.0.1-5.fc26.rpm-changelog).
3. That file automatically treated as a %doc and either included in the /usr/share/doc/packagename directory or maybe all collected in /usr/share/doc/rpm-changelogs.
4. In the package itself, the latest changelog entry included
5. Every older changelog entry not included in the source or binary RPM
6. The line - Older changelog information can be found at {path to wherever the changelog is installed} added to the end of the one included changelog entry.

This will:

1. Let people keep doing whatever they want with RPM changelog in spec files
2. Shrink the size of /var/lib/rpm/Packages
3. Provide latest changelog entry in the way everyone is used to
4. Make older changelog entries easily available on-system for people who want them
5. In fact, improve things since those can be indexed and searched with grep or whatever even more trivially than currently
6. Make it possible to just not include the files by using nodocs

As a fancy additional possibility, rpm --changelog could read and output these files in preference to the embedded information. (I guess if that direction is taken, skip the idea of adding the Older changelog information line.)

The only downside I can think of is increased complexity in RPM when maybe we should just deprecate RPM changelogs entirely. But, I think we're not really ready for that yet and this is a good intermediate approach.

[Conan-Kudo]

@mattdm wrote:

In these days of version control, I think we've outlived the usefulness of having full changelogs in RPM files. Yet, it's still useful to have them in the specfile for workflows and build systems which haven't adapted to using version control changelogs.

This is absolutely silly. Changelogs play an important role in allowing people to see how the package changed. There are plenty of places that have a private VCS but enter useful changelog information (e.g. Red Hat with Red Hat Enterprise Linux). Others don't really have a VCS in the normal sense and use a separate file as a source for changelog entries (SUSE/openSUSE). And of course, there are those who use the VCS as the changelog (e.g. Mageia with its Dist-SVN).

Maybe you don't see the usefulness of them, but I think they need to be in the spec. It's perfectly fine to limit the entries in the binary packages for history, but that should really only apply to Fedora packages. In my opinion, it shouldn't even apply to people who build packages on Fedora systems.

And Fedora's Dist-Git is full of dumb commits that aren't helpful (like "oops" or "forgot sources") that are irrelevant to changelogs, and with no way to make changelog entries nice from VCS to spec, all you're left with is maintaining a changelog file.

In the end, you go nowhere with this.

[mattdm]

Maybe you don't see the usefulness of them, but I think they need to be in the spec.

@Conan-Kudo, it seems like maybe you didn't read the proposal? All of the things you say are exactly why I'm suggesting this instead of getting rid of the spec file changelog.

[Conan-Kudo]

@mattdm I quoted the wrong thing, it was past midnight. I meant to quote the bottom which said:

The only downside I can think of is increased complexity in RPM when maybe we should just deprecate RPM changelogs entirely.

[mattdm]

@Conan-Kudo In that case, what about the very next line, "But, I think we're not really ready for that yet and this is a good intermediate approach"? I think maybe we are in violent agreement here. :)

[Conan-Kudo]

@mattdm The problem with that is that it's now impossible to retrieve the changelog data in the metadata. It would make sense only if you were going to impose a date cutoff, but that changelog information needs to be available somehow and people want it before installing.

There are at least a couple of bugs where people are explicitly requesting the changelog metadata to be exposed in Fedora, and we currently don't. Both RhBug:1066867 and RhBug:1235551 are requests to expose the changelog information to people in DNF to consume, and I know people are doing weird workarounds right now for this, and I loathe to make this information less useful to them than it already is (most people put pretty trivial info in changelogs as is, though that trend is changing lately).

The more important issue, in my opinion, is to get the DNF guys to actually implement these things so that the information isn't mostly deadweight as it has been since the Yum->DNF transition.

[mattdm]

You're right that "not available before installing" is another downside.

[Conan-Kudo]

@n3npq So, the RPM-MD repodata published by Fedora does include the information, it's just that DNF doesn't process it, even though it downloads it.

[mattdm]

@n3npq Well, except, it really will shrink the size of /var/lib/rpm/Packages, because that contains all of the changelog information currently. Sure, other things might shrink it more, but that's not a reason to not do it.

[proyvind]

About (optionally) separating %changelog from spec file, that makes perfect sense, which several distributions does with as pointed out it rather generated on the fly from vcs commit log and appended to spec file during build.

Adding changelog to spec files these days is comparable to mantaining changelogs in any other ~language source code which uses vcs with commit logs.

OpenSuSE is supposedly doing what this RFE asks for already according to #69 ( @mlschroe can you confirm?) , which implements optional, default external changelog file.

Given this RFE, OpenSuSE's already adoption of their own implemented version of this functionality, IMO #69 should be reopened....

[pmatilai]

Technically, I think we have sufficient APIs nowadays that we could actually pull file contents such as a changelog from the payload on non-installed packages. There's some extra cost of course, but just how performance critical is looking at changelogs? Not very I think.

The "problem" is that the changelog is not stored in the form you see with -q --changelog, but three arrays which can be accessed programmatically, such as to pull the last three items, or the unfortunate --changes which shows finegrained timestamps. Of course the content could be parsed back into that form using the same code that puts it into the header, we'd just need to move part of the parsing code to librpm side. But whether that's worth all the extra complexity?

[pmatilai]

The same tag-query-from-payload thing could be beneficial to various other cases too, such as RPMTAG_SPEC which could be placed in the payload rather than the header if we could do this.