Should remove unused dependencies (DNF backend)

[da2x]

Example:

$ dnf remove konqueror
Removing:
  konqueror
  konqueror-libs
Removing unused dependencies:
  kf5-kdesu
  kf5-khtml
  kf5-kjs
  kf5-kpty
  libtidy

Compared to:

$ pkcon remove konqueror
Removing:
  konqueror
  konqueror-libs

See FreeDesktop bug #94750.

[hughsie]

We deliberately didn't do this to avoid doing things behind the users back. The autoremove functionality isn't exactly bug-free too.

[Conan-Kudo]

Implementing this functionality does not imply that frontends will use it. For example, I don't think either GNOME Software or Plasma Discover are actually wired up to use autoremove. However, Apper is, and this has to be disabled for Fedora and Mageia because the Dnf backend doesn't support it.

[AsciiWolf]

Same issue with the APT backend on Ubuntu.

Please, fix this! This is a major issue for regular users, especially since GNOME Software does not report when installing a desktop environment or another big dependency with an application.

[AsciiWolf]

(I have also created a GNOME Software bug report for this issue.)

[hughsie]

Autoremove does not work 100% correctly, hence why it's not used by default.

[ximion]

@hughsie This is a feature people nag me about for years now, and I actually do wonder whether we should address this request in PackageKit. In the APT world, autoremove is very robust and almost never does the wrong thing unless you start to mess with the system.
Running an autoremove after every removal action is a bad idea though because there will be people who actually want to have full manual control over what gets autoremoved, but still use tools that use PK.
So, I wonder whether it makes sense to add a new action to PackageKit to explicitly trigger an autoremove. That way, we could enable the feature on Ubuntu and Debian, where autoremove is robust enough, and have it disabled in other distros. We can also use a Simulate* routine to show the user what would be autoremoved (and GNOME Software could in theory map that to applications, if it wants to).

[hughsie]

I think i'd rather concentrate on supporting an application delivery system that's optimized for applications, e.g. flatpak or snappy -- if you do and do real user testing with this kind of stuff you'll see very very few people actually understand what dependencies are.

[julian-klode]

Doing autoremoval when a package is removed seems to be a very important thing for Ubuntu this cycle, whether it's on the desktop or even the command-line. Now, I'm not sure if gnome-software on Ubuntu uses PackageKit at the moment, so I'm not sure how relevant that is.

For the lower-level stack, my intention is to go with autoremoving new garbage. That is, collect all unused packages before marking a package for removal, and after, and remove new ones too. This way, people can opt out for some actions (at least in apt), and later run autoremoval to clean those up themselves. And we also provide a nice way out since we don't suddenly run autoremove of old garbage automatically from before autoremoving stuff.

[julian-klode]

Currently people can accidentally install an entire desktop environment by installing an application in gnome-software due to package dependencies, and there's no way for them to get rid of it. Or well, other huge package trees. Especially if as you (@hughsie) say, they don't understand what dependencies are, it is even more important to do autoremoves automatically, so this does not happen.

The automatic autoremove I add to apt might be limited to safe sections, but for gnome-software should handle the whole new-garbage removal thing, and PackageKit probably needs support for that. I'd say it seems like a good idea to get proactive here, before it gets patched in downstream in a way you hate :)

[hughsie]

before it gets patched in downstream in a way you hate

Not sure what you mean here. In more positive note, doesn't the RemovePackage already support an autoremove binary flag?

Richard.

[julian-klode]

Oh, well, I don't exactly like the whole automatic removal idea. So I'm trying to steer it somewhere that is OK for me (like restricting it to the cases where it matters the most, rather than just throw a big autoremove hammer at everything :D). I figured you might want to do the same if people start pushing more aggressively for it :)

[ximion]

@hughsie Users also don't understand Flatpak at all (and tbh, explaining OSTree is way harder than explaining dependencies), so the solution here is not not supporting the feature, but instead make it transparent to the user so they don't have to care about it. Distro packages will stay for quite a while, even with Flatpak and Snappy being available (while the latter even attempted to replace all native packages a while back, it seems that the Snappy team has dialed that down slightly to "mostly apps" now).
That being said:

Not sure what you mean here. In more positive note, doesn't the RemovePackage already support an autoremove binary flag?

This is actually true, surprisingly this feature doesn't seem to be used in aptcc - so resolving this is just a matter of implementing a missing feature then.

@julian-klode Ubuntu uses PackageKit in GNOME Software as well now, since the Artful release (and I am very happy about that!). This means we get a good amount of testing for PK and the aptcc backend in particular at the moment.

[julian-klode]

@ximion Awesome! I kind of lost track of a lot of things during the artful cycle, since I'm busy with university stuff.

As I said, from my perspective, it would be optimal if removing A with gnome-software would remove all dependencies of A no longer needed now, but not any other unused packages (the collect before, collect after, mark difference for removal idea). That would solve the problem in the least invasive way possible.

How that's implemented, I don't really care. I'll likely be building the same thing into APT; as mentioned before. Maybe it's possible to reuse stuff for aptcc backend.

[ximion]

@julian-klode Aptcc is really just some glue code between APT and PK, so as soon as APT can do that kind of smart autoremove, we could use it in the aptcc backend.
Glueing things on top of APT is something I really want to avoid, because it's pretty inefficient and any code we can not have in aptcc is good code :-P
For the Bionic cycle we could implement proper traditional autoremove support in aptcc though.

[Conan-Kudo]

DNF also has the ability to do smart autoremovals, but it wasn't originally incorporated due to bugs with how the PK-Dnf backend managed its access to the DB. Nowadays, I think it'd be fine to add support.

[AsciiWolf]

Any news?

[hughsie]

Any news?

No, and this is not the correct way to get developers to work on things. You've done this quite a few times on a few bugs on of my different projects and it's getting really boring.

[AsciiWolf]

Regarding Ubuntu/Debian: The aptcc PackageKit backend developer told me that autoremove is already supported for more than 5 years, since this commit from 2012. But it still does not work when using pkcon or GNOME Software. Maybe it is not enabled by default?

[julian-klode]

Sure, it's an option given to backends, see pk_backend_remove_packages() - it's a parameter in the Dbus API's RemovePackages() method. In the client API, you can specify it as a boolean parameter for pk_client_remove_packages_async.

But it's not the kind of autoremove that would be correct for gnome-software - gnome-software needs to remove "new garbage", that is only packages that become unused as a result of the removal and not any other unrelated garbage that has accumulated over time.

[Conan-Kudo]

DNF does the "remove related/new garbage" kind of autoremove, which would be appropriate to handle with PackageKit and GNOME Software. If I get some time, I'm hoping to look into implementing this.

[kalev]

We haven't implemented autoremove in the dnf backend not because it's hard to do, but because we had a yumdb writing snafu a while back and everything got marked as a dep, leading to devastating results when doing autoremove.

It should be fine to do that now though, I think, because that was in F21 timeframe if I remember correctly and hopefully there aren't so many affected systems around any more.

However, before implementing autoremoval in PackageKit , we'll need UI in gnome-software to confirm app removals, and also something to protect critical stuff from being autoremoved (gnome-software, gnome-shell).

[AsciiWolf]

As discussed on IRC, PackageKit currently even doesn't support marking the unused dependencies as available for removal on Fedora. Because of this, "dnf autoremove" does not find them and there is no easy way to uninstall them. I think that this is really not a good approach and should be fixed.

[Conan-Kudo]

The DNF backend would also need DependsOn and RequiredBy implemented for proper dependency tracking to implement protected packages support.

[AsciiWolf]

Support for uninstallation of unused dependencies is now added and enabled by-default in Ubuntu (18.04.2 and 19.04). As far as I know, there was no additional UI in gnome-software implemented for this, just the PackageKit backend support. Any chance similar approach could be used in Fedora as well (after DependsOn and RequiredBy are implemented)?

[Conan-Kudo]

Yes, chances are pretty good. It just requires time for someone to implement. It's on my infinitely long TODO, though I'd welcome if someone else beat me to it. :)

[AsciiWolf]

@Conan-Kudo By the way, is the "remove related/new garbage" a standard functionality of DNF or is it added by some (autoremove) plugin?

[Conan-Kudo]

It's a standard part of DNF itself.

[AsciiWolf]

This seems to behave even worse now. When uninstalling an app previously installed with PackageKit (GNOME Software or pkcon) on latest Fedora Rawhide using dnf remove, dnf does not find any unused dependencies and uninstalls only the base app. No dependencies are also found when running dnf autoremove afterwards.

[hughsie]

A dnf regression perhaps? PK hasn't changed in a long time.

[AsciiWolf]

Not sure. It works fine when uninstalling a package previously installed with dnf.

[AsciiWolf]

@hughsie I have reported it as a dnf issue.
edit: Nope, it is most likely a PackageKit issue as mentioned here.

[mdemoss]

Relatedly, I've had a problem where dnf history userinstalled stops working as expected when PackageKit has been used to install software.

The manpage describes it this way:

dnf history userinstalled
Show all installonly packages, packages installed outside of DNF and packages not installed as depen‐
dency. I.e. it lists packages that will stay on the system when Autoremove Command or Remove Command
along with clean_requirements_on_remove configuration option set to True is executed. Note the same
results can be accomplished with dnf repoquery --userinstalled, and the repoquery command is more pow‐
erful in formatting of the output.

All packages installed by PackageKit including dependencies are treated as userinstalled.

[hughsie]

I guess the dnf plugin needs to use this new api somehow...