alpm: Eventually it stops refreshing

[aleixpol]

Original report:
https://bugs.kde.org/show_bug.cgi?id=450381

Further investigation shows that sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/ also fixes it.

PackageKit was getting the following error from alpm:
failed to uptate database: invalid or corrupted database (PGP signature)

[Etaash-mathamsetty]

Original report: https://bugs.kde.org/show_bug.cgi?id=450381

Further investigation shows that sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/ also fixes it.

PackageKit was getting the following error from alpm: failed to uptate database: invalid or corrupted database (PGP signature)

I am also getting that alpm error, any way to fix it?

[simonbcn]

Arch Linux
packagekit 1.2.5

The problem is temporarily solved with sudo rm -R /var/lib/PackageKit/alpm and restarting packagekit service but it fails again after some time.
sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/ doesn't work for me.

What commands should I run to detect the source of the problem when it occurs again?

[ItachiSan]

Hi,

I have been investigating things recently as I both have Pamac and Discover and wanted to figure out why one worked and the other not.

I did dig in the code in GitHub interface found a suspicious line, git blamed it and got to #385.

What this does, at least from my understanding (I think myself to be a good developer but I am not proficient in C and know nothing of PackageKit code base), is that unless the refresh is "forced", the database won't see any updates.

I have verified locally that calling pkcon refresh force does indeed update the database and make both pkcon and Discover see the packages.

I would argue the merge request should be reverted, as its idea is incorrect; the argument is that refreshing databases may cause breakages due to partial updates.
I am unsure how that would come to be, but I believe that this is not the proper solution; the solution would be to sync databases always even when upgrading a single package.

[joshlangley]

Arch Linux
packagekit 1.2.5

The problem is temporarily solved with sudo rm -R /var/lib/PackageKit/alpm and restarting packagekit service but it fails again after some time. sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/ doesn't work for me.

What commands should I run to detect the source of the problem when it occurs again?

I believe there is a typo in the original command. The default location of the Arch repository database files is /var/lib/pacman/sync. PackageKit ALPM also seems to hold a sort of duplicate of this location stored at /var/lib/PackageKit/alpm/sync. (Not /var/lib/PackageKit/alpm.) See my screenshot, default repos on the left, PackageKit repos on the right. By deleting /var/lib/PackageKit/alpm you forced PackageKit to remake that directory and thus refresh the databases in order to create them.

As a result the correct command would be sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/sync; sudo systemctl restart packagekit, but this is effectively what pkcon refresh force (see previous comment) does without needing to restart the service. Thus it is better to use pkcon refresh force.

That said, I have also faced the infamous persistent lock file error also, so you might also have to try that. (https://wiki.archlinux.org/title/KDE#Discover_stops_showing_updates_from_Arch_repositories)

[joshlangley]

Original report: https://bugs.kde.org/show_bug.cgi?id=450381
Further investigation shows that sudo pacman -Sy --dbpath /var/lib/PackageKit/alpm/ also fixes it.
PackageKit was getting the following error from alpm: failed to uptate database: invalid or corrupted database (PGP signature)

I am also getting that alpm error, any way to fix it?

@Etaash-mathamsetty This sounds like it is probably ALPM passing on a pacman error related to old repository databases and keys (not PackageKit ALPM related). Basically the mirrorlists and keylists are provided by packages (pacman-mirrorlist and archlinux-keyring I think) that can't be upgraded because the installed mirrors/keys are invalid. This results in a sort of upgrade deadlock. You could try resetting your keys (https://wiki.archlinux.org/title/Pacman/Package_signing#Resetting_all_the_keys) and/or regenerating your mirrorlist from a valid one (https://archlinux.org/mirrorlist/).

[joshlangley]

Hi,

I have been investigating things recently as I both have Pamac and Discover and wanted to figure out why one worked and the other not.

I did dig in the code in GitHub interface found a suspicious line, git blamed it and got to #385.

What this does, at least from my understanding (I think myself to be a good developer but I am not proficient in C and know nothing of PackageKit code base), is that unless the refresh is "forced", the database won't see any updates.

I have verified locally that calling pkcon refresh force does indeed update the database and make both pkcon and Discover see the packages.

I would argue the merge request should be reverted, as its idea is incorrect; the argument is that refreshing databases may cause breakages due to partial updates. I am unsure how that would come to be, but I believe that this is not the proper solution; the solution would be to sync databases always even when upgrading a single package.

@ItachiSan Thank you so much for providing this information. I think I'm now one step closer to solving this thing. Here's my current understanding of the situation:

Originally PackageKit always refreshed the repositories when getting updates, then somebody either ran into the partial upgrade conundrum or noticed that it could occur and made the merge request you mentioned. As a result, partial upgrades are no longer possible unless the user uses pkcon refresh force and then promptly runs pkcon install <somepackage> without running pkcon upgrade. However, pkcon upgrade will also never work anymore unless the user first runs pkcon refresh force. (Or perhaps it also gets refreshed after a successful upgrade? Also hopefully before installing new packages? These cases should be investigated.)

In summary, in patching an edge case, the main functionality was broken. Obviously something wasn't tested along the way, or someone didn't understand what was going on, or something else like that.

I don't agree that the MR should simply be reverted as partial upgrades are indeed rare, but they are a bad thing when they occur. (See https://wiki.archlinux.org/title/System_maintenance#Partial_upgrades_are_unsupported and my diagram below.)

Here is the solution I would propose:

• Refreshing and getting updates does actually refresh the package lists. (Undo the MR, but also implement the following:)
• If the system is up to date, the user can just install packages normally.
• If the system is not up to date, PackageKit presents a dialog warning about the partial upgrade and prompting for one of the following actions when installing a new package:
  • Cancel (abort) the operation
  • Upgrade the system while installing the new package (default option)
  • Force the operation anyway (install the package without updating)

This would be the ideal option, since that is how pacman is designed to be used in the first place. Additionally it would prevent partial upgrades on other rolling release systems.

Another worthwhile thing to confirm is that the package lists are always refreshed before installing new packages. If this is not so it would actually be more egregious than the partial upgrade situation and should be rectified.

[aleixpol]

If the system is not up to date, PackageKit presents a dialog warning...

How would PackageKit present a warning? And so specific no less...

[ItachiSan]

Hi @joshlangley and @aleixpol , any ideas on how to follow up on this?

I can provide inputs, but given my non-invovelment in the project I didn't feel entitled to.

Personally, the warning popup sounds good and user friendly in general.
However, this means that PackageKit now needs to have a consistent way to open the pop up everywhere.
From my experience as an Arch user, the e.g. Qt bindings are a separate package from base PackageKit.

Additionally, in Arch specifically, if you do

pacman -Sy
pacman -S package

this won't upgrade the dependencies and create a partial upgrade situation, as you mentioned.

My opinion is that refreshing should not require the force parameter.
Refreshing packages should work everywhere and force should be required only in a broken state e.g. the lock file issue.
Instead, specific package installation should be inhibited on non-partial-upgrades backends until the other packages are updated.
In case of systems where partial upgrades are supported e.g. the relevant dependencies get updated installing the package, the whole inhibition mechanism can be skipped completely.
I suggest so believing that PackageKit has already a system preventing the user from installing stuff that will break the system.

[ItachiSan]

Hi, I would like to know whether there has been some discussion or not.
I can then help providing code; while I am not a C expert nor an Arch dev, I am quite flexible with code and can test stuff locally.

[joshlangley]

It's been a while since I visited this problem, but I do have a future project that will hinge on it, so I suppose I can't avoid it forever...

Basically the further I dug in, the more I realized there's multiple errors at play here, but I wasn't able to sort them out. I tried to dig into the code base a bit but didn't have enough time, and that's when I started noticing the error that got me totally stuck.

I started noticing a problem where PackageKit would claim the system was fully up to date, even with all the fixes mentioned in this thread, including pkcon refresh force. However, I could literally run sudo pacman -Syu right after it and see I had weeks worth of updates waiting. However, once in a blue moon, PackageKit would start showing some of them at a time. Not all of them at once as expected. The fact that the behaviors of PackageKit and pacman are divergent at all is puzzling; don't they both use libalpm? I hope to have time to dig into this a bit more in the coming weeks.

Either way, this situation is way worse for the update health of an Arch system than a partial upgrade. Moreover, partial upgrades problems are very rare. I can't remember the last time I saw one cause a problem on any of my systems or servers. My conclusion then, is that pkcon refresh should indeed not be a no-op without force. It's causing more problems than it solves. I do not see removing the force requirement as a solution to the real problem, but it's clear that the Arch support in PackageKit needs some work anyway and it may as well function in the meantime.

[joshlangley]

I've also thought about the warning dialog and I do agree PackageKit shouldn't offer it, at least not the way I described. I do think the user should be given the option to update the system while installing new packages, just as you can with pacman. For example sudo pacman -Syu libreoffice-still updates the system while installing LibreOffice, which is fine. Now I think this should be implemented by the software center, with the behavior being activated by some sort of "rolling system" flag that can be looked up with PackageKit, or something more along those lines. (Again, just a rough concept.)

Also, a comment on the force argument, pacman itself sort of implements a flag like this, with sudo pacman -Sy refreshing databases automatically if needed, while sudo pacman -Syy force refreshes all of them no matter what. I think that this should be the proper behavior of the force argument, not a hot fix of a lock file problem that the user shouldn't have to fix in the long run.