Bauh takes a loooong time syncing packages after first installation on Manjaro

[lfom]

Describe the bug
After installing bauh for the first time, it lakes a long time (minutes, more than 5min on an i3 2.66GHZ) syncing packages. Looks like a bug because it shows 0% for a long time, and if the proccess is skeeped than it shows a blank (empty) window and does not show anything for a long time either.

Software Environment
bauh version: 0.9.6
O.S: Manjaro 20.0.3
Python version: 3.8.3
bauh installation method: pamac-manager (GUI)

Everything else is fine, only initial synchronization is painfully slow and looks like a bug as I explained above. I have around 1555 packages installed and nothing else (no Flatpaks nor Snaps). I would suggest that when running for the first time it asks if the user wants to use bauh for managing pacman/AUR and show a warning saying it may take a long time if the user answers yes (I guess most people won't since pamac does already).

[vinifmor]

Hi @lfom , this initial task only organizes data from Arch/AUR packages (the other types are not necessary). Bauh does that to provide a better experience with these packages. The first execution takes more time, since it has a lot of packages to check. The next executions will be faster, since it will only map data from not mapped packages.

Do you have flatpak and snapd installed ? (otherwise bauh will not be able to work with them).

In relation to your suggestion, it is a good point. The first execution could let the users choose which sort of packages they want to work with.

[vinifmor]

To investigate a possible a bug, I would need the console logs. You can get them by resetting the bauh settings and launching it through the command line: bauh --reset && bauh

[lfom]

No errors at all (the warning is harmless AFAIK, related to Wayland):

$ bauh --reset && bauh
[bauh] Cleaning configuration and cache files
[bauh] Deleting directory /home/lfom/.cache/bauh
[bauh] Directory /home/lfom/.cache/bauh deleted
[bauh] Deleting directory /home/lfom/.config/bauh
[bauh] Directory /home/lfom/.config/bauh deleted
[bauh][appimage] Deleting /home/lfom/.local/share/bauh/appimage/releases.db
[bauh][appimage] /home/lfom/.local/share/bauh/appimage/releases.db deleted
[bauh][appimage] Deleting /home/lfom/.local/share/bauh/appimage/apps.db
[bauh][appimage] /home/lfom/.local/share/bauh/appimage/apps.db deleted
[bauh] Cleaning finished
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
qt.qpa.xcb: QXcbConnection: XCB error: 5 (BadAtom), sequence: 587, resource id: 0, major code: 20 (GetProperty), minor code: 0

It just stays in this state for a loooooong time (the number of packages increased because I had to install Vala for other purposes):

[vinifmor]

On my testing machines it generally maps everything in less than 30 seconds on the first time. But I'm going to check the algorithm to see if it can be improved.

[MattMattV]

If it can help, on my system it took ~6 minutes and it stayed a long time on the "Read: 0/2306" too

[vinifmor]

guys, please post your CPU specs. I'm testing even on limited virtual machines and the process takes less than a minute.

[lfom]

more than 5min on an i3 2.66GHZ

[MattMattV]

My work tower where the long index time was experienced use a Intel Xeon E5-2630 v4 @ 3.100GHz

[vinifmor]

ok, so it might be related to the amount of files installed by each package. I'm going to test a different algorithm strategy to try improving this first data organization task.

[MattMattV]

I can provide you the list of package if you want to do some reproduction (:

[vinifmor]

It's ok for now. If I need them, I'm going to ask for you later. By the way, are you guys using SSDs for storage ?

[lfom]

By the way, are you guys using SSDs for storage ?

HDD here, but if IIRIC I didn't see any disk activity (the notebook has a HDD activity led) during the wait. It was latest Manjaro Gnome version, if it makes any difference.

[vinifmor]

Actually this step performs a lot of reading (it analyses the installed files of each package for caching purposes). As I have no HDD to test, I'm going to try performing less reading during this step to reduce its execution time.

[vinifmor]

Guys, I've improved the algorithm and now it takes considerably less time. 1650 packages would take about 70 seconds on my end, and now just 11. This improvement is already available on the staging branch. You can install through AUR as bauh-staging (0.9.7.RC-9). Give me a feedback about your ends (perform a reset first -> bauh-reset && bauh).

[MattMattV]

Just tried with 0.9.7.RC-9 and it took around 20 seconds ! Thank you very much for this great work !

[vinifmor]

No worries. Let me know if any other issue shows up. I'm trying to make this next release as solid as possible.