Tab completion for apt-get install produces only a subset of options

[adamroyjones]

Configuration

I'm running fish 3.6.0 on Debian bookworm. fish has been installed from the Debian package repository using apt. The behaviour is the same when running

sh -c 'env HOME=$(mktemp -d) XDG_CONFIG_HOME= fish'

Description

When I run apt-get install <TAB>, I'm given a list of possible packages to install.

$ apt-get install 
0ad                                                               (Real-time strategy game of ancient warfare)
0ad-data                                             (Real-time strategy game of ancient warfare (data files))
0ad-data-common                               (Real-time strategy game of ancient warfare (common data files))
0xffff                                                                     (Open Free Fiasco Firmware Flasher)
2ping                                                      (Ping utility to determine directional packet loss)
2vcard                                                           (convert an addressbook to VCARD file format)
3dchess                                                                          (Play chess across 3 boards!)
…and 202 more rows

This isn't the full set of possible packages. This should be fairly obvious from the number, but it can be proven by filtering further

$ apt-get install lib
lib/
lib4ti2-0                                      (mathematical tool suite for problems on linear spaces -- libs)
lib4ti2-dev                                     (mathematical tool suite for problems on linear spaces -- dev)
liba52-0.7.4                                                          (library for decoding ATSC A/52 streams)
liba52-0.7.4-dev                                        (library for decoding ATSC A/52 streams (development))
libaa1                                                                                     (ASCII art library)
libaa1-dev                                                                (ASCII art library, development kit)
…and 156 more rows

and then further

$ apt-get install liba
liba52-0.7.4                                                (library for decoding ATSC A/52 streams)
liba52-0.7.4-dev                              (library for decoding ATSC A/52 streams (development))
libaa1                                                                           (ASCII art library)
libaa1-dev                                                      (ASCII art library, development kit)
libaa-bin                                                              (sample programs using aalib)
libabiword-3.0           (efficient, featureful word processor with collaboration -- shared library)
libabiword-dev        (efficient, featureful word processor with collaboration -- development files)
…and 160 more rows

and seeing that the set has now expanded, not narrowed. (It's separately a little interesting that the set isn't refined.)

Does anyone know of any way to produce the full set of possible packages through tab-completion? I imagine I'm missing something fairly obvious but I couldn't find anything from searching around (both in the issues and outside of GitHub).

[faho]

Fixed by 96deaae in master.

Before we tried to use apt-cache show, which is dog slow, and so we had to limit the output to get it to finish in bearable time.

[faho]

Fixed by 96deaae in master.

Aaand unfortunately not - this was based on either a misunderstanding of what /var/lib/dpkg/status is, or a change in its meaning. It will unfortunately only work for installed packages.

Anyway, I've gone back to the old way but increased the limit a bit. Unfortunately dpkg's database layout is pretty horrible, and reading it really does take that long.

If you want to remove it for yourself, funced __fish_print_apt_packages and remove the head.

I feel like if you want a list of all packages there are nicer interfaces than the completion pager, so I don't think that's something that would be worth the trade-off of it taking seconds to load.

[adamroyjones]

Thanks for both the quick response and the ample clarification.

I made the change to __fish_print_apt_packages and poked around. You weren't joking about apt-cache show being baffingly slow.

I looked at commands like dpkg-query but couldn't readily find anything that produces as broad a list as apt-cache show. A brief look at strace showed that apt-cache show was opening a number of files: not just /var/lib/dpkg/status, but also the files in /var/lib/apt/lists/, which look to contain package listings.

Those files may be worth looking at. I don't feel remotely confident in saying that the union of all of those files (together with /var/lib/dpkg/status) contains everything you'd need, but it might.

I feel like if you want a list of all packages there are nicer interfaces than the completion pager, so I don't think that's something that would be worth the trade-off of it taking seconds to load.

I was using fzf for the completions (adapting this function)—it's pleasant, but you're right about the trade-off.

[faho]

a number of files: not just /var/lib/dpkg/status, but also the files in /var/lib/apt/lists/, which look to contain package listings.

I did check those, unfortunately once you do that you end up being about as slow, and I suspect that we would have to redo some more of apt-cache's logic - a naive approach generates about twice the candidates that apt-cache does, presumably because the same package can appear in multiple repos (multiarch is a big thing here). We would have to find the right one if they aren't just direct duplicates.

The database format itself appears to be not conducive to usable performance.