wildcard: Rationalize file/command completions

[faho]

Description

Alternative to #10050. NOTE The rust version is not up-to-date, it's #10050. I based it on that, I will of course adjust that before merging if we do decide that this is an acceptable approach.

This keeps the entry_t as long as possible, and asks it, so especially
on systems with working d_type we can get by without a single stat in
most cases.

Then it guts file_get_desc, because that is only used for command
completions - we have been disabling file descriptions for years,
and so this is never called there.

That means we have no need to print descriptions about e.g. broken symlinks, because those are not executable.

Put together, what this means is that we, in most cases, only do
an access(2) call instead of a stat, because that might be checking
more permissions.

So we have the following constellations:

• If we have d_type:
  • We need a stat() for every symlink to get the type (e.g. dir or regular)
    (this is for most symlinks, if we want to know if it's a dir or executable)
  • We need an access() for every file for executables
• If we do not have d_type:
  • We need a stat() for every file
  • We need an lstat() for every file if we do descriptions
    (i.e. just for command completion)
  • We need an access() for every file for executables

As opposed to the current way, where every file gets one lstat whether
with d_type or not, and an additional stat() for links, and an
access. And this is on top of the stat() that the entry might already have done - even without d_type, we save that.

So we go from two syscalls to one for executables, while improving accuracy compared to #10050.

Numbers

All acquired via strace -c -e %file fish --no-config -c $thething, counting "file" syscalls like access and the stat family. This is an up-to-date linux system on ext4, with d_type.

complete -C "ls ~/complete_test/", where complete_test is a directory with 10k directories and 10k files:

• Current pre-3.7.0 and Assume files with all exec bits set are executable for completions #10050: 20k stat
• This: 0 stat (well, 32, but in comparison it's basically 0)

complete -C "" (command completion), with 2870 files in $PATH, 2865 of which are executable files and 407 are links:

• Current pre-3.7.0: 6239 "file" syscalls (3331 newfstatat, 2869 access)
• Assume files with all exec bits set are executable for completions #10050: 3382, 3328 of which are newfstatat (3.6.1 would have similar numbers)
• This: 3361, 2868 access + 442 newstatat (remember, 407 links)

I will have to see if I can get numbers for without d_type. The gains there are that we can reuse the stat from is_dir() if we are completing directories specifically, and that we can skip the lstat() for non-executables.

TODOs:

• [N/A] Changes to fish usage are reflected in user documentation/manpages.
• [N/A] Tests have been added for regressions fixed
• [N/A] User-visible changes noted in CHANGELOG.rst

[ridiculousfish]

This is fantastic. I love it. This seems like the minimal possible syscalls for command completions.

[ridiculousfish]

This whole function is so much nicer now!

[faho]

(@mqudsi this should really also be a thing on WSL2)

[mqudsi]

LGTM! Nice work, much cleaner.

I agree on the WSL2 bit; I can open a PR with the unification of WSL1 and WSL2 stuff. Maybe there's nothing that it won't hurt to do under WSL2.