fish freezes for autocompletion of killall #4052

jacobsa · May 22, 2017

> fish --version
fish, version 2.5.0

> uname -a
Linux <redacted>.com 4.4.0-75-generic #96~14.04.1-Ubuntu SMP Thu Apr 20 11:06:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

> echo $TERM
screen-256color

Every time I attempt to use killall, the shell freezes, apparently trying to do autocompletion.

For example, I just typed killall vim, but got this:

> killall v

The shell remains entirely unresponsive to input until I press Ctrl-C twice, at which pont I can start typing again.

How can I debug this? Unfortunately it's not 100% reproducible; if I try again with killall vim right afterward, it works fine.

This does happen when I run fish with sh -c 'env HOME=$(mktemp -d) fish'.

Are you trying to do completion by pushing Tab, or is it the autosuggestion alone? The autosuggestion should not call external processes, so my suspicion is that loading the history is what's slowing things down. You could run with strace (eg strace -e trace=file -tt -T -f -o fishtrace fish to run with output to the file "fishtrace") to see if you can work out what's taking all the time.

zanchey · May 22, 2017

Are you trying to do completion by pushing Tab, or is it the autosuggestion alone? The autosuggestion should not call external processes, so my suspicion is that loading the history is what's slowing things down. You could run with strace (eg strace -e trace=file -tt -T -f -o fishtrace fish to run with output to the file "fishtrace") to see if you can work out what's taking all the time.

On the other hand, if it's the completion, I definitely get some strange output with killall v<TAB>.

zanchey · May 22, 2017

On the other hand, if it's the completion, I definitely get some strange output with killall v<TAB>.

@jacobsa

The completion for killall looks broken to me:

        complete -c killall -s u -l user -d 'Kill only processes the specified user owns. Command na
        complete -c killall -s -u -l user -x -a "(__fish_complete_users)"

Note that the -s flag requires an argument which, in this case, would be -u.

@jacobsa, Can you reproduce the problem by just running __fish_complete_users?

krader1961 · May 22, 2017

The completion for killall looks broken to me:

        complete -c killall -s u -l user -d 'Kill only processes the specified user owns. Command na
        complete -c killall -s -u -l user -x -a "(__fish_complete_users)"

Note that the -s flag requires an argument which, in this case, would be -u.

@jacobsa, Can you reproduce the problem by just running __fish_complete_users?

@jacobsa

The seemingly strange output from killall v<TAB> is due to this command:

complete -c killall -xa '(__fish_complete_proc)'

That appears to be working as intended.

@jacobsa: The pause is almost certainly due to the __fish_complete_proc function. I see a barely noticeable pause. When I type killall vim the pause occurs right before the final m. But that is just a quirk of the processes running on my system. Looking at that function it's not obvious what is the most likely source of your slowdown. It's probably the ps command. Perhaps UID to user name lookups are sometimes really slow on your system due to NIS/Yellowpages/LDAP or some other directory service being slow to respond.

krader1961 · May 23, 2017

The seemingly strange output from killall v<TAB> is due to this command:

complete -c killall -xa '(__fish_complete_proc)'

That appears to be working as intended.

@jacobsa: The pause is almost certainly due to the __fish_complete_proc function. I see a barely noticeable pause. When I type killall vim the pause occurs right before the final m. But that is just a quirk of the processes running on my system. Looking at that function it's not obvious what is the most likely source of your slowdown. It's probably the ps command. Perhaps UID to user name lookups are sometimes really slow on your system due to NIS/Yellowpages/LDAP or some other directory service being slow to respond.

To be clear, I'm not pressing tab. I type killall vim, and it hangs with only killall v echoed.

__fish_complete_users hangs for many seconds and then prints a tremendous amount of output, over 400k lines. (This is on a corporate machine in a company with many users.) So perhaps that is the root cause?

jacobsa · May 23, 2017

To be clear, I'm not pressing tab. I type killall vim, and it hangs with only killall v echoed.

__fish_complete_users hangs for many seconds and then prints a tremendous amount of output, over 400k lines. (This is on a corporate machine in a company with many users.) So perhaps that is the root cause?

I don't think so - __fish_complete_users only fires when you push tab.

zanchey · May 23, 2017

I don't think so - __fish_complete_users only fires when you push tab.

Similarly __fish_complete_proc only fires when you press [tab]. However, filename completion does fire. So I'm guessing your CWD is on a remote filesystem like NFS that is slow to respond when the problem occurs.

krader1961 · May 23, 2017

Similarly __fish_complete_proc only fires when you press [tab]. However, filename completion does fire. So I'm guessing your CWD is on a remote filesystem like NFS that is slow to respond when the problem occurs.

This is not on a remote file system. There is a symlink to a fuse file system in the parent of the working directory, but it seems unlikely that would matter.

This really does appear to be related to user completion. Here's a portion of the output of the strace command from above in the thread after I typed (I believe) killall vim and then a backspace or two:

24466 13:34:09.919990 stat("/usr/local/<redacted>/home/jacobsa/.config/fish/fishd.aa00010002d4", {st_mode=S_IFREG|0640, st_size=1339, ...}) = 0 <0.000045>
24466 13:34:09.921087 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000042>
24466 13:34:09.921190 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000031>
24466 13:34:09.921353 access("/usr/local/buildtools/java/jdk/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000038>
24466 13:34:09.921438 access("/usr/local/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000030>
24466 13:34:09.921516 access("/usr/local/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000032>
24466 13:34:09.921596 access("/usr/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000033>
24466 13:34:09.921676 access("/usr/bin/id", X_OK) = 0 <0.000033>
24466 13:34:09.921753 stat("/usr/bin/id", {st_mode=S_IFREG|0755, st_size=35520, ...}) = 0 <0.000030>
25154 13:34:09.927355 execve("/usr/bin/id", ["id", "-u", "<redacted>"], [/* 32 vars */]) = 0 <0.001874>
25154 13:34:09.929520 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000041>
25154 13:34:09.929695 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000037>
25154 13:34:09.929794 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000043>
25154 13:34:09.930051 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.930138 open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3 <0.000035>
25154 13:34:09.930695 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000040>
25154 13:34:09.930803 open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000045>
25154 13:34:09.931290 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.931369 open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3 <0.000070>
25154 13:34:09.931993 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000044>
25154 13:34:09.932110 open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000046>
25154 13:34:09.933493 statfs("/sys/fs/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000064>
25154 13:34:09.933632 statfs("/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000058>
25154 13:34:09.933933 open("/proc/filesystems", O_RDONLY) = 3 <0.000067>
25154 13:34:09.934434 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000119>
25154 13:34:09.935159 open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.935477 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000024>
25154 13:34:09.935670 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000166>
25154 13:34:09.935888 open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000030>
25154 13:34:09.936373 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.936709 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000037>
25154 13:34:09.936912 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000054>
25154 13:34:09.937022 open("/usr/lib/x86_64-linux-gnu/libnss_cache.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000034>
25154 13:34:09.937493 open("/etc/passwd.cache.ixname", O_RDONLY) = 3 <0.000033>
25154 13:34:09.937573 stat("/etc/passwd.cache", {st_mode=S_IFREG|0644, st_size=35555602, ...}) = 0 <0.000025>
25154 13:34:09.938019 open("/etc/passwd.cache", O_RDONLY) = 3 <0.000179>
25154 13:34:09.939166 +++ exited with 0 +++
24466 13:34:09.939299 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=25154, si_status=0, si_utime=0, si_stime=0} ---

This is repeated over and over and over and over again, thousands of times, with the username handed to id -u changing each time.

jacobsa · May 23, 2017

This is not on a remote file system. There is a symlink to a fuse file system in the parent of the working directory, but it seems unlikely that would matter.

This really does appear to be related to user completion. Here's a portion of the output of the strace command from above in the thread after I typed (I believe) killall vim and then a backspace or two:

24466 13:34:09.919990 stat("/usr/local/<redacted>/home/jacobsa/.config/fish/fishd.aa00010002d4", {st_mode=S_IFREG|0640, st_size=1339, ...}) = 0 <0.000045>
24466 13:34:09.921087 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000042>
24466 13:34:09.921190 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000031>
24466 13:34:09.921353 access("/usr/local/buildtools/java/jdk/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000038>
24466 13:34:09.921438 access("/usr/local/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000030>
24466 13:34:09.921516 access("/usr/local/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000032>
24466 13:34:09.921596 access("/usr/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000033>
24466 13:34:09.921676 access("/usr/bin/id", X_OK) = 0 <0.000033>
24466 13:34:09.921753 stat("/usr/bin/id", {st_mode=S_IFREG|0755, st_size=35520, ...}) = 0 <0.000030>
25154 13:34:09.927355 execve("/usr/bin/id", ["id", "-u", "<redacted>"], [/* 32 vars */]) = 0 <0.001874>
25154 13:34:09.929520 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000041>
25154 13:34:09.929695 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000037>
25154 13:34:09.929794 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000043>
25154 13:34:09.930051 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.930138 open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3 <0.000035>
25154 13:34:09.930695 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000040>
25154 13:34:09.930803 open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000045>
25154 13:34:09.931290 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.931369 open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3 <0.000070>
25154 13:34:09.931993 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000044>
25154 13:34:09.932110 open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000046>
25154 13:34:09.933493 statfs("/sys/fs/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000064>
25154 13:34:09.933632 statfs("/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000058>
25154 13:34:09.933933 open("/proc/filesystems", O_RDONLY) = 3 <0.000067>
25154 13:34:09.934434 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000119>
25154 13:34:09.935159 open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.935477 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000024>
25154 13:34:09.935670 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000166>
25154 13:34:09.935888 open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000030>
25154 13:34:09.936373 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.936709 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000037>
25154 13:34:09.936912 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000054>
25154 13:34:09.937022 open("/usr/lib/x86_64-linux-gnu/libnss_cache.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000034>
25154 13:34:09.937493 open("/etc/passwd.cache.ixname", O_RDONLY) = 3 <0.000033>
25154 13:34:09.937573 stat("/etc/passwd.cache", {st_mode=S_IFREG|0644, st_size=35555602, ...}) = 0 <0.000025>
25154 13:34:09.938019 open("/etc/passwd.cache", O_RDONLY) = 3 <0.000179>
25154 13:34:09.939166 +++ exited with 0 +++
24466 13:34:09.939299 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=25154, si_status=0, si_utime=0, si_stime=0} ---

This is repeated over and over and over and over again, thousands of times, with the username handed to id -u changing each time.

I definitely saw this again by typing killall vim (with a trailing space that GitHub is not rendering) followed by a backspace. The shell hung with killall vim (with a trailing space) echoed. This was in a local file system directory without a fuse symlink nearby.

jacobsa · May 23, 2017

I definitely saw this again by typing killall vim (with a trailing space that GitHub is not rendering) followed by a backspace. The shell hung with killall vim (with a trailing space) echoed. This was in a local file system directory without a fuse symlink nearby.

What you're observing is due to an incredibly inefficient, and pointless, feature of the killall completion script. This was reported to us and fixed a month ago. See issue #3996. The fix will be in the fish 2.6.0 release. You can just copy the updated completion script to your system.

krader1961 · May 23, 2017

What you're observing is due to an incredibly inefficient, and pointless, feature of the killall completion script. This was reported to us and fixed a month ago. See issue #3996. The fix will be in the fish 2.6.0 release. You can just copy the updated completion script to your system.

Glad to hear it, thanks. Where do I put that script on my system, and how do I disable the built-in one?

jacobsa · May 24, 2017

Glad to hear it, thanks. Where do I put that script on my system, and how do I disable the built-in one?

You can put it in $HOME/.config/fish/completions or $__fish_datadir/completions. Personal completion and autoloaded functions supercede those in the global directory. Putting the updated script in the global directory is less likely to cause you problems in the future. If you put it in your personal config directory and we update the script in the future you need to remember to update or remove your private copy.

krader1961 · May 24, 2017

You can put it in $HOME/.config/fish/completions or $__fish_datadir/completions. Personal completion and autoloaded functions supercede those in the global directory. Putting the updated script in the global directory is less likely to cause you problems in the future. If you put it in your personal config directory and we update the script in the future you need to remember to update or remove your private copy.

krader1961 added the needs more info label May 22, 2017

krader1961 added a commit that referenced this issue May 23, 2017

fix `killall` completions
This fixes the obvious error in handling the `-u` short flag. See issue #4052.

a71bb03

krader1961 closed this May 23, 2017

krader1961 added duplicate and removed needs more info labels May 23, 2017

zanchey added a commit that referenced this issue May 25, 2017

fix `killall` completions
This fixes the obvious error in handling the `-u` short flag. See issue #4052. (cherry picked from commit a71bb03)

0bf4d39

fish-shell/fish-shell

fish freezes for autocompletion of killall #4052

jacobsa commented May 22, 2017

This comment has been minimized.

zanchey commented May 22, 2017 •

edited

This comment has been minimized.

zanchey commented May 22, 2017

This comment has been minimized.

krader1961 commented May 22, 2017

krader1961 added the needs more info label May 22, 2017

This comment has been minimized.

krader1961 commented May 23, 2017

krader1961 added a commit that referenced this issue May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017 •

edited

This comment has been minimized.

zanchey commented May 23, 2017

This comment has been minimized.

krader1961 commented May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017 •

edited

This comment has been minimized.

krader1961 commented May 23, 2017

krader1961 closed this May 23, 2017

krader1961 added duplicate and removed needs more info labels May 23, 2017

This comment has been minimized.

jacobsa commented May 24, 2017

This comment has been minimized.

krader1961 commented May 24, 2017

zanchey added a commit that referenced this issue May 25, 2017

fish-shell/fish-shell

Join GitHub today

fish freezes for autocompletion of killall #4052

Comments

jacobsa commented May 22, 2017

This comment has been minimized.

zanchey commented May 22, 2017 • edited

This comment has been minimized.

zanchey commented May 22, 2017

This comment has been minimized.

krader1961 commented May 22, 2017

krader1961 added the needs more info label May 22, 2017

This comment has been minimized.

krader1961 commented May 23, 2017

krader1961 added a commit that referenced this issue May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017 • edited

This comment has been minimized.

zanchey commented May 23, 2017

This comment has been minimized.

krader1961 commented May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017

This comment has been minimized.

jacobsa commented May 23, 2017 • edited

This comment has been minimized.

krader1961 commented May 23, 2017

krader1961 closed this May 23, 2017

krader1961 added duplicate and removed needs more info labels May 23, 2017

This comment has been minimized.

jacobsa commented May 24, 2017

This comment has been minimized.

krader1961 commented May 24, 2017

zanchey added a commit that referenced this issue May 25, 2017

zanchey commented May 22, 2017 •

edited

jacobsa commented May 23, 2017 •

edited

jacobsa commented May 23, 2017 •

edited