New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fish freezes for autocompletion of killall #4052

Closed
jacobsa opened this Issue May 22, 2017 · 12 comments

Comments

Projects
None yet
3 participants
@jacobsa

jacobsa commented May 22, 2017

> fish --version
fish, version 2.5.0

> uname -a
Linux <redacted>.com 4.4.0-75-generic #96~14.04.1-Ubuntu SMP Thu Apr 20 11:06:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

> echo $TERM
screen-256color

Every time I attempt to use killall, the shell freezes, apparently trying to do autocompletion.

For example, I just typed killall vim, but got this:

> killall v

The shell remains entirely unresponsive to input until I press Ctrl-C twice, at which pont I can start typing again.

How can I debug this? Unfortunately it's not 100% reproducible; if I try again with killall vim right afterward, it works fine.

This does happen when I run fish with sh -c 'env HOME=$(mktemp -d) fish'.

@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey May 22, 2017

Member

Are you trying to do completion by pushing Tab, or is it the autosuggestion alone? The autosuggestion should not call external processes, so my suspicion is that loading the history is what's slowing things down. You could run with strace (eg strace -e trace=file -tt -T -f -o fishtrace fish to run with output to the file "fishtrace") to see if you can work out what's taking all the time.

Member

zanchey commented May 22, 2017

Are you trying to do completion by pushing Tab, or is it the autosuggestion alone? The autosuggestion should not call external processes, so my suspicion is that loading the history is what's slowing things down. You could run with strace (eg strace -e trace=file -tt -T -f -o fishtrace fish to run with output to the file "fishtrace") to see if you can work out what's taking all the time.

@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey May 22, 2017

Member

On the other hand, if it's the completion, I definitely get some strange output with killall v<TAB>.

Member

zanchey commented May 22, 2017

On the other hand, if it's the completion, I definitely get some strange output with killall v<TAB>.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 May 22, 2017

Contributor

The completion for killall looks broken to me:

        complete -c killall -s u -l user -d 'Kill only processes the specified user owns. Command na
        complete -c killall -s -u -l user -x -a "(__fish_complete_users)"

Note that the -s flag requires an argument which, in this case, would be -u.

@jacobsa, Can you reproduce the problem by just running __fish_complete_users?

Contributor

krader1961 commented May 22, 2017

The completion for killall looks broken to me:

        complete -c killall -s u -l user -d 'Kill only processes the specified user owns. Command na
        complete -c killall -s -u -l user -x -a "(__fish_complete_users)"

Note that the -s flag requires an argument which, in this case, would be -u.

@jacobsa, Can you reproduce the problem by just running __fish_complete_users?

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 May 23, 2017

Contributor

The seemingly strange output from killall v<TAB> is due to this command:

complete -c killall -xa '(__fish_complete_proc)'

That appears to be working as intended.

@jacobsa: The pause is almost certainly due to the __fish_complete_proc function. I see a barely noticeable pause. When I type killall vim the pause occurs right before the final m. But that is just a quirk of the processes running on my system. Looking at that function it's not obvious what is the most likely source of your slowdown. It's probably the ps command. Perhaps UID to user name lookups are sometimes really slow on your system due to NIS/Yellowpages/LDAP or some other directory service being slow to respond.

Contributor

krader1961 commented May 23, 2017

The seemingly strange output from killall v<TAB> is due to this command:

complete -c killall -xa '(__fish_complete_proc)'

That appears to be working as intended.

@jacobsa: The pause is almost certainly due to the __fish_complete_proc function. I see a barely noticeable pause. When I type killall vim the pause occurs right before the final m. But that is just a quirk of the processes running on my system. Looking at that function it's not obvious what is the most likely source of your slowdown. It's probably the ps command. Perhaps UID to user name lookups are sometimes really slow on your system due to NIS/Yellowpages/LDAP or some other directory service being slow to respond.

krader1961 added a commit that referenced this issue May 23, 2017

fix `killall` completions
This fixes the obvious error in handling the `-u` short flag.
See issue #4052.
@jacobsa

This comment has been minimized.

Show comment
Hide comment
@jacobsa

jacobsa May 23, 2017

To be clear, I'm not pressing tab. I type killall vim, and it hangs with only killall v echoed.

__fish_complete_users hangs for many seconds and then prints a tremendous amount of output, over 400k lines. (This is on a corporate machine in a company with many users.) So perhaps that is the root cause?

jacobsa commented May 23, 2017

To be clear, I'm not pressing tab. I type killall vim, and it hangs with only killall v echoed.

__fish_complete_users hangs for many seconds and then prints a tremendous amount of output, over 400k lines. (This is on a corporate machine in a company with many users.) So perhaps that is the root cause?

@zanchey

This comment has been minimized.

Show comment
Hide comment
@zanchey

zanchey May 23, 2017

Member

I don't think so - __fish_complete_users only fires when you push tab.

Member

zanchey commented May 23, 2017

I don't think so - __fish_complete_users only fires when you push tab.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 May 23, 2017

Contributor

Similarly __fish_complete_proc only fires when you press [tab]. However, filename completion does fire. So I'm guessing your CWD is on a remote filesystem like NFS that is slow to respond when the problem occurs.

Contributor

krader1961 commented May 23, 2017

Similarly __fish_complete_proc only fires when you press [tab]. However, filename completion does fire. So I'm guessing your CWD is on a remote filesystem like NFS that is slow to respond when the problem occurs.

@jacobsa

This comment has been minimized.

Show comment
Hide comment
@jacobsa

jacobsa May 23, 2017

This is not on a remote file system. There is a symlink to a fuse file system in the parent of the working directory, but it seems unlikely that would matter.

This really does appear to be related to user completion. Here's a portion of the output of the strace command from above in the thread after I typed (I believe) killall vim and then a backspace or two:

24466 13:34:09.919990 stat("/usr/local/<redacted>/home/jacobsa/.config/fish/fishd.aa00010002d4", {st_mode=S_IFREG|0640, st_size=1339, ...}) = 0 <0.000045>
24466 13:34:09.921087 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000042>
24466 13:34:09.921190 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000031>
24466 13:34:09.921353 access("/usr/local/buildtools/java/jdk/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000038>
24466 13:34:09.921438 access("/usr/local/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000030>
24466 13:34:09.921516 access("/usr/local/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000032>
24466 13:34:09.921596 access("/usr/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000033>
24466 13:34:09.921676 access("/usr/bin/id", X_OK) = 0 <0.000033>
24466 13:34:09.921753 stat("/usr/bin/id", {st_mode=S_IFREG|0755, st_size=35520, ...}) = 0 <0.000030>
25154 13:34:09.927355 execve("/usr/bin/id", ["id", "-u", "<redacted>"], [/* 32 vars */]) = 0 <0.001874>
25154 13:34:09.929520 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000041>
25154 13:34:09.929695 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000037>
25154 13:34:09.929794 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000043>
25154 13:34:09.930051 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.930138 open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3 <0.000035>
25154 13:34:09.930695 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000040>
25154 13:34:09.930803 open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000045>
25154 13:34:09.931290 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.931369 open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3 <0.000070>
25154 13:34:09.931993 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000044>
25154 13:34:09.932110 open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000046>
25154 13:34:09.933493 statfs("/sys/fs/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000064>
25154 13:34:09.933632 statfs("/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000058>
25154 13:34:09.933933 open("/proc/filesystems", O_RDONLY) = 3 <0.000067>
25154 13:34:09.934434 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000119>
25154 13:34:09.935159 open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.935477 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000024>
25154 13:34:09.935670 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000166>
25154 13:34:09.935888 open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000030>
25154 13:34:09.936373 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.936709 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000037>
25154 13:34:09.936912 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000054>
25154 13:34:09.937022 open("/usr/lib/x86_64-linux-gnu/libnss_cache.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000034>
25154 13:34:09.937493 open("/etc/passwd.cache.ixname", O_RDONLY) = 3 <0.000033>
25154 13:34:09.937573 stat("/etc/passwd.cache", {st_mode=S_IFREG|0644, st_size=35555602, ...}) = 0 <0.000025>
25154 13:34:09.938019 open("/etc/passwd.cache", O_RDONLY) = 3 <0.000179>
25154 13:34:09.939166 +++ exited with 0 +++
24466 13:34:09.939299 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=25154, si_status=0, si_utime=0, si_stime=0} ---

This is repeated over and over and over and over again, thousands of times, with the username handed to id -u changing each time.

jacobsa commented May 23, 2017

This is not on a remote file system. There is a symlink to a fuse file system in the parent of the working directory, but it seems unlikely that would matter.

This really does appear to be related to user completion. Here's a portion of the output of the strace command from above in the thread after I typed (I believe) killall vim and then a backspace or two:

24466 13:34:09.919990 stat("/usr/local/<redacted>/home/jacobsa/.config/fish/fishd.aa00010002d4", {st_mode=S_IFREG|0640, st_size=1339, ...}) = 0 <0.000045>
24466 13:34:09.921087 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000042>
24466 13:34:09.921190 access("/usr/local/<redacted>/home/jacobsa/go/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000031>
24466 13:34:09.921353 access("/usr/local/buildtools/java/jdk/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000038>
24466 13:34:09.921438 access("/usr/local/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000030>
24466 13:34:09.921516 access("/usr/local/bin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000032>
24466 13:34:09.921596 access("/usr/sbin/id", X_OK) = -1 ENOENT (No such file or directory) <0.000033>
24466 13:34:09.921676 access("/usr/bin/id", X_OK) = 0 <0.000033>
24466 13:34:09.921753 stat("/usr/bin/id", {st_mode=S_IFREG|0755, st_size=35520, ...}) = 0 <0.000030>
25154 13:34:09.927355 execve("/usr/bin/id", ["id", "-u", "<redacted>"], [/* 32 vars */]) = 0 <0.001874>
25154 13:34:09.929520 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000041>
25154 13:34:09.929695 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000037>
25154 13:34:09.929794 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000043>
25154 13:34:09.930051 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.930138 open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3 <0.000035>
25154 13:34:09.930695 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000040>
25154 13:34:09.930803 open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000045>
25154 13:34:09.931290 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000032>
25154 13:34:09.931369 open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3 <0.000070>
25154 13:34:09.931993 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000044>
25154 13:34:09.932110 open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000046>
25154 13:34:09.933493 statfs("/sys/fs/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000064>
25154 13:34:09.933632 statfs("/selinux", 0x7ffd9cc5ff90) = -1 ENOENT (No such file or directory) <0.000058>
25154 13:34:09.933933 open("/proc/filesystems", O_RDONLY) = 3 <0.000067>
25154 13:34:09.934434 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000119>
25154 13:34:09.935159 open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.935477 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000024>
25154 13:34:09.935670 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000166>
25154 13:34:09.935888 open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000030>
25154 13:34:09.936373 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3 <0.000028>
25154 13:34:09.936709 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000037>
25154 13:34:09.936912 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000054>
25154 13:34:09.937022 open("/usr/lib/x86_64-linux-gnu/libnss_cache.so.2", O_RDONLY|O_CLOEXEC) = 3 <0.000034>
25154 13:34:09.937493 open("/etc/passwd.cache.ixname", O_RDONLY) = 3 <0.000033>
25154 13:34:09.937573 stat("/etc/passwd.cache", {st_mode=S_IFREG|0644, st_size=35555602, ...}) = 0 <0.000025>
25154 13:34:09.938019 open("/etc/passwd.cache", O_RDONLY) = 3 <0.000179>
25154 13:34:09.939166 +++ exited with 0 +++
24466 13:34:09.939299 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=25154, si_status=0, si_utime=0, si_stime=0} ---

This is repeated over and over and over and over again, thousands of times, with the username handed to id -u changing each time.

@jacobsa

This comment has been minimized.

Show comment
Hide comment
@jacobsa

jacobsa May 23, 2017

I definitely saw this again by typing killall vim (with a trailing space that GitHub is not rendering) followed by a backspace. The shell hung with killall vim (with a trailing space) echoed. This was in a local file system directory without a fuse symlink nearby.

jacobsa commented May 23, 2017

I definitely saw this again by typing killall vim (with a trailing space that GitHub is not rendering) followed by a backspace. The shell hung with killall vim (with a trailing space) echoed. This was in a local file system directory without a fuse symlink nearby.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 May 23, 2017

Contributor

What you're observing is due to an incredibly inefficient, and pointless, feature of the killall completion script. This was reported to us and fixed a month ago. See issue #3996. The fix will be in the fish 2.6.0 release. You can just copy the updated completion script to your system.

Contributor

krader1961 commented May 23, 2017

What you're observing is due to an incredibly inefficient, and pointless, feature of the killall completion script. This was reported to us and fixed a month ago. See issue #3996. The fix will be in the fish 2.6.0 release. You can just copy the updated completion script to your system.

@jacobsa

This comment has been minimized.

Show comment
Hide comment
@jacobsa

jacobsa May 24, 2017

Glad to hear it, thanks. Where do I put that script on my system, and how do I disable the built-in one?

jacobsa commented May 24, 2017

Glad to hear it, thanks. Where do I put that script on my system, and how do I disable the built-in one?

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 May 24, 2017

Contributor

You can put it in $HOME/.config/fish/completions or $__fish_datadir/completions. Personal completion and autoloaded functions supercede those in the global directory. Putting the updated script in the global directory is less likely to cause you problems in the future. If you put it in your personal config directory and we update the script in the future you need to remember to update or remove your private copy.

Contributor

krader1961 commented May 24, 2017

You can put it in $HOME/.config/fish/completions or $__fish_datadir/completions. Personal completion and autoloaded functions supercede those in the global directory. Putting the updated script in the global directory is less likely to cause you problems in the future. If you put it in your personal config directory and we update the script in the future you need to remember to update or remove your private copy.

zanchey added a commit that referenced this issue May 25, 2017

fix `killall` completions
This fixes the obvious error in handling the `-u` short flag.
See issue #4052.

(cherry picked from commit a71bb03)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment