Very high CPU usage on fish 3.2 when displaying large strings

[henrikhorluck]

Hi! I am having an issue with strings in commands that are longer than ~1048 characters is causing 100% CPU usage on 1 or more cores. I have confirmed that it is not an issue with fish 3.1.2, and it reproduces when using sh -c 'env HOME=$(mktemp -d) fish'

Version-stuff (was also a problem on 3.2.0):

> fish --version
fish, version 3.2.1
> echo $version
v3.2.1

> uname -u
Darwin Henriks-Macbook.local 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

Example of command that triggers high CPU-usage just by being pasted into the terminal, or by being written yourself, it triggers simply being displayed.

touch "qg5e6rnmfzQZh18hMUbgDiwHfN+Abd22mIWuuMYemSvfdetIog4B5HPJgTC5hAkwNMPpn+TuRqJnvn3EVWkZN42ksyH/WawZ6Zq5Z+EwAJLs5dzVuzw6cwI2UfHWIZrew90W4XG8Ad/y+rwPt4iHVRIYaSkIGj6mzsD0w7rJdvHpIbeNDUcggCWlR80wsd4IrhoLFdw7Okhy0vj36tOd9M18uQpFJENEd1zFWUaKiV9PFBZepS71rqEeuUZ8K5JgGBqlOoX3qePy9ryd+AMGAsBKi/ivxekj05qFSOuHgGWBabNlyQJfGhQt81QD3s8DRS+QH2fUaiTrLDfWhwu1qg5e6rnmfzQZh18hMUbgDiwHfN+Abd22mIWuuMYemSvfdetIog4B5HPJgTC5hAkwNMPpn+TuRqJnvn3EVWkZN42ksyH/WawZ6Zq5Z+EwAJLs5dzVuzw6cwI2UfHWIZrew90W4XG8Ad/y+rwPt4iHVRIYaSkIGj6mzsD0w7rJdvHpIbeNDUcggCWlR80wsd4IrhoLFdw7Okhy0vj36tOd9M18uQpFJENEd1zFWUaKiV9PFBZepS71rqEeuUZ8K5JgGBqlOoX3qePy9ryd+AMGAsBKi/ivxekj05qFSOuHgGWBabNlyQJfGhQt81QD3s8DRS+QH2fUaiTrLDfWhwu1qg5e6rnmfzQZh18hMUbgDiwHfN+Abd22mIWuuMYemSvfdetIog4B5HPJgTC5hAkwNMPpn+TuRqJnvn3EVWkZN42ksyH/WawZ6Zq5Z+EwAJLs5dzVuzw6cwI2UfHWIZrew90W4XG8Ad/y+rwPt4iHVRIYaSkIGj6mzsD0w7rJdvHpIbeNDUcggCWlR80wsd4IrhoLFdw7Okhy0vj36tOd9M18uQpFJENEd1zFWUaKiV9PFBZepS71rqEeuUZ8K5JgGBqlOoX3qePy9ryd+AMGAsBKi/ivxekj05qFSOuHgGWBabNlyQJfGhQt81QD3s8DRS+QH2fUaiTrLDfWhwu132TuRqJnvn3EVWkZN42ksyH/w

If you have one less character in the string it does not cause the high CPU-usage. One thing I noticed when testing fish 3.1.2 was that it didn't add line breaks to the commands, while 3.2.1 had it over multiple lines.

Please let me know if something is unclear, you need any more information to reproduce it, or debug information I can supply

[faho]

I can't reproduce on linux.

Which terminal are you using? Does this also happen in other terminals? Any customization (including prompts, vi-mode, third-party packages)?

Does the text matter at all? Does this also happen when you just have, say, 2000 "a"s (string repeat -n 2000 a)? Unquoted, inside single-quotes?

[henrikhorluck]

I am using the built in terminal on macOS, VI-mode and tmux (v.3.1c).

It must be after a command, so e.g. cd or touch first, while "<characters>" on itself doesn't trigger it. The text inside the string does not matter, I also do not think it needs quotes, so touch <lots of a> also triggers it.

Perhaps relevant stuff from config.fish

if status --is-interactive
	tmux ^ /dev/null; and exec true
end

# Base16 Shell
# from https://github.com/chriskempson/base16-shell
if status --is-interactive
    set BASE16_SHELL "$HOME/.config/base16-shell/"
    source "$BASE16_SHELL/profile_helper.fish"
end

# Fish should not add things to clipboard when killing
# See https://github.com/fish-shell/fish-shell/issues/772
set FISH_CLIPBOARD_CMD "cat"

Update: It also reproduces in Alacritty

[faho]

It must be after a command

What do you mean by that?

Have you tried disabling vi-mode and commenting out your config.fish?

Sidenote: FISH_CLIPBOARD_CMD has been dead for 4.5 years by this point, and the ^ redirection is deprecated (use 2> instead).

[faho]

Ah, I think you're not getting bracketed paste for some reason - this tells fish that the text it's getting is pasted, so it can skip a bunch of things (like, say, executing it when there's a newline, but also highlighting).

Since you're using tmux, and from fish's point of view that's its terminal, you should check if that is disabling it for some reason (fish enables it unconditionally).

[faho]

If you use tmux's pasting, you need to use paste-buffer -p (because of course there is no way for tmux to detect that... oh, wait, it detects if it's off and then doesn't do it even with "-p". lovely)

[ross2084]

I have the same problem without tmux, in fish version 3.2.1. I'm not sure, but I think the problem appeared in 3.2.0. Pasting the original touch command text causes my shell to immediately begin consuming at least 100% CPU.

Darwin 20G-T0WXJG5M-35K 19.6.0 Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020; root:xnu-6153.141.10~1/RELEASE_X86_64 x86_64

[faho]

Alright, this seems like mac-specific stuff, so I can't be of any more assistance.

[henrikhorluck]

Okey, it reproduces without tmux, and with commented out config.fish, and fish_user_default_keybindings.

By it must be after a command I mean that

> aaaaaaaaaaa (a little longer)

Doesn't trigger the issue at hand, but

> touch aaaaaaaaaaaa

Does trigger it

[jsatk]

This is also happening for me as well.

[ross2084]

One additional bit of possibly useful behavior I've noticed is that even when typing at the shell prompt, % CPU spikes quite a bit until I stop typing, where it then drops back down again. The faster I type, the higher the percentage increase.

With Zsh or Bash, % CPU jumps only slightly if I rapidly type at my prompt, maybe a tenth or a fifth of a percent increase. But Fish jumps well into the 20'ish percent from simply typing text at rapid speed.

[krobelus]

Can you reproduce inside HOME=(mktemp -d) fish?

when typing at the shell prompt, % CPU spikes quite a bit until I stop typing

Brief spikes are expected and harmless; fish is just computing autosuggestions (from shell history).

[ross2084]

Thanks for the clarification on the temporary spikes.

Yes I can reproduce with HOME=(mktemp -d) fish. Pasting the above example text causes an immediate spike to 100% or more CPU until the shell is closed.

[zanchey]

I can reproduce this on macOS with the string above.

The sample shows spending two threads spending their whole time (2681 samples) in each of the following:

Call graph:
    2681 Thread_4248685   DispatchQueue_1: com.apple.main-thread  (serial)
    + 2681 start  (in libdyld.dylib) + 1  [0x7fff59edc3d5]
    +   2681 main  (in fish) + 7938  [0x100e187ae]
    +     2681 reader_read(parser_t&, int, io_chain_t const&)  (in fish) + 1406  [0x100ede5f9]
    +       2681 reader_data_t::readline(int)  (in fish) + 1124  [0x100edd722]
    +         2681 reader_data_t::read_normal_chars(readline_loop_state_t&)  (in fish) + 249  [0x100ed87cd]
    +           2681 inputter_t::readch(std::__1::function<void (std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > > const&)> const&)  (in fish) + 107  [0x100ea3d0f]
    +             2681 input_event_queue_t::readch()  (in fish) + 254  [0x100eaa15a]
    +               2681 input_event_queue_t::readb()  (in fish) + 357  [0x100ea9a93]
    +                 2681 __select  (in libsystem_kernel.dylib) + 10  [0x7fff5a018616]
    2681 Thread_4352150
      2681 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff5a0cf40d]
        2681 _pthread_start  (in libsystem_pthread.dylib) + 66  [0x7fff5a0d3249]
          2681 _pthread_body  (in libsystem_pthread.dylib) + 126  [0x7fff5a0d02eb]
            2681 thread_pool_t::run_trampoline(void*)  (in fish) + 14  [0x100eada56]
              2681 thread_pool_t::run()  (in fish) + 243  [0x100ead869]
                2681 std::__1::__function::__func<debounce_t::perform_impl(std::__1::function<void ()>, std::__1::function<void ()>)::$_0, std::__1::allocator<debounce_t::perform_impl(std::__1::function<void ()>, std::__1::function<void ()>)::$_0>, void ()>::operator()()  (in fish) + 22  [0x100eaf7d4]
                  2681 debounce_t::impl_t::run_next(unsigned long long)  (in fish) + 251  [0x100eae619]
                    2681 std::__1::__function::__func<iothread_trampoline_t<std::__1::function<(anonymous namespace)::highlight_result_t ()>, reader_data_t::super_highlight_me_plenty()::$_2, (anonymous namespace)::highlight_result_t>::iothread_trampoline_t(std::__1::function<(anonymous namespace)::highlight_result_t ()> const&, reader_data_t::super_highlight_me_plenty()::$_2 const&)::'lambda'(), std::__1::allocator<iothread_trampoline_t<std::__1::function<(anonymous namespace)::highlight_result_t ()>, reader_data_t::super_highlight_me_plenty()::$_2, (anonymous namespace)::highlight_result_t>::iothread_trampoline_t(std::__1::function<(anonymous namespace)::highlight_result_t ()> const&, reader_data_t::super_highlight_me_plenty()::$_2 const&)::'lambda'()>, void ()>::operator()()  (in fish) + 39  [0x100ee329f]
                      2681 std::__1::__function::__func<get_highlight_performer(parser_t&, std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, bool)::$_8, std::__1::allocator<get_highlight_performer(parser_t&, std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, bool)::$_8>, (anonymous namespace)::highlight_result_t ()>::operator()()  (in fish) + 241  [0x100ee0ec3]
                        2681 highlight_shell(std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, std::__1::vector<highlight_spec_t, std::__1::allocator<highlight_spec_t> >&, operation_context_t const&, bool)  (in fish) + 180  [0x100e92c58]
                          2681 highlighter_t::highlight()  (in fish) + 624  [0x100e91400]
                            2681 is_potential_path(std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > > const&, operation_context_t const&, unsigned int)  (in fish) + 1540  [0x100e8f648]
                              2681 wdirname(std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&)  (in fish) + 57  [0x100ef5839]
                                2681 str2wcstring(char const*)  (in fish) + 21  [0x100e56b35]
                                  2681 _platform_strlen  (in libsystem_platform.dylib) + 18  [0x7fff5a0c46f2]