Pasting in multi-line prompt causes error in __fish_tokenizer_state

[mqudsi]

I'm creating an issue for this for changelog purposes, since it is a regression introduced in 3.2.0

Some of the changes to fish_clipboard_paste in 3.2.0 broke pasting when the command line consisted of more than one line, caused by our old friend the forcible splitting of substituted process output on new lines with no good way to opt out.

e.g.

$ echo "
>     hello^v

This would result in an error:

__fish_tokenizer_state: Expected at most 1 args, got 2

because the output of (commandline -ct) is passed to __fish_tokenizer_state in fish_clipboard_paste, but it is forcibly split on new lines. (I'm pretty sure the exact same underlying process substitution has a lot of other bugs in completions everywhere.)

The problem is further exacerbated by the fact that any errors emitted by fish while the prompt is being edited cause corruption of the in-memory terminal state (actual and expected row/cell info differs) meaning that further input at the prompt is not visible or causes corruption of existing content until the screen is cleared (^L) or the commandline is repainted (commandline -f repaint).

There was no issue pasting in a multi-line prompt in 3.1.0.

[z3ntu]

I'm still seeing this with $FISH_VERSION 3.2.1 (Arch Linux package version 3.2.1-1) with exactly this error: __fish_tokenizer_state: Expected at most 1 args, got 2

[faho]

@z3ntu Do check that you have the correct __fish_tokenizer_state function (i.e. functions -D __fish_tokenizer_state says it's from /usr/share/fish/functions/__fish_tokenizer_state.fish) and are running the actual correct fish. I just tested the arch package and it works for me.

[z3ntu]

Not sure, I'm pretty sure I run proper fish 3.2.1 without any messing up any internal functions myself.. 😉

I do echo " enter & ctrl+shift+v for paste in Konsole/Yakuake

I also have the contents of this commit (also in 3.2.1) installed on my file system 10ef0d9

[z3ntu]

Seems the fix wasn't applied to share/functions/__fish_commandline_is_singlequoted.fish ? Editing /usr/share/fish/functions/__fish_commandline_is_singlequoted.fish and doing so it reads (commandline -ct | string collect) seems to work

[faho]

functions -D fish_clipboard_paste?

[faho]

Ah, yeah, that's it.

__fish_commandline_is_singlequoted should be removed (the comments say as much), but it's still used in the shared bindings for bracketed paste (i.e. not fish_clipboard_paste, which this was about and which is in many ways superior)

[z3ntu]

Thanks! Seems __fish_start_bracketed_paste was the function that was called.. Applying that fix locally works fine (also with the __fish_commandline_is_singlequoted.fish change reverted) :)