Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign uppaste: redesign, fixes, 10x speedup #4448
Conversation
c9b85ab
to
8898ee5
| @@ -3435,11 +3435,11 @@ getchar([expr]) *getchar()* | |||
| :endfunction | |||
| < | |||
| You may also receive synthetic characters, such as | |||
| |<CursorHold>|. Often you will want to ignore this and get | |||
| |<LeftMouse>|. Often you will want to ignore this and get | |||
This comment was marked as resolved.
This comment was marked as resolved.
justinmk
Mar 14, 2016
Author
Member
<CursorHold> pseudokey doesn't exist any more (replaced by K_EVENT).
This comment was marked as resolved.
This comment was marked as resolved.
bfredl
Aug 20, 2019
Member
<LeftMouse> is user input, just like <F10> or any other special key without canonical unicode representation, of which there are many. Seems weird to ignore only it specifically. CursorHold needed extra treatment here because it was generated by nvim without user intervention so getchar() terminated early, other special chars do not have this problem.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
bfredl
Aug 20, 2019
Member
I think so. The concrete examples above of how to actually handle pseudokeys (including mouse) should be enough.
| // didn't match ui_toggle_key and didn't try the whole typebuf, | ||
| // check the 'pastetoggle' | ||
| match = typebuf_match_len(p_pt, &mlen); | ||
| } |
This comment has been minimized.
This comment has been minimized.
justinmk
Mar 14, 2016
Author
Member
Now we don't need this special-case logic in this low-level function...
| @@ -233,9 +237,12 @@ static void tk_getkeys(TermInput *input, bool force) | |||
| } | |||
| } | |||
|
|
|||
| if (result != TERMKEY_RES_AGAIN || input->paste_enabled) { | |||
| if (result != TERMKEY_RES_AGAIN /*|| input->paste_started */) { | |||
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tarruda
Mar 14, 2016
Member
No, but this probably isn't needed anymore(I don't think it is possible to have result == TERMKEY_RES_AGAIN && input->paste_enabled because read_cb will only push data up to the next \x1b(esc). I'm not sure why this is there, probably an oversight or leftover from previous code.
| if (enable) { | ||
| loop_schedule(&loop, event_create(1, apply_pastepre, 0)); | ||
| } else { | ||
| enqueue_input(input, PASTEPOST_KEY, sizeof(PASTEPOST_KEY) - 1); |
This comment has been minimized.
This comment has been minimized.
justinmk
Mar 14, 2016
Author
Member
@tarruda / @oni-link, Wondering if you can help my understanding. loop_schedule() on the "start paste" case (line 306) works well, because it puts the event on the main queue before any input gets processed. But for the "stop paste" case, I had to enqueue this PASTEPOST_KEY in order to guarantee that all paste data was processed before queuing the PastePost event.
Do we currently have any better mechanism for placing an event at a precise location in the input stream (I'm guessing the answer is "no", hence why we still have these pseudokeys)? I also tried using a mutex, but that had problems and isn't really desirable anyways.
I am wondering if it would be a good idea to provide some internal generic mechanism for placing K_EVENT in the input stream and associating it with a callback.
This comment has been minimized.
This comment has been minimized.
tarruda
Mar 14, 2016
Member
Do we currently have any better mechanism for placing an event at a precise location in the input stream (I'm guessing the answer is "no", hence why we still have these pseudokeys)
No we don't.
I am wondering if it would be a good idea to provide some internal generic mechanism for placing K_EVENT in the input stream and associating it with a callback.
K_EVENT is not meant to be used by UIs, it is only a general mechanism for waking nvim when an asynchronous event happens. An UI placing a bunch of keys in the input stream is an asynchronous event, but not "certain key gets processed"(And the only use case I could think for this is toggling paste, so I'm not sure it would be useful)
This comment has been minimized.
This comment has been minimized.
justinmk
Mar 14, 2016
Author
Member
"certain key gets processed"(And the only use case I could think for this is toggling paste, so I'm not sure it would be useful)
@tarruda So does that mean we can safely use loop_schedule() to schedule FocusGained/FocusLost instead of the K_FOCUSGAINED/K_FOCUSLOST?
This comment has been minimized.
This comment has been minimized.
tarruda
Mar 14, 2016
Member
@tarruda So does that mean we can safely use loop_schedule() to schedule FocusGained/FocusLost instead of the K_FOCUSGAINED/K_FOCUSLOST?
Possibly yes. I may be missing something, but FocusGained/FocusLost doesn't seem to be events that needs to be handled at an exact point during input processing.
| // Make sure the next input escape sequence fits into the ring buffer | ||
| // without wrap around, otherwise it could be misinterpreted. | ||
| // CSI and K_SPECIAL are double-escaped later (input_enqueue). | ||
| // Make sure they can fit into the ring buffer without wrap around. | ||
| rbuffer_reset(input->read_stream.buffer); |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
tarruda
Mar 14, 2016
Member
No, input->read_stream.buffer is only read by the loop above. There could be an multi-byte key sequence that is trimmed by the end of the buffer(wrapping around to the beginning), but termkey_push_bytes is not aware of that, so it could misinterpret the key. rbuffer_reset simply moves the data so it is all in contiguous memory.
This comment was marked as resolved.
This comment was marked as resolved.
justinmk
Mar 14, 2016
Author
Member
Oh, I get it: this problem occurs because of rbuffer_read_ptr() exposing the raw buffer to callers (defeating the ring-buffer abstraction).
This comment has been minimized.
This comment has been minimized.
|
@justinmk I'm not sure I understand the goal behind this PR, is it to decouple the logic of automatically toggling If so, I don't see much gain here: We still have bracketed paste parsing hardcoded into the TUI, only the
If you could implement this, then yes, we could completely decouple bracketed paste(and possibly many other terminal sequences) from the tui. But it should not be However, this seems to overlap with @bfredl's work on #4419 : map <command> <Paste> if &paste | set nopaste | else | set pasteMaybe it would be even better if we simply supported user-defined synthetic keys: map <command> <User:SOME_KEY_ID> if &paste | set nopaste | else | set pasteThe question is: How can a plugin tell the UI to put such synthetic keys into the input buffer? Right now we can't, but the TUI will soon allow configuration, so a bracketed paste plugin could be summarized to something like this: let g:tui_keys_override = {
\ '\x1b[200~': '<User:Paste>'
\ '\x1b[201~': '<User:Paste>'
\}
imap <command> <User:Paste> if &paste | set nopaste | else | set pasteThat is, the plugin asks the tui to translate bracketed pastes into |
| @@ -328,6 +326,9 @@ int main(int argc, char **argv) | |||
| "'\\c\\m" PROTO "\\(.\\{-}\\)//'), 1, '')})"); | |||
| #undef PROTO | |||
|
|
|||
| do_cmdline_cmd("autocmd PastePre * set paste|if mode()!~#'c\\|t'|call feedkeys('\x1C\xEi','n')|endif"); | |||
| do_cmdline_cmd("autocmd PastePost * set nopaste|if mode()!~#'c\\|t'|call feedkeys('\x1C\xE','n')|endif"); | |||
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
ZyX-I
Mar 14, 2016
Contributor
And I would also suggest to create autocmd group __Builtin for all built-in autocommands (currently only this one and term://* BufReadCmd), so that they are easy to identify, making user aware of what he sees and, possibly, changes.
This comment has been minimized.
This comment has been minimized.
ZyX-I
Mar 14, 2016
Contributor
Regex is better written as stridx('ct', mode()) == -1 or mode() !~# '[ct]'.
This comment was marked as resolved.
This comment was marked as resolved.
This fixes several bugs, and removes some special-case logic.
Hard-coded handling of bracketed paste belongs in
I addressed that here: #3906 (comment) I'll also add that some users would want to control whether they return to normal-mode after a paste, or whether a newline is always created. Moreover, @bfredl mentioned wanting
I don't think so. The The old code was sending |
This comment was marked as resolved.
This comment was marked as resolved.
|
|
This comment was marked as resolved.
This comment was marked as resolved.
|
@bfredl See the PR description where I mention that blackhole register could be used to mean "external paste" (called "bracketed paste" in terminals, but for all other UIs it's a "clipboard paste"). IOW, though this PR is narrow at the moment, it's just a first step (which fixes very old bugs). I can rename the events to |
This comment was marked as outdated.
This comment was marked as outdated.
Not really, |
This comment has been minimized.
This comment has been minimized.
|
@justinmk Bracketed paste is terminal-UI-specific thing. GUIs do not have anything like this, all they have are events from user: clipboard and mouse. There is no “external paste”, there are things like |
This comment has been minimized.
This comment has been minimized.
|
@justinmk |
This comment was marked as resolved.
This comment was marked as resolved.
|
That two different operations sometimes can have the the same goal does not make them the same operation, I don't see any pedantry in that. |
This comment was marked as resolved.
This comment was marked as resolved.
|
@justinmk It is only TUI which has to “raise PastePre, send bytes, PastePost” (and “has” is mostly because Neovim needs to be prepared to missing PastePost). Other UIs may say “paste this”, but they don’t need this at all, there is no “external” paste, nobody will raise PastePre, PastePost and send bytes because pasting is done by When “Paste*” is used for handling pastes from register |
This comment was marked as resolved.
This comment was marked as resolved.
Not sure I follow. We increment
Why couldn't we recommend that UIs raise the events? This would enable users to have consistent paste behavior for any UI. E.g., some users may want to return to normal-mode always after a paste. Some users may want to respond to a non-modifiable buffer by opening a new tab instead of just showing an error.
Fair point, then we can drop that idea. It's not part of this PR. |
This comment was marked as resolved.
This comment was marked as resolved.
|
Sorry meant TextPutPre: It could change the text/type pair being pasted but that doesn't make any sense for PastePre. |
This comment has been minimized.
This comment has been minimized.
|
By the way, consider that user pressed Also I saw that you test ignoring second
(with
: no escaping. Spurious |
This comment has been minimized.
This comment has been minimized.
The text is not provided in any register or
Thanks, I will fix it. |
This comment has been minimized.
This comment has been minimized.
Alright. What about your suggestion of adding a generic mechanism to allow execution of vimscript when a certain synthetic keys are processed? Assuming the TUI allows the user to override terminal code parsing(it will), a bracketed paste plugin could do this: let g:tui_keys_override = {
\ '\x1b[200~': '<User:PasteEnable>'
\ '\x1b[201~': '<User:PasteDisable>'
\}It seems like something that could work. Even FocusGain/FocusLost could work nicely on such infrastructure. |
This comment has been minimized.
This comment has been minimized.
|
I also do not see tests for pasting while in all other modes (visual, select, operator-pending, replace, virtual replace). |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
I suppose that would be fine if no one is convinced that PastePre/PastePost will be useful for GUIs (I gave use-cases above). But:
|
This comment has been minimized.
This comment has been minimized.
|
Hmm I wonder how to implement "Send paste sequences to :terminal" with that. Is the remapping done by the tui module or do you mean runtime will add those mappings to insert mode? |
This comment has been minimized.
This comment has been minimized.
|
@bfredl Neither of suggested variants allow implementing “send paste sequences to |
This is "readfile()-style", see also ":help channel-lines".
Fixes strange behavior where sometimes the buffer contents of a series of paste chunks (vim._paste) would be out-of-order. Now the tui_spec.lua screen-tests are much more reliable. But they still sometimes fail because of off-by-one cursor (caused by "typeahead race" resulting in wrong mode; fixed later in this patch-series).
- Send `phase` parameter to the paste handler. - Redraw at intervals and when paste terminates. - Show "..." throbber during paste to indicate activity.
Workaround this failure:
[ ERROR ] test/functional/terminal/tui_spec.lua @ 192: TUI paste: exactly 64 bytes
test/functional/helpers.lua:403:
retry() attempts: 478
test/functional/terminal/tui_spec.lua:201: Expected objects to be the same.
Passed in:
(table: 0x47cd77e8) {
*[1] = 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz endz' }
Expected:
(table: 0x47cd7830) {
*[1] = 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz end' }
This happens because `curwin->w_cursor.col` is sometimes decremented at
the end of `do_put`... because the editor is in Normal-mode instead of
the expected Insert-mode.
Caused by "typeahead race" (#10826): there may be queued input in the
main thread not yet processed, thus the editor mode (`State` global)
will be "wrong" during paste. Example: input "i" followed immediately by
a paste sequence:
i<start-paste>...<stop-paste>
^
"i" does not get processed in time, so the editor is in
Normal-mode instead of Insert-mode while handling the paste.
Attempted workarounds:
- vim.api.nvim_feedkeys('','x',false) in vim._paste()
- exec_normal() in tinput_wait_enqueue()
- LOOP_PROCESS_EVENTS(&main_loop,…,0) in tinput_wait_enqueue()
ref #10826
- Normal-mode redo idiom(?): prepend "i" and append ESC. - Insert-mode only needs AppendToRedobuffLit(). - Cmdline-mode: only paste the first line.
HACK: The cursor does not get repositioned after the paste completes. Scheduling a dummy event seems to fix it. Test case: 0. Revert this commit. 1. Paste some text in Normal-mode. 2. Notice the cursor is still in the cmdline area.
- Show error only once per "paste stream". - Drain remaining chunks until phase=3. - Lay groundwork for "cancel". - Constrain semantics of "cancel" to mean "client must stop"; it is unrelated to presence of error(s).
- nvim_paste(): Marshal through luaeval() instead of nvim_execute_lua()
because the latter seems to hide some errors.
- Handle 'nomodifiable' in `nvim_put()` explicitly.
- Require explicit `false` from `vim.paste()` in order to "cancel",
otherwise assume true ("continue").
This comment has been minimized.
This comment has been minimized.
Resolution/Summary
Notes:
Locked to keep the summary visible. You can always chat or open a ticket if you have new information/topics to discuss. |
99a1753
to
fee2e1c
- Introduce TRY_WRAP() until we have an *architectural* solution.
- TODO: bfredl idea: prepare error-handling at "top level" (nv_event).
- nvim_paste(): Revert luaeval() hack (see parent commit).
- With TRY_WRAP() in nvim_put(), 'nomodifiable' error now correctly
"bubbles up".
82d52b2
into
neovim:master
justinmk commentedMar 14, 2016
•
edited
This PR improves paste behavior and performance.
filetype=cpporfiletype=ruby)vim.paste(Lua)nvim_pastefor UIs/clientsnvim_putinserts text (weird but true: this was difficult before)Tasks:
string_to_array()?nvim_put#6819nvim_pasteTODO (future):
p_ptnvim_inputchunk. #10826TextPutPre/TextPutPost(analogs forTextYankPost) ?:terminalp,:put, etc.