Tmux copy-mode copied text loss #1002

sekirocc · 2018-01-07T02:48:33Z

Which operating system does the issue occur on?
elementary OS
If on linux, are you using X11 or Wayland?
X11.

I use tmux and tmux-plugins/tmux-yank to copy text to system clipboard (through copy-mode y command). When I copied, say 30 lines, and CTRL-V paste them to sublime, there are only 10 lines pasted, the other lines are lost.

copied

➜  / ll
total 100K
drwxr-xr-x   2 root root 4.0K 1月   6 12:09 bin
drwxr-xr-x   4 root root 4.0K 1月   6 11:25 boot
drwxrwxr-x   2 root root 4.0K 1月   6 11:16 cdrom
drwxr-xr-x  20 root root 4.3K 1月   7 09:08 dev
drwxr-xr-x 141 root root  12K 1月   7 09:15 etc
drwxr-xr-x   3 root root 4.0K 1月   6 11:17 home
lrwxrwxrwx   1 root root   33 1月   6 11:24 initrd.img -> boot/initrd.img-4.4.0-104-generic
lrwxrwxrwx   1 root root   33 1月   6 11:17 initrd.img.old -> boot/initrd.img-4.10.0-32-generic
drwxr-xr-x  23 root root 4.0K 1月   6 11:23 lib
drwxr-xr-x   2 root root 4.0K 1月   6 11:23 lib32
drwxr-xr-x   2 root root 4.0K 8月  15 03:24 lib64
drwx------   2 root root  16K 1月   6 11:13 lost+found
drwxr-xr-x   3 root root 4.0K 1月   6 15:39 media
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 mnt
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 opt
dr-xr-xr-x 255 root root    0 1月   7 09:08 proc
drwx------   3 root root 4.0K 8月  15 03:39 root
drwxr-xr-x  28 root root  860 1月   7 09:13 run
drwxr-xr-x   2 root root  12K 1月   6 11:23 sbin
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 srv
dr-xr-xr-x  13 root root    0 1月   7 10:44 sys
drwxrwxrwt  14 root root 4.0K 1月   7 10:38 tmp
drwxr-xr-x  12 root root 4.0K 1月   6 11:23 usr
drwxr-xr-x  13 root root 4.0K 8月  15 03:39 var
lrwxrwxrwx   1 root root   30 1月   6 11:24 vmlinuz -> boot/vmlinuz-4.4.0-104-generic
lrwxrwxrwx   1 root root   30 1月   6 11:17 vmlinuz.old -> boot/vmlinuz-4.10.0-32-generic

pasted

➜  / ll
total 100K
drwxr-xr-x   2 root root 4.0K 1月   6 12:09 bin
drwxr-xr-x   4 root root 4.0K 1月   6 11:25 boot
drwxrwxr-x   2 root root 4.0K 1月   6 11:16 cdrom
drwxr-xr-x  20 root root 4.3K 1月   7 09:08 dev
drwxr-xr-x 141 root root  12K 1月   7 09:15 etc
drwxr-xr-x   3 root root 4.0K 1月   6 11:17 home
lrwxrwxrwx   1 root root   33 1月   6 11:24 initrd.img -> boot/initrd.img-4.4.0-104-generic
lrwxrwxrwx   1 root root   33 1月   6 11:17 initrd.img.old -> boot/initrd.img-4.10.0-32-generic
drwxr-xr-x  23 root root 4.0K 1月   6 11:23 lib
drwxr-xr-x   2 root root 4.0K 1月   6 11:23 lib32
drwxr-xr-x   2 root root 4.0K 8月  15 03:24 lib64
drwx------   2 root root  16K 1月   6 11:13 lost+found
drwxr-xr-x   3 root root 4.0K 1月   6 15:39 media

Tested with system terminal, It works fine.

The text was updated successfully, but these errors were encountered:

maximbaz · 2018-01-07T11:44:26Z

Curiously I use the same setup and I cannot reproduce your issue.

Arch Linux, X11, tmux 2.6, alacritty b600e40

sekirocc · 2018-01-08T02:35:42Z

Maybe it's my tmux is too old? I'm using tmux 2.3, and alacritty is built from master.
I will upgrade to latest tmux and test it again.

update : when upgrade to tmux 2.6, the issue was gone. It works as expected. I didnot use any mouse related config.

2solt · 2018-01-08T19:37:01Z

Im loosing lines as well, but only with set -g mouse on
So as a workaround is witched it off.

tmux 2.6, alacritty b600e40

maximbaz · 2018-01-08T19:55:44Z

Can you give me your .tmux.conf and alacritty.yml and exact steps of how you reproduce it? How you select, how you paste, etc? I do have set -g mouse on and it works, so I'm curious to reproduce.

2solt · 2018-01-08T21:24:04Z

.tmux.conf

set -g mouse on
unbind -T copy-mode MouseDragEnd1Pane
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-yank'
set -g @plugin 'laktak/extrakto'
run '~/.tmux/plugins/tpm/tpm'

alacritty.yaml is the default (i deleted to regenerate)

So its probably unbind -T copy-mode MouseDragEnd1Pane is the problem.
If I use key combinations to enter to copy-mode and yank [y] without mouse, everything is ok.
As soon as I select lines with the mouse, the problem start to show up, so evey time when I try to paste, it only around 16 lines.
I just doing an ls -la and try to copy ~40 lines.

2solt · 2018-01-08T21:49:00Z

It happens without unbind -T copy-mode MouseDragEnd1Pane as well.
Its easiest to reproduce when you try to select more than a screenful of lines, so you have to scroll up while you in copy mode.

maximbaz · 2018-01-08T22:42:13Z

All right, it took me a while but I managed to reproduce at least something! Here's what I've got:

tmux-yank doesn't have any issue, how ever I try to select, it just works flawlessly.
MouseDragEnd1Pane is the issue, when it is not unbound! By default, MouseDragEnd1Pane is bound to copy-selection-and-cancel, and this is what copies only parts of the selection. As soon as I add unbind -T copy-mode MouseDragEnd1Pane, the default copy doesn't kick in anymore and I can use y to copy text with tmux-yank and it always copies the selection in full.

In summary, I reproduce the issue with the default behavior of MouseDragEnd1Pane. I don't reproduce in in daily usage because in my own .tmux.conf I also do unbind -T copy-mode MouseDragEnd1Pane and I always use tmux-yank, which seems to work flawlessly.

Alternative to unbinding MouseDragEnd1Pane is to add set -g set-clipboard off to your .tmux.conf, this basically disables (broken) automatic copy on selection.

And now it all makes sense, because set-clipboard was implemented only recently in #970.

Let's kindly invite @geertj to the thread. TL;DR: set-clipboard doesn't copy the entire selection.

geertj · 2018-01-10T02:27:28Z

@maximbaz, I've not used tmux-yank myself, but from your description I understand that it's a tmux plugin to copy a selection to the system clipboard. Do you know how it does that? Does it use the OSC 52 escape code that #970 implemented? I guess not, because that's only there since last week.

Since tmux-yank and set-clipboard do essentially the same thing, I think you'd have to use one or the other. If you disable tmux-yank and enable set-clipboard, do you still see partial data on the clipboard? If so, could you add a println!() to the OSC 52 handler in src/ansi.rs to see if it may be called multiple times, and what the parameters are?

maximbaz · 2018-01-10T11:53:00Z

tmux-yank pipes the contents to xclip or xsel, so it doesn't use OSC 52 escape code. Yes, set-clipboard is copying partial data with or without tmux-yank enabled. Personally I always had set-clipboard disabled, that's why I haven't noticed this issue, but people above had it enabled and thus after it got implemented, it started conflicting with tmux-yank.

Something weird happened, I can't get set-clipboard to work today even with the same ~/.tmux.conf pasted above, very confusing. I'll try again in the evening, but @2solt, @Nickelchen if you could try to add the println!() to get the data, please do.

maximbaz · 2018-01-10T16:07:46Z

Mystery solved, apparently set-clipboard works only when TERM=xterm*, and usually I use TERM=alacritty-256colors.

Anyway, now I can answer your question @geertj. OSC 52 handler is called only once per selection, and the parameters are [[53, 50], [], [...]], where the last argument is the array of numbers of variable size (they represent ascii codes of base64-encrypted string to be precise). Now what's interesting is that no matter how large selection I make, the length of this third argument never exceeds 1022 elements.

So I'm not sure what to blame exactly, since osc_dispatch function is already called with the cut-off data, and it is not being called multiple times.

Click to see one example

Here's the arguments for osc_dispatch when I copy the result of tree command over the alacritty's repo:

[[53, 50], [], [76, 103, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 87, 120, 104, 89, 51, 74, 112, 100, 72, 82, 53, 76, 109, 82, 108, 99, 50, 116, 48, 98, 51, 65, 75, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 70, 115, 89, 87, 78, 121, 97, 88, 82, 48, 101, 83, 53, 112, 98, 109, 90, 118, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 108, 102, 98, 87, 70, 106, 98, 51, 77, 117, 101, 87, 49, 115, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 107, 117, 101, 87, 49, 115, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 99, 51, 78, 108, 100, 72, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 98, 51, 78, 52, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 66, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 107, 117, 89, 88, 66, 119, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 57, 117, 100, 71, 86, 117, 100, 72, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 68, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 83, 87, 53, 109, 98, 121, 53, 119, 98, 71, 108, 122, 100, 65, 114, 105, 108, 73, 76, 67, 111, 77, 75, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 83, 90, 88, 78, 118, 100, 88, 74, 106, 90, 88, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 85, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 70, 115, 89, 87, 78, 121, 97, 88, 82, 48, 101, 83, 53, 112, 89, 50, 53, 122, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 105, 100, 87, 108, 115, 90, 67, 53, 121, 99, 119, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 70, 121, 90, 50, 56, 117, 98, 71, 57, 106, 97, 119, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 70, 121, 90, 50, 56, 117, 100, 71, 57, 116, 98, 65, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 89, 50, 57, 119, 101, 88, 66, 104, 99, 51, 82, 104, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 69, 78, 104, 99, 109, 100, 118, 76, 110, 82, 118, 98, 87, 119, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 84, 69, 108, 68, 82, 85, 53, 84, 82, 83, 49, 66, 85, 69, 70, 68, 83, 69, 85, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 99, 51, 74, 106, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 115, 97, 87, 73, 117, 99, 110, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 49, 104, 89, 50, 57, 122, 76, 110, 74, 122, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 52, 77, 84, 69, 117, 99, 110, 77, 75, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 82, 118, 89, 51, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 89, 87, 53, 122, 97, 87, 78, 118, 90, 71, 85, 117, 100, 72, 104, 48, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 109, 98, 50, 53, 48, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 69, 78, 104, 99, 109, 100, 118, 76, 110, 82, 118, 98, 87, 119, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 99, 51, 74, 106, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 107, 89, 88, 74, 51, 97, 87, 52, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74]]

If I convert this manually to a string and then try to decode the string, I'll get the following:

.
├── Alacritty.desktop
├── alacritty.info
├── alacritty_macos.yml
├── alacritty.yml
├── assets
│   └── osx
│       └── Alacritty.app
│           └── Contents
│               ├── Info.plist
│               └── Resources
│                   └── alacritty.icns
├── build.rs
├── Cargo.lock
├── Cargo.toml
├── copypasta
│   ├── Cargo.toml
│   ├── LICENSE-APACHE
│   └── src
│       ├── lib.rs
│       ├── macos.rs
│       └── x11.rs
├── docs
│   └── ansicode.txt
├── font
│   ├── Cargo.toml
│   └── src
│       ├── darwin
│       │   ase64: invalid input

jwilm · 2018-01-10T17:36:11Z

According to the tmux manual entry for set-clipboard,

This option is on by default if there is an Ms entry in the terminfo(5) description for the client terminal.

It sounds like if we support OSC 52, we need to state as much in our terminfo file.

jwilm · 2018-01-10T17:37:32Z

Here's xterm's entry: Ms=\E]52;%p1%s;%p2%s\007

maximbaz · 2018-01-10T18:33:12Z

I thought that this line literally means "whatever xterm defines, I have that too":

https://github.com/jwilm/alacritty/blob/632ef0df7f55cad7c7694acfff69a8ba76fac4a0/alacritty.info#L1-L2

However even manually adding Ms=\E]52;%p1%s;%p2%s\007, to alacritty.info and then regenerating with sudo tic -o /usr/share/terminfo alacritty.info doesn't fix the issue, it still works only with TERM=xterm-256color.

By the way, this has also been reported to tmux, closed with "use set -g set-clipboard off".

tmux/tmux#1119

jwilm · 2018-01-10T20:32:34Z

Ah yeah, we actually have an OSC length limit from the vte state machine.

jwilm · 2018-01-10T20:33:39Z

vte could potentially fall-back to storing OSC data in a Vec<u8> when it exceeds the fast buffer's size.

This patch is a workaround for issues like alacritty/alacritty#1002 .

Valloric · 2018-04-02T00:56:56Z

To add extra info to this issue, I've had literally the exact same issue with tmux, except I can completely eliminate alacritty from the equation; the issue happens in urxvt too.

And the solution is the same: putting set -g set-clipboard off in .tmux.conf solves the problem.

HUGE thanks to @maximbaz; I've been pulling my hair out over this for the last few days.

More info here: alacritty/alacritty#1002

Whilst playing around with vte and processing iTerm2's image protocol, I ran into the relatively small statically sized OSC buffer. This diff switches the buffer from a static array to use a `Vec<u8>`. I did briefly experiment with just making this buffer much larger (as alacritty#15 does), but even at 1MB (which is too small for images) that size can lead to a stack overflow during initialization. In order to preserve building in a `no_std` environment this is done conditionally. I've followed the pattern used in other crates with similar requirements: * Introduced a `std` feature to signal that `std` is ok * The `std` feature is on by default * `no_std` crates that consume this one will need to set `vte` to `default-features = false` in their deps * I've bumped the version in order to avoid breaking `no_std` consumers Test Plan: ``` $ cargo test --no-default-features $ cargo test ``` Refs: alacritty/alacritty#1002

This change includes dynamic escape buffer support in VTE, which allows us to have arbitrary escape sizes. Since tmux could potentially use very long escapes for the clipboard escape, this allows copying more text. Fixes alacritty#1002.

This change includes dynamic escape buffer support in VTE, which allows us to have arbitrary escape sizes. Since tmux could potentially use very long escapes for the clipboard escape, this allows copying more text. Fixes #1002.

**Why** is the change needed? It is very important to quickly be able to copy output from given commands. **How** is the need addressed? 1. Don't copy on selection in alacritty 2. Use xclip on copy in tmux 3. Search forward instead of backwards to the last prompt (tmux) 4. Add helpful bindings 5. Don't set the clipboard on selection (alacritty/alacritty#1002) Closes #73

This change includes dynamic escape buffer support in VTE, which allows us to have arbitrary escape sizes. Since tmux could potentially use very long escapes for the clipboard escape, this allows copying more text. Fixes alacritty#1002.

**Why** is the change needed? It is very important to quickly be able to copy output from given commands. **How** is the need addressed? 1. Don't copy on selection in alacritty 2. Use xclip on copy in tmux 3. Search forward instead of backwards to the last prompt (tmux) 4. Add helpful bindings 5. Don't set the clipboard on selection (alacritty/alacritty#1002) Closes #73

amosbird added a commit to amosbird/vte that referenced this issue Mar 19, 2018

make osc52 buffer bigger to cover more use cases

47924e2

This patch is a workaround for issues like alacritty/alacritty#1002 .

amosbird mentioned this issue Mar 19, 2018

make osc52 buffer bigger to cover more use cases alacritty/vte#15

Closed

amalbuquerque added a commit to amalbuquerque/dotfiles that referenced this issue Sep 17, 2018

Fix bug that didn't allow to copy everything + Numbering start from 1

0bb51e7

More info here: alacritty/alacritty#1002

wez mentioned this issue Mar 17, 2019

Use dynamically sized buffer for OSC alacritty/vte#20

Closed

chrisduerr added this to the Version 0.3.5 milestone Jul 14, 2019

nixpulvis added the S - clipboard label Sep 18, 2019

chrisduerr mentioned this issue Nov 22, 2019

Add dynamic buffer support alacritty/vte#33

Merged

chrisduerr added the C - waiting on maintainer label Jan 2, 2020

chrisduerr modified the milestones: Version 0.4.1, Version 0.4.2 Jan 5, 2020

chrisduerr mentioned this issue Jan 12, 2020

Bump VTE to 0.5.0 #3194

Merged

chrisduerr removed the C - waiting on maintainer label Jan 12, 2020

chrisduerr closed this as completed in #3194 Jan 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tmux copy-mode copied text loss #1002

Tmux copy-mode copied text loss #1002

sekirocc commented Jan 7, 2018

maximbaz commented Jan 7, 2018

sekirocc commented Jan 8, 2018 •

edited

2solt commented Jan 8, 2018

maximbaz commented Jan 8, 2018

2solt commented Jan 8, 2018

2solt commented Jan 8, 2018

maximbaz commented Jan 8, 2018

geertj commented Jan 10, 2018 •

edited

maximbaz commented Jan 10, 2018

maximbaz commented Jan 10, 2018

jwilm commented Jan 10, 2018 •

edited

jwilm commented Jan 10, 2018

maximbaz commented Jan 10, 2018

jwilm commented Jan 10, 2018

jwilm commented Jan 10, 2018

Valloric commented Apr 2, 2018

Tmux copy-mode copied text loss #1002

Tmux copy-mode copied text loss #1002

Comments

sekirocc commented Jan 7, 2018

maximbaz commented Jan 7, 2018

sekirocc commented Jan 8, 2018 • edited

2solt commented Jan 8, 2018

maximbaz commented Jan 8, 2018

2solt commented Jan 8, 2018

2solt commented Jan 8, 2018

maximbaz commented Jan 8, 2018

geertj commented Jan 10, 2018 • edited

maximbaz commented Jan 10, 2018

maximbaz commented Jan 10, 2018

jwilm commented Jan 10, 2018 • edited

jwilm commented Jan 10, 2018

maximbaz commented Jan 10, 2018

jwilm commented Jan 10, 2018

jwilm commented Jan 10, 2018

Valloric commented Apr 2, 2018

sekirocc commented Jan 8, 2018 •

edited

geertj commented Jan 10, 2018 •

edited

jwilm commented Jan 10, 2018 •

edited