Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tmux copy-mode copied text loss #1002

Open
hoopchen opened this Issue Jan 7, 2018 · 16 comments

Comments

Projects
None yet
6 participants
@hoopchen
Copy link

hoopchen commented Jan 7, 2018

Which operating system does the issue occur on?
elementary OS
If on linux, are you using X11 or Wayland?
X11.

I use tmux and tmux-plugins/tmux-yank to copy text to system clipboard (through copy-mode y command). When I copied, say 30 lines, and CTRL-V paste them to sublime, there are only 10 lines pasted, the other lines are lost.

copied

➜  / ll
total 100K
drwxr-xr-x   2 root root 4.0K 1月   6 12:09 bin
drwxr-xr-x   4 root root 4.0K 1月   6 11:25 boot
drwxrwxr-x   2 root root 4.0K 1月   6 11:16 cdrom
drwxr-xr-x  20 root root 4.3K 1月   7 09:08 dev
drwxr-xr-x 141 root root  12K 1月   7 09:15 etc
drwxr-xr-x   3 root root 4.0K 1月   6 11:17 home
lrwxrwxrwx   1 root root   33 1月   6 11:24 initrd.img -> boot/initrd.img-4.4.0-104-generic
lrwxrwxrwx   1 root root   33 1月   6 11:17 initrd.img.old -> boot/initrd.img-4.10.0-32-generic
drwxr-xr-x  23 root root 4.0K 1月   6 11:23 lib
drwxr-xr-x   2 root root 4.0K 1月   6 11:23 lib32
drwxr-xr-x   2 root root 4.0K 8月  15 03:24 lib64
drwx------   2 root root  16K 1月   6 11:13 lost+found
drwxr-xr-x   3 root root 4.0K 1月   6 15:39 media
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 mnt
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 opt
dr-xr-xr-x 255 root root    0 1月   7 09:08 proc
drwx------   3 root root 4.0K 8月  15 03:39 root
drwxr-xr-x  28 root root  860 1月   7 09:13 run
drwxr-xr-x   2 root root  12K 1月   6 11:23 sbin
drwxr-xr-x   2 root root 4.0K 8月  15 03:23 srv
dr-xr-xr-x  13 root root    0 1月   7 10:44 sys
drwxrwxrwt  14 root root 4.0K 1月   7 10:38 tmp
drwxr-xr-x  12 root root 4.0K 1月   6 11:23 usr
drwxr-xr-x  13 root root 4.0K 8月  15 03:39 var
lrwxrwxrwx   1 root root   30 1月   6 11:24 vmlinuz -> boot/vmlinuz-4.4.0-104-generic
lrwxrwxrwx   1 root root   30 1月   6 11:17 vmlinuz.old -> boot/vmlinuz-4.10.0-32-generic

pasted

➜  / ll
total 100K
drwxr-xr-x   2 root root 4.0K 1月   6 12:09 bin
drwxr-xr-x   4 root root 4.0K 1月   6 11:25 boot
drwxrwxr-x   2 root root 4.0K 1月   6 11:16 cdrom
drwxr-xr-x  20 root root 4.3K 1月   7 09:08 dev
drwxr-xr-x 141 root root  12K 1月   7 09:15 etc
drwxr-xr-x   3 root root 4.0K 1月   6 11:17 home
lrwxrwxrwx   1 root root   33 1月   6 11:24 initrd.img -> boot/initrd.img-4.4.0-104-generic
lrwxrwxrwx   1 root root   33 1月   6 11:17 initrd.img.old -> boot/initrd.img-4.10.0-32-generic
drwxr-xr-x  23 root root 4.0K 1月   6 11:23 lib
drwxr-xr-x   2 root root 4.0K 1月   6 11:23 lib32
drwxr-xr-x   2 root root 4.0K 8月  15 03:24 lib64
drwx------   2 root root  16K 1月   6 11:13 lost+found
drwxr-xr-x   3 root root 4.0K 1月   6 15:39 media

Tested with system terminal, It works fine.

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 7, 2018

Curiously I use the same setup and I cannot reproduce your issue.

Arch Linux, X11, tmux 2.6, alacritty b600e40

@hoopchen

This comment has been minimized.

Copy link
Author

hoopchen commented Jan 8, 2018

Maybe it's my tmux is too old? I'm using tmux 2.3, and alacritty is built from master.
I will upgrade to latest tmux and test it again.

update : when upgrade to tmux 2.6, the issue was gone. It works as expected. I didnot use any mouse related config.

@2solt

This comment has been minimized.

Copy link

2solt commented Jan 8, 2018

Im loosing lines as well, but only with set -g mouse on
So as a workaround is witched it off.

tmux 2.6, alacritty b600e40

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 8, 2018

Can you give me your .tmux.conf and alacritty.yml and exact steps of how you reproduce it? How you select, how you paste, etc? I do have set -g mouse on and it works, so I'm curious to reproduce.

@2solt

This comment has been minimized.

Copy link

2solt commented Jan 8, 2018

.tmux.conf

set -g mouse on
unbind -T copy-mode MouseDragEnd1Pane
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-yank'
set -g @plugin 'laktak/extrakto'
run '~/.tmux/plugins/tpm/tpm'

alacritty.yaml is the default (i deleted to regenerate)

So its probably unbind -T copy-mode MouseDragEnd1Pane is the problem.
If I use key combinations to enter to copy-mode and yank [y] without mouse, everything is ok.
As soon as I select lines with the mouse, the problem start to show up, so evey time when I try to paste, it only around 16 lines.
I just doing an ls -la and try to copy ~40 lines.

@2solt

This comment has been minimized.

Copy link

2solt commented Jan 8, 2018

It happens without unbind -T copy-mode MouseDragEnd1Pane as well.
Its easiest to reproduce when you try to select more than a screenful of lines, so you have to scroll up while you in copy mode.

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 8, 2018

All right, it took me a while but I managed to reproduce at least something! Here's what I've got:

  1. tmux-yank doesn't have any issue, how ever I try to select, it just works flawlessly.
  2. MouseDragEnd1Pane is the issue, when it is not unbound! By default, MouseDragEnd1Pane is bound to copy-selection-and-cancel, and this is what copies only parts of the selection. As soon as I add unbind -T copy-mode MouseDragEnd1Pane, the default copy doesn't kick in anymore and I can use y to copy text with tmux-yank and it always copies the selection in full.

In summary, I reproduce the issue with the default behavior of MouseDragEnd1Pane. I don't reproduce in in daily usage because in my own .tmux.conf I also do unbind -T copy-mode MouseDragEnd1Pane and I always use tmux-yank, which seems to work flawlessly.

Alternative to unbinding MouseDragEnd1Pane is to add set -g set-clipboard off to your .tmux.conf, this basically disables (broken) automatic copy on selection.

And now it all makes sense, because set-clipboard was implemented only recently in #970.

Let's kindly invite @geertj to the thread. TL;DR: set-clipboard doesn't copy the entire selection.

@geertj

This comment has been minimized.

Copy link
Contributor

geertj commented Jan 10, 2018

@maximbaz, I've not used tmux-yank myself, but from your description I understand that it's a tmux plugin to copy a selection to the system clipboard. Do you know how it does that? Does it use the OSC 52 escape code that #970 implemented? I guess not, because that's only there since last week.

Since tmux-yank and set-clipboard do essentially the same thing, I think you'd have to use one or the other. If you disable tmux-yank and enable set-clipboard, do you still see partial data on the clipboard? If so, could you add a println!() to the OSC 52 handler in src/ansi.rs to see if it may be called multiple times, and what the parameters are?

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 10, 2018

tmux-yank pipes the contents to xclip or xsel, so it doesn't use OSC 52 escape code. Yes, set-clipboard is copying partial data with or without tmux-yank enabled. Personally I always had set-clipboard disabled, that's why I haven't noticed this issue, but people above had it enabled and thus after it got implemented, it started conflicting with tmux-yank.

Something weird happened, I can't get set-clipboard to work today even with the same ~/.tmux.conf pasted above, very confusing. I'll try again in the evening, but @2solt, @nickelchen if you could try to add the println!() to get the data, please do.

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 10, 2018

Mystery solved, apparently set-clipboard works only when TERM=xterm*, and usually I use TERM=alacritty-256colors.

Anyway, now I can answer your question @geertj. OSC 52 handler is called only once per selection, and the parameters are [[53, 50], [], [...]], where the last argument is the array of numbers of variable size (they represent ascii codes of base64-encrypted string to be precise). Now what's interesting is that no matter how large selection I make, the length of this third argument never exceeds 1022 elements.

So I'm not sure what to blame exactly, since osc_dispatch function is already called with the cut-off data, and it is not being called multiple times.

Click to see one example

Here's the arguments for osc_dispatch when I copy the result of tree command over the alacritty's repo:

[[53, 50], [], [76, 103, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 87, 120, 104, 89, 51, 74, 112, 100, 72, 82, 53, 76, 109, 82, 108, 99, 50, 116, 48, 98, 51, 65, 75, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 70, 115, 89, 87, 78, 121, 97, 88, 82, 48, 101, 83, 53, 112, 98, 109, 90, 118, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 108, 102, 98, 87, 70, 106, 98, 51, 77, 117, 101, 87, 49, 115, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 107, 117, 101, 87, 49, 115, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 104, 99, 51, 78, 108, 100, 72, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 98, 51, 78, 52, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 66, 98, 71, 70, 106, 99, 109, 108, 48, 100, 72, 107, 117, 89, 88, 66, 119, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 57, 117, 100, 71, 86, 117, 100, 72, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 68, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 83, 87, 53, 109, 98, 121, 53, 119, 98, 71, 108, 122, 100, 65, 114, 105, 108, 73, 76, 67, 111, 77, 75, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 83, 90, 88, 78, 118, 100, 88, 74, 106, 90, 88, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 85, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 70, 115, 89, 87, 78, 121, 97, 88, 82, 48, 101, 83, 53, 112, 89, 50, 53, 122, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 105, 100, 87, 108, 115, 90, 67, 53, 121, 99, 119, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 70, 121, 90, 50, 56, 117, 98, 71, 57, 106, 97, 119, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 81, 50, 70, 121, 90, 50, 56, 117, 100, 71, 57, 116, 98, 65, 114, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 89, 50, 57, 119, 101, 88, 66, 104, 99, 51, 82, 104, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 69, 78, 104, 99, 109, 100, 118, 76, 110, 82, 118, 98, 87, 119, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 122, 105, 108, 73, 68, 105, 108, 73, 65, 103, 84, 69, 108, 68, 82, 85, 53, 84, 82, 83, 49, 66, 85, 69, 70, 68, 83, 69, 85, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 99, 51, 74, 106, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 115, 97, 87, 73, 117, 99, 110, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 49, 104, 89, 50, 57, 122, 76, 110, 74, 122, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 108, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 52, 77, 84, 69, 117, 99, 110, 77, 75, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 71, 82, 118, 89, 51, 77, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 89, 87, 53, 122, 97, 87, 78, 118, 90, 71, 85, 117, 100, 72, 104, 48, 67, 117, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 109, 98, 50, 53, 48, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 52, 112, 83, 99, 52, 112, 83, 65, 52, 112, 83, 65, 73, 69, 78, 104, 99, 109, 100, 118, 76, 110, 82, 118, 98, 87, 119, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74, 84, 105, 108, 73, 68, 105, 108, 73, 65, 103, 99, 51, 74, 106, 67, 117, 75, 85, 103, 115, 75, 103, 119, 113, 65, 103, 73, 67, 65, 103, 73, 79, 75, 85, 110, 79, 75, 85, 103, 79, 75, 85, 103, 67, 66, 107, 89, 88, 74, 51, 97, 87, 52, 75, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 65, 103, 73, 67, 65, 103, 52, 112, 83, 67, 119, 113, 68, 67, 111, 67, 68, 105, 108, 74]]

If I convert this manually to a string and then try to decode the string, I'll get the following:

.
├── Alacritty.desktop
├── alacritty.info
├── alacritty_macos.yml
├── alacritty.yml
├── assets
│   └── osx
│       └── Alacritty.app
│           └── Contents
│               ├── Info.plist
│               └── Resources
│                   └── alacritty.icns
├── build.rs
├── Cargo.lock
├── Cargo.toml
├── copypasta
│   ├── Cargo.toml
│   ├── LICENSE-APACHE
│   └── src
│       ├── lib.rs
│       ├── macos.rs
│       └── x11.rs
├── docs
│   └── ansicode.txt
├── font
│   ├── Cargo.toml
│   └── src
│       ├── darwin
│       │   ase64: invalid input
@jwilm

This comment has been minimized.

Copy link
Owner

jwilm commented Jan 10, 2018

According to the tmux manual entry for set-clipboard,

This option is on by default if there is an Ms entry in the terminfo(5) description for the client terminal.

It sounds like if we support OSC 52, we need to state as much in our terminfo file.

@jwilm

This comment has been minimized.

Copy link
Owner

jwilm commented Jan 10, 2018

Here's xterm's entry: Ms=\E]52;%p1%s;%p2%s\007

@maximbaz

This comment has been minimized.

Copy link
Contributor

maximbaz commented Jan 10, 2018

I thought that this line literally means "whatever xterm defines, I have that too":

alacritty| alacritty,
use=xterm-256color,

However even manually adding Ms=\E]52;%p1%s;%p2%s\007, to alacritty.info and then regenerating with sudo tic -o /usr/share/terminfo alacritty.info doesn't fix the issue, it still works only with TERM=xterm-256color.

By the way, this has also been reported to tmux, closed with "use set -g set-clipboard off".

tmux/tmux#1119

@jwilm

This comment has been minimized.

Copy link
Owner

jwilm commented Jan 10, 2018

Ah yeah, we actually have an OSC length limit from the vte state machine.

@jwilm

This comment has been minimized.

Copy link
Owner

jwilm commented Jan 10, 2018

vte could potentially fall-back to storing OSC data in a Vec<u8> when it exceeds the fast buffer's size.

amosbird added a commit to amosbird/vte that referenced this issue Mar 19, 2018

make osc52 buffer bigger to cover more use cases
This patch is a workaround for issues like jwilm/alacritty#1002 .
@Valloric

This comment has been minimized.

Copy link

Valloric commented Apr 2, 2018

To add extra info to this issue, I've had literally the exact same issue with tmux, except I can completely eliminate alacritty from the equation; the issue happens in urxvt too.

And the solution is the same: putting set -g set-clipboard off in .tmux.conf solves the problem.

HUGE thanks to @maximbaz; I've been pulling my hair out over this for the last few days.

amalbuquerque added a commit to amalbuquerque/dotfiles that referenced this issue Sep 17, 2018

wez added a commit to wez/vte that referenced this issue Mar 17, 2019

Use dynamically sized buffer for OSC
Whilst playing around with vte and processing iTerm2's image protocol,
I ran into the relatively small statically sized OSC buffer.

This diff switches the buffer from a static array to use a `Vec<u8>`.  I
did briefly experiment with just making this buffer much larger (as
jwilm#15 does), but even at 1MB (which is
too small for images) that size can lead to a stack overflow during
initialization.

In order to preserve building in a `no_std` environment this is done
conditionally.  I've followed the pattern used in other crates with
similar requirements:

* Introduced a `std` feature to signal that `std` is ok
* The `std` feature is on by default
* `no_std` crates that consume this one will need to set `vte` to
  `default-features = false` in their deps
* I've bumped the version in order to avoid breaking `no_std`
  consumers

Test Plan:

```
$ cargo test --no-default-features
$ cargo test
```

Refs: jwilm/alacritty#1002
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.