Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Mouse/Touch/Pointer Bindings (like middle-click paste, right-click context menu, etc.) #1553

Open
1 of 14 tasks
carlos-zamora opened this issue Jun 24, 2019 · 68 comments
Assignees
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Area-Settings Issues related to settings and customizability, for console or terminal Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Help Wanted We encourage anyone to jump in on these. Issue-Scenario Product-Terminal The new Windows Terminal.

Comments

@carlos-zamora
Copy link
Member

carlos-zamora commented Jun 24, 2019

Summary of the new feature/enhancement

Expand settings to be able to define remappable mouse bindings. Arguably, different touch events should fall under this too. So let's just call this "pointer bindings" for now.

Proposed technical implementation details (optional)

Mouse bindings are a bit trickier than keybindings in that the mouse has a location where the event occurs. For example, right-clicking a tab should have a different effect on the terminal than right-clicking the terminal.

As a super early idea, consider this format:

"pointerbindings": [
{
  "device": "mouse",
  "event": "rightClick",
  "where": "tab",
  "command": "flyoutMenu"
},
{
  "device": "mouse",
  "event": "doubleClick",
  "where": "terminal",
  "command": "wordSelection"  
},
{
  "device": "touch",
  "event": "swipe",
  "where": "terminal",
  "command": "scroll"
}
]

We definitely need a spec for this because it'll be a bit hefty. We'll also need to update a decent amount of documentation (particularly settings schema) to be able to describe what combinations are acceptable (again, the JSON mentioned up here is just me rambling about a potential implementation.)

There may be overlap between some commands in keybindings. Be sure to think that through.

Mike notes:

we should go back and collect up all the threads we've said "this would be a good mouse bindings feature" below


spec draft

Mouse bindings

Abstract

We've had numerous requests to configure how the mouse behaves in the Terminal.
The original behavior was a simple duplication of how conhost behaved: a right
click will copy the a selection if there is one, or paste the clipboard if there
isn't. Over time, we've accumulated a number of scenarios that we believe can
all be addressed by allowing more fine-grained mouse binding support. However,
upon further review, ultimately, we don't really need deep mouse binding
support.

Scenarios

The following is a list of all the feature requests we've linked to mouse
bindings in the past, grouped into categories of related requests:

Change how mouse multi-click selects

Change the action that L/M/R-mouse performs

  • #7646 xterm-style selection, paste on middle click, copy on mouse release
  • #10802 - VK_XBUTTON1/2, etc.
  • #6250 - separate "Paste Clipboard" vs "Paste Selection" actions
  • #3337 - Add a right-click context menu

Other

These are smaller, independent features that could all have an individual setting (if needed)

  • #11710 Request: Setting to disable zooming with ctrl+mousewheel
  • #13598 Add option to configure URL clicking behavior
  • #11906 Option to disable "Pinch to Zoom"
  • #6182 Fast scroll by holding down Alt while scrolling with mouse wheel
  • Block selection by default (without alt) (see mail thread "RE: How to disable line wrap selection in Terminal")
  • #17610 Configure block select to use ctrl, not alt

Solution design

Following the above scenarios, solutions are proposed below:

Change how mouse multi-click selects

Across the requests here, we've got the following requests:

  • double-click: selects a "word" between 2 same delimiters
  • triple-click: selects an entire string separated by spaces
  • 4-click: entire line

Currently, Ctrl+A will select the entire command/text input, however, triple
clicking (mouse version of Select All selects the entire line (including the
prompt). GIF shows selecting with mouse vs with keyboard:
...
I would like the triple click action to align to the Ctrl+A selection method.

Could we maybe add shift double click to select using alternate word
delimiters?

I was really thinking more of regex though, because it can be a good starting
point for implementing more advanced features like type-specific smart
highlighting and hyperlinking of terminal text, not just smart selection.

To boil this down, users want to be able to configure the behavior of double,
triple, and quadruple clicks. The most common request is to change the
delimiters for double-click selection. But users also want to be able to
configure the delimiters to change on
Shift/Alt/Ctrl clicks.

"mouse": {
    "clicks": {
        { "click": "double", "command": { "action": "expandSelection", "delimeters": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502" } }
        { "click": "shift+double", "command": { "action": "expandSelection", "delimeters": " " } }
        { "click": "triple", "command": { "action": "expandSelection", "regex": "^.*$" } }
    }
}

Alternatively,

"mouse": {
    "doubleClick": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502",
    "tripleClick": { "regex": "^.*$" }
}
@carlos-zamora carlos-zamora added the Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. label Jun 24, 2019
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Jun 24, 2019
@carlos-zamora carlos-zamora added Area-Interaction Interacting with the vintage console window (as opposed to driving via API or hooks) Area-Settings Issues related to settings and customizability, for console or terminal Help Wanted We encourage anyone to jump in on these. Issue-Docs It's a documentation issue that really should be on MicrosoftDocs/Console-Docs Product-Terminal The new Windows Terminal. and removed Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Jun 24, 2019
@carlos-zamora carlos-zamora self-assigned this Jun 24, 2019
@carlos-zamora
Copy link
Member Author

A few things:

  • Treat this as a megathread so feel free to share your ideas here 😁
  • I'm a bit busy with Accessibility (TOP PRIORITY) at the moment so if anybody wants to start working on this, just let me know.

@philippeller
Copy link

Probably related, make for example scrolling work when using vim etc....used to work fine for example in WSL terminal

@DHowett-MSFT
Copy link
Contributor

DHowett-MSFT commented Jun 25, 2019

make .. scrolling work .. vim

That is not related. That is #376 then #545.

@schorrm
Copy link
Contributor

schorrm commented Jun 27, 2019

Can I pile on here -- can we get two finger scrolling on the mousepad?

@cerebrate
Copy link

A small point that's probably obvious, but just to make sure it's said, can we make sure the default is and the defaults are to pass through mouse events within the terminal to the underlying VT, per #545?

Customizability I like, but not as much as not breaking anything that would otherwise work. If someone's using, for example, tmux mouse support in conhost, they shouldn't have to grovel through the options to figure out why it doesn't work in Terminal.

@zadjii-msft
Copy link
Member

Though it's also being discussed in #524, I want to make sure it's tracked here that some people might not want copy-on-select. So that needs to somehow be included in these bindings. Though, that might make sense as a different setting, since I'm not sure if there are any other "onSelect" actions that would possibly make sense.

@vyevdokimov
Copy link

Add something like "lines to scroll" option - in some cases scroll is too fast in terminal using touchpad.

@TheZetner
Copy link

simple autohotkey script to paste on middle button:

; Paste with middle mouse in Windows Terminal
#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
MButton::Send ^{vk56} ; Send Ctrl-V with middle button
#IfWinActive

Just updated this to AHK 2.0:

; Paste with middle mouse in Windows Terminal
#HotIf WinActive("ahk_class CASCADIA_HOSTING_WINDOW_CLASS")
MButton::Send "^{vk56}" ; Send Ctrl-V with middle button

@psergiu
Copy link

psergiu commented May 23, 2024

It's 2024 and it's still not added :( Some companies don't allow the use of AutoHotKey on work machines. Please just add a simple "X11-style middle mouse button paste" option and it's perfect. If there's time to add more options in the future, even better.
Thank you.

@johny-mnemonic

This comment has been minimized.

@psergiu
Copy link

psergiu commented Feb 23, 2025

The lack of this simple feature has lost Microsoft more licensing revenue just at companies I worked at than it would have cost to pay a developer for a full year to implement-it. Apple is very thankful for all the Macs sold because "Windows Terminal is not compatible with our Linux servers"

@jemand771
Copy link

can y'all PLEASE stop venting your frustration in github issues?
this feature request has been liked by 133 people, most of which I'd assume are subscribed to notifications.

please consider how many patiently waiting users (who are in the same situation as you!) you're pinging vs. what you actually achieve here before posting.

@zadjii-msft
Copy link
Member

Prelle's comment was deleted because it was whinging. This:

Image

does not bring anything valuable to this conversation. I won't stand for whinging. Productive, constructive discussion - that I'm more than happy to have.

The lack of this simple feature has lost Microsoft more licensing revenue

I'd love to see concrete evidence that the lack of middle-click-to-paste in the Terminal is truly causing Microsoft to lose revenue. Genuinely: solid evidence of that would probably allow us to instantaneously prioritize this work. Rarely do we get the opportunity to directly correlate individual features in a developer tool to direct dollars and cents. Please, if you have that data, I'd love to share it.


Once again: I'd be more than willing to listen to and review a proposal here. It's a complicated problem space, to try and appropriately map all the requests to proper mouse bindings here.

My notes from: #1553 (comment) still apply.

The last attempt I made at this was in 2024, and I pushed my notes here: main...dev/migrie/s/1553-mouse-bindings. 1 I got bogged down in the potential syntax to address the multitude of linked features here.

Footnotes

  1. genuinely, I remember writing more than that, but I've had a kid since then so I forgot where I may have put any other notes I had. Push early and often folks.

@johny-mnemonic
Copy link

@zadjii-msft Unfortunately it looks like we are asking for making one small fixed window openable and your reply is that until your team figures out how to completely rebuild the whole cathedral that small window will remain fixed😢

I hope this explains the source of frustration and maybe even sparks some interest in actually solving this simple issue without the need to wait for the complete re-factor of the Terminal controls🙏

@zadjii-msft
Copy link
Member

zadjii-msft commented Feb 24, 2025

Implementing it in the control itself would be fairly trivial. Authoring a JSON schema that can be used to consisely describe the breadth of what people what to be able to configure about the mouse - that's hard. That's what I'm very open to suggestions on.

@psergiu
Copy link

psergiu commented Feb 24, 2025

For starters, just being able to paste with the middle mouse button would appease a lot of Unix/Linux people.
X11 XTerm has middle-mouse: paste, right-mouse: extend existing selection.
Thank you !

@venzy
Copy link

venzy commented Feb 24, 2025 via email

@jypeter
Copy link

jypeter commented Feb 25, 2025

For the record, and for those who don't use PuTTY, there is a screenshot of the PuTTY selection settings in my old duplicate issue

I mostly use PuTTY (for other reasons than just copy/paste), but I have learned to deal with the Windows Terminal (that also has nice features) when I need it, by simply using:

  • the option (specified in Settings => Interaction) where raw selected text is automatically copied to the clipboard (same behavior as xterm). I appreciate the fact that double clicking selects a word, and triple selects a line, the same way you do this in an xterm
  • pasting with the right button (or ctrl+v)

I can live with that, even if it feels a bit awkward. The trouble is that when we try to teach modern students that they have to remember different ways of copy/pasting, depending on the application, they don't want to bother

@Nahor
Copy link

Nahor commented Feb 25, 2025

Implementing it in the control itself would be fairly trivial. Authoring a JSON schema that can be used to consisely describe the breadth of what people what to be able to configure about the mouse - that's hard. That's what I'm very open to suggestions on.

  1. "Perfect is the enemy of good": given that this issue has existed in one form or another since Windows and Unix have co-existed, I think waiting for the perfect configuration is a mistake. I think a configuration with simple bindings with a path forward to extend it later (aka some ideas on how to do it) for other options/features/parameters should be enough. And if a completely new setting or way of doing things is needed later, while not ideal, it should also be fairly easy to have Terminal either support both methods or to automatically upgrade the configuration to the new format.
  2. There is already a schema for bindings: keyboard bindings. I don't think the mouse should be a separate thing:
    • I believe we should have click, double-click, right-click, ... "keys" and be in the same keybindings section. Alternatively (or in addition), keys could be parameterized (more about that later)
    • For the action parameters (regex, delimiters, amount of scrolled lines, ...), this isn't a mouse-only issue, the keyboard shortcuts could also make use of those
    • Having a single binding configuration would also help support/unify non standard keys/devices (scroll wheels on fancy keyboards, accessibility devices, ...)
  3. The existing schema supports nearly everything needed:
    • actions can have parameters, e.g. the splitPane command
    • actions are separated from keybindings, which allows for actions to be used by multiple bindings (i.e. shared action configuration)
    • one missing thing is parameters for the keybindings1, e.g. number of clicks but that's trivial (by some definition I guess) to fix by allowing the JSON schema to accept both "values" (like today) and "objects (map/array) for keys entries, or having a different entry (e.g. keys for simple key, keyConfig for complex/parameterized ones). This would allow things like click: {button: [left,right,middle,<number>], numberOfClicks: [double, triple, <number>], delayBetweenClicks: <milliseconds>} or scrollwheel: {speed: <number>, wheel: [vertical,horizontal,<number>]}
      @zadjii-msft mentioned verbosity as being a problem, but I don't think that can be avoided. That it be in the name of the key (DoubleLeftClick) or in the parameters (Click(left, 2)), details are verbose by definition. The only way to simplify is by omitting the details and defaulting them (e.g. Click == SingleLeftClick == Click(left, 1) == Click(button: "left", count: 1))

The TLDR:

  • there is already a perfectly good schema which should also be easily extended if necessary
  • this particular issue has existed since the beginning of Terminal, and the general form since cmd.exe, IMHO it's worth fixing it even in a non-perfect way just to silence the majority of the complaints.

Footnotes

  1. I see mention of KeyChordSegment in the schema, so maybe there is already some form of parameterization

@inko94

This comment has been minimized.

@dofuuz
Copy link
Contributor

dofuuz commented Mar 3, 2025

Here is my suggestion.

  • The mouse has few buttons, so we can simplify the configuration.
  • We don't need double middle-click or triple right-click. 99.99% of users do not use such complex click combinations, so they should not be implemented.
  • We can assume that left-clicks are dedicated to selections.
  • Supporting only Shift key as modifier can simplify the configuration.

First, implement the following settings as a priority.

"mouse": {
    "selection": {
      "double": "selectBetweenSpaces",
      "triple": "selectBetweenSymbols",
      "quad": "selectLine",
      "shift+double": "selectBetweenSpaces"
      // no "shift+triple", "shift+quad" for simplicity
    },
    "action": {
      "middle": "paste",
      "right": "openContextMenu",
      "shift+middle": "pasteSelection",
      "shift+right": { "action": "copy", "singleLine": false }
      // also, no "alt+middle", "ctrl+right"... for simplicity
    }
}

Then, extend the functionality as shown below.

"mouse": {
    "selection": {
      "double": "selectBetweenSpaces",
      "triple": { "action": "select", "delimiters": " /\\()'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502" },
      "quad": "selectLine",
      "shift+double": { "action": "select", "regex": "^.*$" }
      // no "shift+triple", "shift+quad" for simplicity
    },
    "action": {
      "middle": "paste",
      "right": "openContextMenu",
      "shift+middle": "pasteSelection",
      "shift+right": { "action": "copy", "singleLine": false }
      // also, no "alt+middle", "ctrl+right"... for simplicity
    }
}

edit: "shift+left", "alt+middle", "ctrl+right" may be added later, but only for config file, not on settings UI.

@Nahor
Copy link

Nahor commented Mar 3, 2025

  • The mouse has few buttons, so we can simplify the configuration.

Some mouse such as the Razer Naga has 17 buttons. A lot of mouse have 5 (thumb buttons), or even 7 when the left/right thumb buttons are considered different.

  • We don't need double middle-click or triple right-click. 99.99% of users do not use such complex click combinations, so they should not be implemented.
  • We can assume that left-clicks are dedicated to selections.
  • Supporting only Shift key as modifier can simplify the configuration.

IMHO this is over-simplification based on personal preference (a "quad-click" but not "ctrl-click"? wut?). This eliminates so many possibilities that one could even question the point of fixing the issue. After all, "99.95%" of users won't even bother with rebinding.

This proposal is also contrary to why the issue hasn't been closed yet. The MS team wants to support as many usages as possible, to the point of being frozen in indecision. This proposal would support none of them but the most basic ones. And it will most certainly leads to more feature requests for the missing options.

Last, this also similar to one of the biggest complaints about modern UX, where customization is limited for the sake of simplification (e.g. see Firefox, or Gnome, or Windows vs Linux).
And as they say, common thing should be simple, but advanced things should be possible. Potentially, the Terminal UI could limit the choices (within reason) to keep things simple for "normal users", but the configuration file should not and should allow tweaking by the "advanced users".

@dofuuz
Copy link
Contributor

dofuuz commented Mar 4, 2025

Some mouse such as the Razer Naga has 17 buttons.

Well then, what about touchscreen, pen tablet, touchpad, etc.? If we try to support every device, there will be no end. These devices have their own driver software, which should handle such configurations.

This proposal would support none of them but the most basic ones.

What we need is gradual implementation, not all-in-one super ultimate HID binding which will never be implemented. My suggestion can solve most issues and can be expanded later if needed.

... the configuration file should not and should allow tweaking by the "advanced users".

We can add "ctrl+click" or "alt+middle" to config file later, but not now for the reasons mentioned above.

Sorry, but all I need is a simple middle-click to paste, and it looks like I'll have to wait another 5 years without simplification.

@Nahor
Copy link

Nahor commented Mar 4, 2025

Well then, what about touchscreen, pen tablet, touchpad, etc.? If we try to support every device, there will be no end.

There is no need to go from one extreme to the other. You affirmed that mice has few buttons. I'm saying that's not quite true. Supporting more buttons is trivial compared to supporting different device types.

These devices have their own driver software, which should handle such configurations.

If this is all it takes, then feel free to use such software to remap right-click to the middle button.

We can add [...] to config file later

Your proposal only mention the easy config. How do you see your format supporting more buttons/more modifiers/more devices/...? To clarify what I mentioned before, the reason the ticket stalled is not necessarily because the MS team wants to support everything at once, or because it's complicated to implement, but because they want a schema that can be easily extended to support advanced scenarios. If your solution is to just start with a simple schema and redo it later for the advanced scenario, then the MS teams already said they were not interested in doing so.

Sorry, but all I need is a simple middle-click to paste

This ticket is not just about you.


Unrelated: @sweth, @adrian-dinu, no need to promote the post from inko94, it's a copy/paste of the top post. That and the activity of that account indicate it's very likely a fake account trying to build a history

@dofuuz
Copy link
Contributor

dofuuz commented Mar 4, 2025

@Nahor

Not every mouse driver supports per app mouse binding. And also, #1553 (comment)

#1553 (comment)
MS teams are open for suggestion about breadth of configure about the mouse.

There are many, many comments requesting similar simple middle click paste.
#1553 (comment) #1553 (comment) #1553 (comment) and so on...
Sure this ticket is not just about me. But I believe I'm allowed to share my suggestions as well, aren't I?

@zoredache
Copy link

I guess when I followed this I imagined it just being a something simple you could add to the existing 'keys' section of the 'actions'.

https://learn.microsoft.com/en-us/windows/terminal/customize-settings/actions#modifier-keys

I would assume you have a bunch of names like this Assume you have a bunch of names like below, where the names follow a pattern like mouse_BUTTONNAME_EVENTTYPE.

  • mouse_left_click
  • mouse_left_doubleclick
  • mouse_right_click
  • mouse_middle_click

Button name could like left, right, middle, button1, ..., buttonNN and any other obscure names that might be added later.

The event names could start by following similar names as used in WinForms.

So click, doubleclick, mousedown, mouseup, and so on. Possibly only starting with click and doubleclick and adding support for some kind of touchpads, other devices, and the less common events later.

https://learn.microsoft.com/en-us/dotnet/desktop/winforms/input-mouse/events?view=netdesktop-9.0#mouse-events

Anyway, you would then be able to have a configuration like this.

{
    "actions": 
    [
        {
            "command": "paste",
            "keys": "mouse_middle_click"
        },
        {
            "command": "showContextMenu",
            "keys": "mouse_right_click"
        },
        {
            "command": "openAbout",
            "keys": "shift+mouse_right_click"
        },
        {
            "command":
            {
                "action": "copy",
                "singleLine": false
            },
            "keys": "ctrl+c"
        }
    ],
    ...
}

Perhaps I am missing some complications, but I am not sure why you would want a new 'mouse' or 'pointerbindings' section in the configuration.

@Nahor
Copy link

Nahor commented Mar 4, 2025

I guess when I followed this I imagined it just being a something simple you could add to the existing 'keys' section of the 'actions'.

Same idea as mine.

* mouse_left_click
* mouse_left_doubleclick
* mouse_right_click
* mouse_middle_click

I would remove the mouse_ prefix to reduce the verbosity, or make it optional, since "left_click" pretty much always implies a mouse event [...]

Button name could like [...] button1, ..., buttonNN and any other obscure names that might be added later.

[...] but for those, keeping the mouse_ prefix would be useful since a "button" can be anything.
I would however make button1/2/3 be aliases for left/right/middle, or only supports "button4" and up. I think this is more common.

Anyway, you would then be able to have a configuration like this.
[...]

And for future extensions, the keys values, currently JSON strings, could become JSON objects so one can add more details:

{
   "command": ...
   "keys":
   {
       /* aka "shift+alt+mouse_button4_tripleclick" but without requiring hard coding a key name in the code,
          i.e. extensible to any button and any number of clicks */
       "deviceType": "mouse"      /* or "keyboard", "touchpad", ... for other scenarios */
       "button": 4                /* also accepts the usual suspects: "left", "right", "middle" */
                                  /* alternatively, `"key": 4` if we want to standardized */
                                  /* the main action button/key across the device types */
       "modifiers": "shift+alt"   /* or maybe an array to avoid the extra parsing of the value: [ "shift", "alt" ] */
       "clickCount": 3            /* also accepts the usual suspects: "double", "triple" */
   }
}

@jypeter
Copy link

jypeter commented Mar 5, 2025

Indeed, I don't need double middle-click or triple right-click. I only use multiple clicks with the left mouse button

I do wonder how things would behave for left-handed people who decide to revert the order of the buttons. I have at least one left-handed colleague who uses his mouse left of the keyboard, with the left mouse button acting as a regular right-mouse, and the right acting as left. But maybe the drivers will generate events/callbacks/whatever where whatever the user wants to be the left button is always called the left button

I have added below a screenshot (unfortunately in French) of what I get with : Win+I => Devices => Mouse. There is an option to choose if you want the main button to be the left or the right one

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Area-Settings Issues related to settings and customizability, for console or terminal Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Help Wanted We encourage anyone to jump in on these. Issue-Scenario Product-Terminal The new Windows Terminal.
Projects
Status: Spec In Progress ✏
Development

No branches or pull requests