Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add multi-touch support #14003

Merged
merged 7 commits into from
May 3, 2024
Merged

add multi-touch support #14003

merged 7 commits into from
May 3, 2024

Conversation

na-na-hi
Copy link
Contributor

Currently, mpv's input system has no real touch handling. Touch messages are either not requested, or very awkwardly translated to emulated mouse, which results in some bugs on Wayland and severely limits mpv's gesture recognition ability.

This adds multi-touch support to mpv, in the form of added touch-pos property, and implements the support for Win32 and Wayland. The property contains the positions of all current touch points, each corresponds to a finger on the touch surface. Scripts can implement custom gesture recognition algorithms to process these values and determine the appropriate action to perform.

For compatibility, --input-touch-emulate-mouse and --native-touch options are added to control whether touch events need to be emulated as mouse events.

The following example script demonstrates the recognition of 2-finger vertical scroll gesture to control the video brightness:

local scroll = false
local y = 0
local brightness = 0

function ypos(points)
    return (points[1].y + points[2].y) / 2
end

function handle_scroll(name, points)
    if #points ~= 2 then
        scroll = false
    elseif scroll == false then
        scroll = true
        y = ypos(points)
        brightness = mp.get_property_native("brightness")
    else
        mp.set_property_native("brightness", brightness + (y - ypos(points)) / 5)
    end
end

mp.observe_property("touch-pos", "native", handle_scroll)

The goal of multitouch support is to interpret as little as possible:
mpv just stores the touch point information and makes the position
available for query. Does not participate in the deprecated input section
system.

The API is modeled after the common part of the touch input APIs of
Win32, X11, and Wayland, to make sure the platform-specific implementations
are as simple as possible.
@sfan5 sfan5 self-requested a review April 27, 2024 10:02
Copy link

github-actions bot commented Apr 27, 2024

Download the artifacts for this pull request:

Windows
macOS

dyphire referenced this pull request in dyphire/mpv-config Apr 29, 2024
得益于 verygoodlee/mpv-menu-plugin@e59bf0f 提出的方法,现在可以切换到 mpv 的内部实现了
归档 C 插件
@na-na-hi na-na-hi force-pushed the multitouch branch 2 times, most recently from 1d04647 to 932c7ae Compare April 29, 2024 13:43
DOCS/man/input.rst Outdated Show resolved Hide resolved
DOCS/man/input.rst Show resolved Hide resolved
DOCS/man/options.rst Show resolved Hide resolved
{
// queue dummy cmd so that touch-pos can notify observers
mp_cmd_t *cmd = mp_input_parse_cmd(ictx, bstr0("ignore"), "<internal>");
queue_cmd(ictx, cmd);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to be necessary because mouse-pos is married to MP_EVENT_INPUT_PROCESSED which is triggered in mp_process_input. could be improved some day..

player/command.c Outdated Show resolved Hide resolved
player/command.c Outdated Show resolved Hide resolved
video/out/w32_common.c Outdated Show resolved Hide resolved
video/out/w32_common.c Outdated Show resolved Hide resolved
video/out/w32_common.c Outdated Show resolved Hide resolved
video/out/w32_common.c Show resolved Hide resolved
@na-na-hi na-na-hi force-pushed the multitouch branch 3 times, most recently from 7164ea8 to baaf07a Compare April 29, 2024 19:14
Copy link
Member

@sfan5 sfan5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a device to test this so a code review is the best I can do

@kasper93 kasper93 added this to the Release v0.39.0 milestone Apr 29, 2024
This adds touch-pos property, which contains the information of all
current touch points. The property has sub properties:

touch-pos/count: the number of current touch points
touch-pos/N/x,y: the position of touch point N
touch-pos/N/id: unique identifier of the touch point
This adds --input-touch-emulate-mouse option, which controls whether to
enable legacy touch handling where touch inputs are emulated as mouse
inputs. This establishes a primary touch point (the one with the lowest
index) as the emulated mouse position, and MBTN_LEFT up/down with the
appearance of the first touch point and the disappearance of the last
touch point.

This fixes some problems with touch handling on Wayland, for example
attempting to pinch results in a double click.
Release all touch points. Used by some VOs like Wayland touch cancel
event.
@na-na-hi na-na-hi force-pushed the multitouch branch 2 times, most recently from 786d1d5 to a5e55a1 Compare May 1, 2024 00:40
Use the multitouch API. No need to emulate mouse input as it's already
done by the input system.
For platforms which send emulated mouse inputs for touch-unaware clients,
such as Win32 and X11, this option enables the native multitouch handling
and disables emulated mouse inputs. This enables interested scripts to
handle multitouch events while keeping compatibility with touch-unaware
scripts.
Use the multitouch API. To disable system's defualt mouse emulation,
set --native-touch=yes.
Copy link
Contributor

@kasper93 kasper93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. There will likely be quirks with touch support, but we can work them out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants