Support touchscreen input

[flipflop97]

My laptop has a touch screen but trying to control my phone with it doesn't seem to do anything at all. It would be nice if touch input was also supported.

[rom1v]

Hmm, I thought a touch screen just generated mouse events.

Does it work with a mouse?

[parkerlreed]

Works with a mouse. Touchscreen is just flat out ignored. Acts as a mouse elsewhere which is weird. It still shows the cursor going over the scrcpy window but tapping/dragging does nothing.

[MCMic]

I guess you need to catch https://wiki.libsdl.org/SDL_TouchFingerEvent
https://wiki.libsdl.org/SDL_MouseMotionEvent#Remarks does say that mouse motion can have a touch input as source so it seems consistent with the fact that moving works and not clicking.

[renedecandido]

I had the same issue on Linux and an external touch monitor. As a workaround you can use Touchégg with some configuration and additional gesture
<gesture type="TAP" fingers="1" direction=""> <action type="MOUSE_CLICK">BUTTON=1</action> </gesture>

[liaoqingmo]

just mouse event

[parkerlreed]

Any update on this?

[rom1v]

Hi,

I started to implement it, but I have no touch screen, so I cannot test.

If someone want to test (and help to fix if it does not work), I pushed a branch: finger.

[parkerlreed]

It seems to be way too sensitive towards long presses. And I don't think it's matching the location where I'm touching.

[rom1v]

@parkerlreed Thank you for your feedbacks. I'm not very surprised 😄 I really need to get a laptop with a touchscreen.

[rom1v]

I got a computer with a touch screen! (thanks @bonrry!).

Here is my current wip: finger.5.

It works in image viewers or OSMAnd, but not in webbrowsers (I can't zoom with multitouch in Firefox, while it works on the real device). I don't know why yet.

[rom1v]

Seems to work now: finger.5.

Please give me your feedbacks. I will rewrite the commits properly when I have time.

[rom1v]

Here are some binaries, if you can't build it:

• a release for Windows: scrcpy-win64-v1.10-31-g783eb95.zip (SHA256: a286f7b4532e0b0a29fd758ba1f32826079eb621b7a2194cbfa3b02e3aa3b517)
• the prebuilt server: scrcpy-server-v1.10-31-g783eb95.jar (SHA256: 942878b2eaa93930441865b0520ef1217aba7f8dbfb1a152e4713ec7a093f730)

[parkerlreed]

Figured I'd post here too for good keeping

[rom1v]

Merged in dev branch.

[unclehowell]

Quick question, if I update my files to reflect the changes you've made on the development branch can I expect touchscreen to work, or will I have to rebuild from source/ wait for the snap release? ??

[rom1v]

Quick question, if I update my files to reflect the changes

Which files?

You can reflect the changes on the source files, then build from the (modified) sources, but then why not just take the dev branch and build?

[unclehowell]

I replicated all the changes to all the files you made on Oct 02 & 3rd. Rebooted and reloaded everything. Touchscreen inputs still not recognised by scrcpy. I tried manual build from source before but didn't work. Ended up using snap

[rom1v]

What did you do exactly? What did you "reloaded"? If you edit the source files, you need to build from these new sources…

[unclehowell]

I noticed the last update to the Snap package was the 5th October so I tried re-installing with Snap using the edge channel. No touchscreen. I removed snap and I am now revisiting building from source (a git clone of the ´dev´ branch), but building from source has issues, which I´m looking into. To be more specific I get these errors after running the command: meson x --buildtype release --strip -Db_lto=true

WARNING: Project targetting '>= 0.37' but tried to use feature introduced in '0.48.0': console arg in custom_target DEPRECATION: build_always is deprecated. Combine build_by_default and build_always_stale instead Build targets in project: 8
WARNING: Project specifies a minimum meson_version '>= 0.37' but uses features which were added in newer versions:
* 0.48.0: {'console arg in custom_target'} Found ninja-1.8.2 at /usr/bin/ninja

If I then switch to a non-root user and run the ninja command, like the guide suggests, I get the following error message: \ninja: error: opening build log: Permission denied. So I try sudo ninja and get the following error: [36/39] Generating scrcpy-server with a custom command. (not invoking gradle, since we are root) [39/39] Linking target app/scrcpy.

If I proceed to do sudo ninja install I get this error [0/2] Generating scrcpy-server with a custom command. (not invoking gradle, since we are root) [1/2] Installing files. Installing app/scrcpy to /usr/local/bin Stripping target 'app/scrcpy' using strip. Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/mesonbuild/mesonmain.py", line 129, in run return options.run_func(options) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 521, in run installer.do_install(datafilename) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 343, in do_install self.install_targets(d) File "/usr/local/lib/python3.6/dist-packages/mesonbuild/minstall.py", line 436, in install_targets raise RuntimeError('File {!r} could not be found'.format(t.fname)) RuntimeError: File 'server/scrcpy-server.jar' could not be found FAILED: meson-install /usr/local/bin/meson install --no-rebuild ninja: build stopped: subcommand failed.

I also ran release.sh while playing around and this is what I got:

The Meson build system
Version: 0.52.0
Source dir: /opt/scrcpy
Build dir: /opt/scrcpy/build_test
Build type: native build
Project name: scrcpy
Project version: 1.10
C compiler for the host machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
C linker for the host machine: GNU ld.bfd 2.30
Host machine cpu family: x86
Host machine cpu: i686
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency libavformat found: YES 57.83.100
Run-time dependency libavcodec found: YES 57.107.100
Run-time dependency libavutil found: YES 55.78.100
Run-time dependency sdl2 found: YES 2.0.8
Configuring config.h using configuration
Program ./scripts/build-wrapper.sh found: YES (/opt/scrcpy/server/./scripts/build-wrapper.sh)
WARNING: Project targetting '>= 0.37' but tried to use feature introduced in '0.48.0': console arg in custom_target
DEPRECATION: build_always is deprecated. Combine build_by_default and build_always_stale instead.
Build targets in project: 8
WARNING: Project specifies a minimum meson_version '>= 0.37' but uses features which were added in newer versions:

• 0.48.0: {'console arg in custom_target'}
  Found ninja-1.8.2 at /usr/bin/ninja
  ninja: Entering directory `build_test'
  [36/40] Generating scrcpy-server with a custom command.
  (not invoking gradle, since we are root)
  [39/40] Running all tests.
  1/5 test_cbuf OK 0.02 s
  2/5 test_control_event_serialize OK 0.07 s
  3/5 test_device_event_deserialize OK 0.07 s
  4/5 test_queue OK 0.07 s
  5/5 test_strutil OK 0.07 s

Ok: 5
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0

Full log written to /opt/scrcpy/build_test/meson-logs/testlog.txt

Configure project :server
Checking the license for package Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/licenses
Warning: License for package Android SDK Build-Tools 28.0.3 not accepted.

FAILURE: Build failed with an exception.

• What went wrong:
  A problem occurred configuring project ':server'.

Failed to install the following Android SDK packages as some licences have not been accepted.
build-tools;28.0.3 Android SDK Build-Tools 28.0.3
To build this project, accept the SDK license agreements and install the missing components using the Android Studio SDK Manager.
Alternatively, to transfer the license agreements from one workstation to another, see http://d.android.com/r/studio-ui/export-licenses.html

Using Android SDK: /usr/lib/android-sdk

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 4s

[unclehowell]

I ran the command sdkmanager --licenses and in Android-Studio I changed the SDK from Android 10 (Q) to Android 9 (Pie), which eliminated the above error but produced the following:

> Configure project :server
Checking the license for package Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/licenses
License for package Android SDK Build-Tools 28.0.3 accepted.
Preparing "Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)".
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" ready.
Installing Android SDK Build-Tools 28.0.3 in /usr/lib/android-sdk/build-tools/28.0.3
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" complete.
"Install Android SDK Build-Tools 28.0.3 (revision: 28.0.3)" finished.
Checking the license for package Android SDK Platform 29 in /usr/lib/android-sdk/licenses
License for package Android SDK Platform 29 accepted.
Preparing "Install Android SDK Platform 29 (revision: 3)".
"Install Android SDK Platform 29 (revision: 3)" ready.
Installing Android SDK Platform 29 in /usr/lib/android-sdk/platforms/android-29
"Install Android SDK Platform 29 (revision: 3)" complete.
"Install Android SDK Platform 29 (revision: 3)" finished.

> Task :server:compileDebugAidl FAILED
/usr/lib/android-sdk/build-tools/28.0.3/aidl: 1: /usr/lib/android-sdk/build-tools/28.0.3/aidl: Syntax error: redirection unexpected


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:compileDebugAidl'.
> com.android.ide.common.workers.WorkerExecutorException: 1 exception was raised by workers:
  java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: com.android.ide.common.process.ProcessException: Error while executing process /usr/lib/android-sdk/build-tools/28.0.3/aidl with arguments {-p/usr/lib/android-sdk/platforms/android-29/framework.aidl -o/opt/scrcpy/server/build/generated/aidl_source_output_dir/debug/compileDebugAidl/out -I/opt/scrcpy/server/src/main/aidl -I/opt/scrcpy/server/src/debug/aidl -d/tmp/aidl5784619688143858877.d /opt/scrcpy/server/src/main/aidl/android/view/IRotationWatcher.aidl}

Ended up going with VNC and Emteria until the snap version of ScrCpy (with the touchscreen fixes) is released. Keep up the great work

[sisco311]

@unclehowell, the snap from the edge channnel now is built from the dev branch.

[unclehowell]

You rock pal. Keep up the great work.

[gandhiomkar]

hey @rom1v ! I was trying to get this to work on my ubuntu touchscreen laptop as my phone display is dead. but i couldnt build this specific branch finger.5. I installed latest branch with install_release.sh and it worked fine. but there is no such script in this branch so i copied it over and ran it but i get error " with open('META-INF/MANIFEST.MF', 'r+', encoding='utf-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'META-INF/MANIFEST.MF". can you help me? how can i build this?

[rom1v]

but i couldnt build this specific branch finger.5

That's very old. The feature has been merged in the master branch for a long time.

I installed latest branch with install_release.sh and it worked fine.

OK, great! What's the problem then?

but there is no such script

Which script? What are you trying to do?

[gandhiomkar]

@rom1v Thanks for replying!

That's very old. The feature has been merged in the master branch for a long time.

I installed latest branch with install_release.sh and it worked fine.

OK, great! What's the problem then?

it works fine except the touch feature. so i went searching i found this issue and thought that i should install finger branch. i didnt realize if was merged. sorry I'm a newbie. I have ubuntu 22.04 lts on my laptop and the touch works with even two finger zoom gestures. could this be a problem of ubuntu? do you have any suggestions?

but there is no such script

Which script? What are you trying to do?

i figured out how to run finger.5 branch but i still get following error when i run it with ./run x [server] ERROR: Exception on thread Thread[main,5,main] java.lang.IllegalArgumentException: Expecting 6 parameters at com.genymobile.scrcpy.Server.createOptions(Server.java:71) at com.genymobile.scrcpy.Server.main(Server.java:132) at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method) at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:339) ERROR: Server connection failed but i don't think it really matters now that i know that it has been merged into master.

[gandhiomkar]

problem description: the touch works for laptop for clicking , scrolling or zooming but it is not recognized at all in scrcpy window
when i start a session.

[rom1v]

What is the reslut of scrcpy --version?

[gandhiomkar]

scrcpy 2.5 https://github.com/Genymobile/scrcpy

Dependencies (compiled / linked):

• SDL: 2.0.20 / 2.0.20
• libavcodec: 58.134.100 / 58.134.100
• libavformat: 58.76.100 / 58.76.100
• libavutil: 56.70.100 / 56.70.100
• libavdevice: 58.13.100 / 58.13.100
• libusb: - / 1.0.25

[rom1v]

OK, it's not the latest version (2.6.1), but it should still work.

I just tested on my compter with a touch screen, it works.

Run scrcpy -Vverbose, do some clicks with your touchscreen. What is the full output in the console?

[gandhiomkar]

android device version android 10 and ubuntu version 22.04 lts

[gandhiomkar]

[server] INFO: Device: [samsung] samsung SM-A750F (Android 10)
DEBUG: Server connected
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
DEBUG: Using icon: /usr/local/share/icons/hicolor/256x256/apps/scrcpy.png
[server] WARN: Audio disabled: it is not supported before Android 11
[server] DEBUG: Audio encoder stopped
INFO: Renderer: opengl
INFO: OpenGL version: 4.6 (Compatibility Profile) Mesa 23.2.1-1ubuntu3.1~22.04.2
INFO: Trilinear filtering enabled
DEBUG: Demuxer 'video': starting thread
DEBUG: Demuxer 'audio': starting thread
WARN: Demuxer 'audio': stream explicitly disabled by the device
[server] DEBUG: Using video encoder: 'OMX.Exynos.AVC.Encoder'
INFO: Texture: 1080x2216
…
VERBOSE: input: touch [id=4] move position=690,1329 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=692,1321 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=700,1304 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=702,1284 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=707,1266 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=710,1256 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=712,1246 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=715,1226 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=717,1206 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=720,1184 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=720,1164 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=722,1144 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=722,1124 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=722,1104 pressure=1.000000 action_button=000000 buttons=000000
VERBOSE: input: touch [id=4] move position=725,1079 pressure=1.000000 action_button=000000 buttons=000000

when i touch or swipe it does print up or down events in console. so it is picking it up but they are not reflected on the phone.

[rom1v]

I guess you posted just an excerpt, but in before the move events, you should get a down event, and after the move events, you should get an up event, correct?

[rom1v]

Can you test with another device on the same computer?

[gandhiomkar]

yeah, i just tested on another phone redmi 10A , android 11 and its the same thing. Also because previously some people have mentioned getting a cursor on touch. On my ubuntu laptop when i use touch the cursor disappears and the touch is used like mobile phone. and when i use the mouse again the cursor reappears. if that means anything.

[gandhiomkar]

do you need any additional information about my computer? i can provide that

[rom1v]

If that's the same problem on several devices, the problem is on the client. So maybe the SDL version has a problem with that.

Just to be sure, can you install and test the very last version (2.6.1), there were some changes related to mouse events (even if touch is expected to work also on 2.5).

[gandhiomkar]

I'm onto that. I'll let you know

[gandhiomkar]

@rom1v Thanks a lot! ('-')7. I just installed latest release and it works, on both devices. I'm sorry I troubled you without first testing on the latest version. There's another issue i want to mention on Samsung when i first connect the cable then the allow file access pop up comes. if I tap allow it instantly kills the session. here are the logs
utton=000000 buttons=000000 VERBOSE: input: touch [id=160] up position=822,2073 pressure=1.000000 action_button=000000 buttons=000000 DEBUG: Demuxer 'video': end of frames DEBUG: Receiver stopped WARN: Device disconnected DEBUG: quit... DEBUG: Controller stopped DEBUG: Server disconnected DEBUG: Server terminated

[rom1v]

There's another issue i want to mention on Samsung when i first connect the cable then the allow file access pop up comes. if I tap allow it instantly kills the session.

That's "expected": when you change between USB modes, the device resets the USB layer, and the adb connection is closed.

This is not specific to scrcpy: if you run separately adb shell or adb logcat, they will also be closed at the exact same moment.

[gandhiomkar]

Oh okay. and thanks again!