Please Add Remote Control.

[the-mentor]

This app is great the only thing its missing IMO is the ability to remote control the phone that is being streamed.
so i would like to suggest that improvement.

thank you!
-DM

[dkrivoruchko]

What kind of remote control you suggest?

[the-mentor]

I recommend a way to click and drag with the mouse on the website and relay these clicks to the device.
Also keyboard support will also be very useful.

[dkrivoruchko]

You not the first one with such ideas. The is one issue - for security reasons Android system deny any way to interact one app with other. There is only limited options completely controlled by system. So without rooting device there is no way to do this or I don't know such way. Let me know if you have any.

[the-mentor]

I think apps like Vysor use the adb over network or adb over USB to control a device without root.
And I would very much prefer an open source solution like screen stream

[dkrivoruchko]

Yes, but this is differently no a simple solution as Start-and-Go.
I'll not do this, but any one can do this.

[ItsShadowl]

It can be implemented using accessibility services, the user has to enabled it for the app before the controls can work

[soutieslak]

@noobshow I am currently looking at trying to implement remote control functionality into this. Have you perhaps already attempted it?

[ItsShadowl]

Hi @soutieslak,
I eventually stopped the project, the others on the team stopped responding around the COVID outbreak. It should be easy I think.

[nian0114]

@soutieslak @ItsShadowl try this one which is based this project if u have ROOT access. it need ROOT so i will not create pr to this proj.And big thanks to dkrivoruchko. fork proj url is https://github.com/nian0114/ScreenStreamRemoteControl

[KopiasCsaba]

Maybe these guys have it figured out? Indeed this would be super cool to have.

https://github.com/h-mdm

[aurium]

@KopiasCsaba H-MDM looks interesting for automated tests, but I don't really understand it. Can you point how it can simulate screen touch on the device? Thank you!

[KopiasCsaba]

An update on this, here is a proof of concept that it is possible!

1. Install this awesome tool: https://play.google.com/store/search?q=automate&c=apps

2. import this flow:
   Gesture!.zip
   (You need to unzip it first).
   This will create a flow that looks like this:
   

And you can see it in this video in action:

• I start the flow
• I switch to the browser
• Automate injects the swipe and the browser interprets that as a reload gesture

rec.mp4

So upon doing those things, you'll notice it asks permission to be an accessibility service.

So I'm pretty sure that is the key to this, through accessibility service you can inject gestures somehow.

[KopiasCsaba]

@KopiasCsaba H-MDM looks interesting for automated tests, but I don't really understand it. Can you point how it can simulate screen touch on the device? Thank you!

Great news!

I had hard time finding it in that repo, but based on my prev post i started to dig around, and found it!

So KDEConnect is an opensource tooling from where you can learn how to do it!

I believe the key code parts are here:

• https://github.com/KDE/kdeconnect-android/blob/49a9cd5ea7d50a0d5fe267ba7b8e568d094adbb2/src/org/kde/kdeconnect/Plugins/MouseReceiverPlugin/MouseReceiverService.java#L147
• https://github.com/KDE/kdeconnect-android/blob/49a9cd5ea7d50a0d5fe267ba7b8e568d094adbb2/AndroidManifest.xml#L384
• https://github.com/KDE/kdeconnect-android/blob/49a9cd5ea7d50a0d5fe267ba7b8e568d094adbb2/AndroidManifest.xml#L63C7-L63C20 (I've read this might matter as well)

Here is a confusing video on kdeconnect in action, where I draw with my mouse through wifi on a split screen, so you can see both my mouse and what events happening on the paint canvas.

document_5778563323477890199.mp4

It would be really awesome to be able to interact with the phone through the browser. I hope I could have shown some useful resources for it.

[dkrivoruchko]

Thanks for research. Will have a look. But no promises.

[Sigmun]

Indeed, TeslaDisplay made it real. But I prefer your app with resizing feature. Another feature they have is using local domaine name rather than IP.