Adding an option --serve tcp:localhost:port #1419

Darkroll76 · 2020-05-22T14:12:51Z

Hi everyone,

Following this posts: #1073 and #1159 and suggested by @rom1v , I worked on an option --serve. This option would allow forwarding the video stream to any client connected. It could be a C# Unity App, Java App or C# UWP App or whatever.

The command is like below :

--serve tcp:[localhost|IP]:[port]
# For example
--serve tcp:localhost:27115

It could be a great solution for people like me, who wants to display the video stream inside a desktop application without redeveloping and rebuilding the scrcpy client in the desired language.

- Add current working files

rom1v · 2020-05-22T17:59:43Z

Thank you for the PR.

However, it seems there's a rebase issue, your commit 17947e1 contains a lot of code which already exists on master (and which is not related to the feature you implement).

overvox · 2020-06-04T07:35:27Z

Hey @Darkroll76,
first of all big thanks for your pull request and added features. (My usecase is similar to yours, as I want to feed the stream in realtime to Unity) I checked out your latest commit and managed to build it for windows. In general the serving feature works, however I noticed a very noticable lag as well as a lot of artifacts when testing the tcp stream in the vlc player (all on the same host machine).

I tested multiple downgrades in terms of bitrate and resolution without any improvements.
Is this an issue with vlc or just the nature of the tcp stream?

Darkroll76 · 2020-06-04T07:53:45Z

Hi @overvox ,

Thanks! Apparently it's due to VLC buffering. See this post

Since I'm working on a big project, I didn't find the time to remove all the code which already exists to have a clean commit.

Darkroll76 · 2020-06-05T07:02:07Z

@overvox I understood that I was doing the wrong thing. When you use the --serve command, it waits for the client to connect to start the stream. But during this time, scrcpy is already receiving packets from the scrcpy server and apparently, it keeps the frames until it can send them. I used the same behaviour as the decoder and the recorder, but I should rather look for the same thing as the display window.

This is why there is a time lag between the action we perform on our device and the frame received and displayed. Because it sends the frames it kept since the start of scrcpy.

hgedik · 2020-06-06T20:20:01Z

hi @Darkroll76 I have a question. Will be able to use it in webrtc after --serve update?
thanks

Darkroll76 · 2020-06-08T09:31:38Z

Hi @hgedik, I don't know I never used it. For now, I'm testing as localhost but if you can connect to an IP Address on a port I should work. Currently --serve allows forwarding the stream to another socket using TCP.

@rom1v, I'm trying to remove the time lag between the action we perform on our device and the frame displayed. Is ScrCpy keep each frame since the server is started? How could I clean the pending frames?

rom1v · 2020-06-08T12:15:06Z

I'm trying to remove the time lag between the action we perform on our device and the frame displayed. Is ScrCpy keep each frame since the server is started? How could I clean the pending frames?

IMO, the server should not be started until a client connects to the socket. Or it could start some initialization (especially to detect device disconnection) but wait until a client is connected to the socket before capturing the screen.

Maciejszuchta · 2020-07-03T07:19:55Z

Any updates on this feature. Rly looking forward to it.

Darkroll76 · 2020-07-03T09:45:04Z

Hi @Maciejszuchta, I'm trying to fix the rebase issue on commit 17947e1 like @rom1v said :

it contains a lot of code which already exists on master (and which is not related to the feature you implement).

But I'm a bit lost in how to do it properly without breaking anything

andreluisos · 2020-07-18T09:59:33Z

This feature would be great.

pranshuthegamer · 2020-10-10T16:18:25Z

where is this serve repo?
i wanna use it

DreamInjection · 2021-07-18T22:10:21Z

Hi, Any update on this serve repo? I also would like to use the --serve command but I somehow can't even find the branch on SourceTree. Am I just noob?

rom1v · 2022-01-16T22:23:23Z

Hi,

A quick follow up.

Despite I initially suggested this feature, I'm finally not convinced a --serve argument is a good idea now.

Firstly, it adds complexity:

it impacts scrcpy starting
it is limited to video, and could possibly constrain what can be done about audio in the future (if there is a real solution, which is not the case for now)

But above all, the same feature (receive the video stream for another app) can now be achieved differently (it was not the case when this PR was created):

on Linux, via V4L2
using scrcpy-server directly

For using scrcpy-server, recent changes (in particular 04e5537 and cabcbc2) facilitate reusing the server for receiving the raw video.

Here is a sample script which starts the server:

adb forward tcp:1234 localabstract:scrcpy
adb push scrcpy-server /data/local/tmp/scrcpy-server.jar
adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar \
    app_process / com.genymobile.scrcpy.Server 1.21 \
    send_frame_meta=false tunnel_forward=true control=false max_size=1920

From another terminal:

# skip the 69 first bytes (1 zero-byte, 64-bytes device name string, and 4 bytes for the device size)
nc localhost 1234 | tail -c+70 | vlc --demux h264 -

(we could even add a server option to avoid the 69 first bytes, and run directly vlc tcp://localhost:1234)

This will not be real time at all because VLC will add buffering, but this is to illustrate that it is possible to get the raw H.264 stream.
Instead of running nc and vlc, you probably use your own client.

Moreover, that way, the client can decide if it wants to receive the "frame meta" in front of each Annex B H.264 packet or not, to avoid an additional 1-frame latency (see #646).

For convenience, this new option forces the 3 following options: - send_device_meta=false - send_frame_meta=false - send_dummy_byte=false This allows to send a raw H.264 stream on the video socket. Concretely: adb push scrcpy-server /data/local/tmp/scrcpy-server.jar adb forward tcp:1234 localabstract:scrcpy adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar \ app_process / com.genymobile.scrcpy.Server 1.21 \ raw_video_stream=true tunnel_forward=true control=false As soon as a client connects via TCP to localhost:1234, it will receive the raw H.264 stream. Refs #1419 comment <#1419 (comment)> PR #2971 <#2971>

rom1v · 2022-01-26T10:37:20Z

See #2971.

For convenience, this new option forces the 3 following options: - send_device_meta=false - send_frame_meta=false - send_dummy_byte=false This allows to send a raw H.264 stream on the video socket. Concretely: adb push scrcpy-server /data/local/tmp/scrcpy-server.jar adb forward tcp:1234 localabstract:scrcpy adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar \ app_process / com.genymobile.scrcpy.Server 1.21 \ raw_video_stream=true tunnel_forward=true control=false As soon as a client connects via TCP to localhost:1234, it will receive the raw H.264 stream. Refs #1419 comment <#1419 (comment)> PR #2971 <#2971>

Maciejszuchta · 2022-02-16T08:03:35Z

Hello, is there a possibility to dynamically take a screenshot (encode it to png or jpg) of a live stream from the device? What I need is to take a screenshot of a device screen as fast as it is possible.

rom1v · 2022-02-16T08:12:25Z

Hello, is there a possibility to dynamically take a screenshot (encode it to png or jpg)

Currently, no: #684 #2040.

But you could record the video stream and extract frames later:

scrcpy --record file.mp4

DreamInjection · 2022-03-09T19:43:16Z

For using scrcpy-server, recent changes (in particular 04e5537 and cabcbc2) facilitate reusing the server for receiving the raw video.

Here is a sample script which starts the server:
adb forward tcp:1234 localabstract:scrcpy
adb push scrcpy-server /data/local/tmp/scrcpy-server.jar
adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar \
    app_process / com.genymobile.scrcpy.Server 1.21 \
    send_frame_meta=false tunnel_forward=true control=false max_size=1920

I am unable to run these commands. It seems that my phone needs to be rooted in order for me to be able to push the scrcpy server to the data folder. I found a workaround by drag-and-dropping my file to a folder I had access to and changing the path in the third command. However, that command fails as well, saying that app_process cannot be found, even though it is in the folder it should be in, and included in the PATH as well. I'm wondering if it has to do with the phone not being rooted again? Or does the scrcpy-server absolutely have to be in the data/local/temp folder?

rom1v · 2022-03-09T20:12:42Z

I am unable to run these commands.

What is the exact output for each one?

DreamInjection · 2022-03-09T21:11:50Z

For the adb push command, I was getting something like push: permission denied. I tried again just now and can't reproduce the error, the push command now somehow works even though I still don't have root access. For the adb shell command, I realized I hadn't deleted the \ from the command, which gave me /system/bin/sh: app_process: inaccessible or not found. But now that this 3rd command runs, it simply outputs "Aborted".

I'm sorry for bothering you with this, this is something I'm fairly new at and I don't quite understand everything.

varundtsfi · 2022-06-18T09:03:10Z

For the adb push command, I was getting something like push: permission denied. I tried again just now and can't reproduce the error, the push command now somehow works even though I still don't have root access. For the adb shell command, I realized I hadn't deleted the \ from the command, which gave me /system/bin/sh: app_process: inaccessible or not found. But now that this 3rd command runs, it simply outputs "Aborted".

I'm sorry for bothering you with this, this is something I'm fairly new at and I don't quite understand everything.

Hi @Darkroll76
I guess you are using Xiaomi or Realme device please cofirm.

Killian Richard added 5 commits February 24, 2020 09:56

Adding --serve option on README

81a6b8e

Adding --serve option :

b4dc78a

- Add current working files

Update --serve option with scrcpy v1.13 + Cleaning code.

17947e1

Fix missing char and error

4078f7b

Allowing other ip address by converting it to hex

b666c20

Killian Richard added 2 commits June 16, 2020 16:33

Change to run server only when serve is ready to forward the stream

556e3bb

Add meson configuration folder in git ignore

f1fafa5

Killian Richard and others added 2 commits July 3, 2020 11:29

Cleaning code for pull request

1e607c6

Delete .gitignore

97f5ae9

rom1v mentioned this pull request Jul 17, 2020

Another Capture Method in Windows #1594

Open

r3pwn mentioned this pull request Aug 10, 2020

Rebase of --serve commits onto v1.16 #1659

Closed

rom1v mentioned this pull request Jan 26, 2022

Add server options to send raw H.264 without using the client #2971

Merged

rom1v closed this Jan 26, 2022

unique-EJ mentioned this pull request Sep 16, 2022

A library in scrcpy for clients is wanted #3498

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding an option --serve tcp:localhost:port #1419

Adding an option --serve tcp:localhost:port #1419

Darkroll76 commented May 22, 2020

rom1v commented May 22, 2020

overvox commented Jun 4, 2020

Darkroll76 commented Jun 4, 2020

Darkroll76 commented Jun 5, 2020 •

edited

Loading

hgedik commented Jun 6, 2020

Darkroll76 commented Jun 8, 2020 •

edited

Loading

rom1v commented Jun 8, 2020 •

edited

Loading

Maciejszuchta commented Jul 3, 2020

Darkroll76 commented Jul 3, 2020

andreluisos commented Jul 18, 2020

pranshuthegamer commented Oct 10, 2020

DreamInjection commented Jul 18, 2021

rom1v commented Jan 16, 2022 •

edited

Loading

rom1v commented Jan 26, 2022

Maciejszuchta commented Feb 16, 2022

rom1v commented Feb 16, 2022

DreamInjection commented Mar 9, 2022

rom1v commented Mar 9, 2022

DreamInjection commented Mar 9, 2022

varundtsfi commented Jun 18, 2022

Adding an option --serve tcp:localhost:port #1419

Adding an option --serve tcp:localhost:port #1419

Conversation

Darkroll76 commented May 22, 2020

rom1v commented May 22, 2020

overvox commented Jun 4, 2020

Darkroll76 commented Jun 4, 2020

Darkroll76 commented Jun 5, 2020 • edited Loading

hgedik commented Jun 6, 2020

Darkroll76 commented Jun 8, 2020 • edited Loading

rom1v commented Jun 8, 2020 • edited Loading

Maciejszuchta commented Jul 3, 2020

Darkroll76 commented Jul 3, 2020

andreluisos commented Jul 18, 2020

pranshuthegamer commented Oct 10, 2020

DreamInjection commented Jul 18, 2021

rom1v commented Jan 16, 2022 • edited Loading

rom1v commented Jan 26, 2022

Maciejszuchta commented Feb 16, 2022

rom1v commented Feb 16, 2022

DreamInjection commented Mar 9, 2022

rom1v commented Mar 9, 2022

DreamInjection commented Mar 9, 2022

varundtsfi commented Jun 18, 2022

Darkroll76 commented Jun 5, 2020 •

edited

Loading

Darkroll76 commented Jun 8, 2020 •

edited

Loading

rom1v commented Jun 8, 2020 •

edited

Loading

rom1v commented Jan 16, 2022 •

edited

Loading