On the basis of Chapter 11, browser access to 3D games running on the remote desktop is slow

[yanbx]

The ubuntu - 18 11-4-1-04 - gnome - spice based on this chapter, I set up a remote desktop environment and support 3 d games, I run a 3 d game, find visit beginning in the browser, access speed is quite smooth, as this service placed for a long time, I found that the CPU is very high Takes up about 70%, the browser to access remote desktop appear caton is very serious, I want to ask, I want to optimize the problem, from which to start?I analyze whether the problem in Spice-Web-Client consumes too much CPU resources, because it is implemented in Python. I just analyze and guess, do you have time to help analyze it?

[fadams]

I'm not totally sure what you are asking but from this:
"
find visit beginning in the browser, access speed is quite smooth, as this service placed for a long time, I found that the CPU is very high Takes up about 70%, the browser to access remote desktop appear caton is very serious
"

It sounds like you are saying that when you first connect via the Spice-Web-Client the performance is OK, but over time the performance appears to degrade, is that correct?

When you say "I found that the CPU is very high Takes up about 70%" are you talking about the CPU utilisation of the browser or are you talking about the remote desktop container CPU utilisation? I think you are meaning browser CPU utilisation from this but it's not totally clear.

On "I analyze whether the problem in Spice-Web-Client consumes too much CPU resources, because it is implemented in Python" the Spice-Web-Client isn't implemented in Python, it is implemented in JavaScript.

In all honestly it's pretty hard for me to give a huge amount of help on this because all that I've really done is provide a way to install and run this stuff in a Docker container and although I've applied a couple of small patches to provide a rudimentary login to the Spice-Web-Client I haven't changed it in any really meaningful way - to be clear it is not my code, I wish it were 'cause it's pretty impressive, but you might well need to speak to the developers - this is the repo https://github.com/flexVDI/spice-web-client the Dockerfile basically just pulls the release from https://github.com/flexVDI/spice-web-client/archive/3.1.0.tar.gz

If it were me I'd try to compare the results using the Spice-Web-Client with doing the same thing using the native Remote Viewer SPICE client. That is to say if you don't get issues running for an extended period of time using Remote Viewer then you can be more sure it's definitely the HTML5 client.

When you say "Takes up about 70%" does the performance actually degrade or is it just the CPU utilisation, I mean the Web client is doing quite a lot of work so it's not totally surprising it's using a fair amount of CPU, it's essentially video decoding in JavaScript.

A few things you could try.

1. is your app windowed or maximised - try running full screen the SPICE encoder might try to optimise how it sends data - I think it tries to h264 encode things it thinks are videos
2. Try changing the display dimensions of the remote desktop - you don't say what display resolution you have set, but the smaller you make it that is still usable for you application the less bandwidth you will need and the kess work the client will have to do.
3. Try running the client in different browsers and see if you observe any difference. At a guess as things appear to degrade over time it's possibly a JavaScript garbage collection thing and different JS engines have different strategies and optimisations so it's definitely worth trying different browsers.

Another thing you could try is instead of your game try running a browser inside your remote desktop and do something like play a YouTube video on that and see if the performance degrades with that too.

As I say the ubuntu-18.04-gnome-spice Dockerfile is really just taking the Spice-Web-Client and Xspice pretty much as is, so I'm a bit limited on any advice I'm able to give I'm afraid. I'm quite busy at the moment but if I get time next weekend I'll see if I can replicate what you are seeing though fair warning even if I can replicate it I'm not sure how much that'll help because the Spice-Web-Client code is pretty complex.

[yanbx]

Thank you for your reply. Next, I will try the suggestions as you said.
Explain that the CPU boost is not the browser CPU boost, but the Docker container CPU boost is 70%, the maximum can be 90%.
The phenomenon is that the browser access starts to be relatively smooth. After a period of time, when the Docker container does not stop, the browser is opened again, and the desktop and game that were originally running are running slowly.
It doesn't matter. If you have time and interest, please help me analyze my problem or give me some directions. I have several doubts as follows

1. Spice-Web-Client 2. Resolution 3. Docker Container Tuning
   I am looking forward to your reply again

---

In addition, just as you suggested, I tried to connect the desktop remotely with the remote-viewer and play videos on the desktop, but the result was also that the desktop was stuck and the operation was slow, so I doubt whether I should debug things related to Docker

---

supplement
I have tested it according to your suggestion, and found a phenomenon that when the host machine locks the screen, the problem of running slowly when using other machines for remote connection will appear. When the screen is opened, this phenomenon will disappear, so I doubt whether X11 is the reason for occupying host resources?

[fadams]

On "I tried to connect the desktop remotely with the remote-viewer and play videos on the desktop, but the result was also that the desktop was stuck and the operation was slow" That sounds like you simply connected via remote viewer to a desktop that was already slow because of your original problem. What I was meaning was trying to reproduce the issue by starting up a fresh run by restarting the container and only using remote viewer.

That said it sounds like it probably isn't Spice-Web-Client given that you are saying "but the Docker container CPU boost is 70%, the maximum can be 90%" but it would be useful to rule out the problem being an interaction between the SPICE server and Web client vice the SPICE server and another client.

On the CPU usage in the container you are going to have to be more specific about exactly what is using the CPU, I mean it is a full desktop and there are lots of processes running in that container so running something like top in the desktop to get a view of the processes consuming the most CPU is important.

On your last point - you say:
"
found a phenomenon that when the host machine locks the screen, the problem of running slowly when using other machines for remote connection will appear. When the screen is opened, this phenomenon will disappear
"
That's interesting so to be clear in my own mind you are saying that you have a host machine that is running the ubuntu-18.04-gnome-spice container and you have another machine running the Spice-Web-Client and you are saying that when the machine running the ubuntu-18.04-gnome-spice container has its screen lock kick in that is when you see things running slowly, but when the screen is unlocked again things go back to normal.

If that's the case I have a hunch about what the problem might be.

So you are using this for a 3D game, right well the way the remote 3D/OpenGL stuff works is that I use VirtualGL https://www.virtualgl.org/ which basically intercepts the GLX calls and redirects them to a 3D X Server for rendering - explained in detail here https://virtualgl.org/About/Background in particular in the GLX Forking section "In-Process GLX Forking with an X Proxy"

Now the "main" X Server used by the desktop is the Xspice Server, however if you look in the ubuntu.sh launch script you will see:

    -v $DOCKER_XAUTHORITY:/root/.Xauthority.docker:ro \
    -v $DOCKER_XAUTHORITY:/home/$(id -un)/.Xauthority.docker:ro \
    -v /tmp/.X11-unix/X0:/tmp/.X11-unix/X0:ro \
    $GPU_FLAGS \

That stuff is authenticating with the host's X Server and connecting to DISPLAY :0 on the host as it is the host's X Server that is acting as the 3D X Server (if you look on the page I linked what I'm saying should hopefully make sense).

So what I'm suspecting might be happening, given this important piece of information about the screen locking, is that when the screen locking kicks in on the host then the container loses access to the 3D X Server that it is using for accelerated 3D rendering and falls back to software rendering using the llvmpipe OpenGL path. That would account for the slow down and increase in CPU utilisation (software rendering is slow and CPU intensive) when the screen gets unlocked the container can "see" the 3D X Server again and so it returns to using 3D acceleration hence "When the screen is opened, this phenomenon will disappear".

That is all just a hunch on my part, but I think that it is a pretty plausible hunch. If I get time over next weekend I'll try to reproduce, but that additional information is useful and I suspect my hunch might be right.

Update - I've just done a quick Google search. Are you by any chance using Nvidia graphics hardware? In this page https://wiki.archlinux.org/index.php/VirtualGL#Confirming_that_VirtualGL_rendering_is_active I came across the following:

Problem: All applications run with 1 frame per second

If you use newer Nvidia drivers (e.g., version 440) you might be affected by a screen locking problem, which will reduce the framerate to approx. 1 frame per second according to the VirtualGL mailing list. Instead of downgrading the Nvidia driver one workaround is to set HardDPMS to false in your X server configuration (see NVIDIA/Troubleshooting#Driver 415: HardDPMS for details).

I can't say for sure if any of this is the reason for your issues, but that's where I'd start looking.

[yanbx]

Your reply is very important. At the same time, I would like to tell you about the graphics card configuration of my own server. One of my current server configurations is as follows: NVDIA Corporation TU116[GeForce GTX 1660]
Should be the latest graphics card, I also use the latest driver.But I also suspect that you said that the path of GPU acceleration and the graphics card driver were closed due to the access of local resources and the lock screen, but I don't know how to solve it specifically.
Then I will first test it according to your prompt. I also look forward to your time to copy and run it and help me analyze it!Looking forward to your reply

[fadams]

You need to read the links I posted. The point is if that is the issue, and as you are using Nvidia hardware it quite probably is, then

one workaround is to set HardDPMS to false in your X server configuration

So I would try that first.

Is the monitor connected to the host running the container via DisplayPort or HDMI? I noticed in the Nvidia Troubleshooting page it says

Proprietary driver 415 includes a new feature called HardDPMS. This is reported by some users to solve the issues with suspending monitors connected over DisplayPort.

I don't know whether the DisplayPort bit is important in this context, but you could try connecting the monitor using HDMI to see if that has any impact. I've no idea if it will but it might be worth trying before messing with X server settings.

[yanbx]

In addition, I found another problem in the afternoon test:
When the screen is not locked and the browser is left for about an hour after the browser is remote, the browser will also stall and run slowly

I did a test, a long time to place, resulting in the browser resources exhausted, eventually jammed access to the browser, resulting in the client CPU up to 80% of the ratio.Then I also doubt that this program is also a lot of client browser resources?How does spice-web-client affect browser resources?

[fadams]

On the Nvidia - to be clear HardDPMS needs set to false - from the VGL mailing list

From the changelog for 440.26:

• Enabled HardDPMS by default. See the README entry on the X configuration
  option "HardDPMS" for more information.

Try,

Option "HardDPMS" "false"

and

This solved the issue for me on CentOS 7.7 with NVIDIA 440.36 and VirtualGL 2.6.3.
Thanks Patrik !

On the other thing "When the screen is not locked and the browser is left for about an hour after the browser is remote" not sure what that is saying? Do you mean you are running a remote browser in the virtual desktop. If so is it the remote browser that runs slowly or the client browser?

Just seen your update "resulting in the client CPU up to 80%""How does spice-web-client affect browser resources?" well spice-web-client is entirely written in JavaScript and is running in the client browser so of course it will affect the browser resources. Why it goes to 80% after a period of time I have no clue, you'd need to ask the developers that.

I really need to go.

[yanbx]

Client Browser My own locally accessed browser is slow
The first you busy
"sed -i 's/#Option "HardDPMS" "false"/Option "HardDPMS" "false"/' /etc/X11/spiceqxl.xorg.conf && "
I tried to add this configuration to Dockfile, but it didn't work, I don't know if this is correct?

[fadams]

I think you really need to read the links I posted above properly.

The HardDPMS stuff has got nothing to do with anything in the Dockerfile. I've already said that this is most likely to be an issue that relates to the 3D X Server. which is the one on the host.

BTW normally Nvidia doesn't use a regular X configuration file, so I think you'd have to open up the NVIDIA X Server Settings GUI and under X Server Display Configuration press the Save to X Configuration - and then edit it to add the HardDPMS stuff. And again to be clear I have no idea if any of that will work I'm just trying to be helpful by pointing you at the links I found.

Given that particular issue is happening when the host screen lock kicks in did you read this https://virtualgl.org/About/Background? I really think you should as it would give you more understanding of how the remote 3D acceleration works.

On the issue with Spice-Web-Client I think it's an unrelated issue/ I tried earlier running the glxspheres64 executable and I saw the same sort of thing. One thing that you could try with that - and this suggestion does relate to the Dockerfile - is to change the --deferred-fps 60 value in the Xspice command line. Maybe reduce it to 30fps and see what happens, The Xspice documentation says https://manpages.debian.org/testing/xserver-xspice/Xspice.1.en.html

--deferred-fps DEFERRED_FPS
If given, render to a buffer and send updates only this many times per second

I've no idea if that will help the browser CPU issue, but it seems plausible that it will, because if you reduce the fps being pushed at the client then it should be under less load.

[yanbx]

First of all, the first lock screen is causing slow performance, it is exactly what you said NVDIA parameter problem fixed "HardDPMS" "false", there is no lock screen lag and slow performance problem.
The second problem is the performance problem of the browser. I set the FPS to 30 as you said, which can only prolong the deadlock problem, but too long time still leads to the deadlock of the browser, which may be a difficult problem to deal with.
The way I do it is to refresh the browser's resources regularly in my web front end

[yanbx]

Hello, this problem myself this way, the first lock screen that question, is, indeed, you say Nvidia's problem Correction parameters, as you said the second browser performance problems, my side will spice - web - the client modify this part of the js code, spiceobject. Js, return to the browser output blob logic optimization, they are using method is not release the memory, and returns the image multiplied into memory, so lead to browser card dead, so you also pay attention to this problem, if your 3 d games, frames are high,The problem is obvious.

In addition, could you please tell me which size the browser displays on the desktop? Can I set it by myself?

[fadams]

I'm sorry I think the translation software has garbled what you are trying to say in your last post.

Could you maybe try and rephrase it?

I understand the first part - I believe that you are saying that the change to HardDPMS false described in https://wiki.archlinux.org/index.php/NVIDIA/Troubleshooting#Driver_415:_HardDPMS resolved the screen lock issue, correct?

What I don't fully understand is what you are trying to say with

my side will spice - web - the client modify this part of the js code, spiceobject. Js, return to the browser output blob logic optimization, they are using method is not release the memory, and returns the image multiplied into memory, so lead to browser card dead, so you also pay attention to this problem, if your 3 d games, frames are high,The problem is obvious.

My suspicion is that there is a memory leak or a problem with garbage collection in the spice-web-client. Are you saying that you have looked at the code and know where the issue is or are you just saying that you think it is a memory leak too?

As I've said before the spice-web-client isn't my software, I've only pulled the release from https://github.com/flexVDI/spice-web-client/archive/3.1.0.tar.gz in my Dockerfile.

If you have actually identified the problem and have a patch the best course of action is to contact the authors of spice-web-client, though if you actually have a patch I'd be very interested myself.

On your last point

In addition, could you please tell me which size the browser displays on the desktop? Can I set it by myself?

How do you normally set your Display size? It works the same as a regular Gnome desktop - in the top right there are buttons that you click to power down and also to bring up the settings window (it has a little spanner and screwdriver icon). In the settings window if you navigate to Devices then Display there are setting for Orientation and Resolution - you can change the Display size in there.

[yanbx]

The browser performance problem is that there is a side code in the spice-web-client that does not free the browser's memory for that purpose. You can monitor it through Chrome and look at its download resources. It is a superposition that does not free the memory.
The HTML code is as follows: spiceobjects/spiceobjects. Js:
BytesToURI: function (data) {
Var blob = new blob ([data], {type: "image/jpeg"});
The return URL. CreateObjectURL (blob);This method does not free memory
},
The correction is as follows:
BytesToURI: function (data) {
Var blob = new blob ([data], {type: "image/jpeg"});
Return new Promise((resolve, reject) = >;{
Var reader = new FileReader();
Reader. ReadAsDataURL (blob);// Free memory but slow down the return speed
Reader. Onload = function () {
Resolve (reader. The result);
};
})
/ / the return URL. CreateObjectURL (blob);
},

---

There is also a piece of code as follows: \spice-web-client-3.1.0\lib\ rasterengine.js
img.onload = function() {
URL.revokeObjectURL(url);
var box = stream.computedBox;
// we only rotate the stream if spice tells us so through the TOP_DOWN flag mask
if (!(stream.flags & wdi.SpiceStreamFlags.SPICE_STREAM_FLAGS_TOP_DOWN)) {
var offsetX = box.x + (this.width/2);
var offsetY = box.y + (this.height/2);
context.save();
context.translate(offsetX, offsetY);
context.rotate(Math.PI);
The context. The scale (1, 1);
context.drawImage(this, box.x-offsetX, box.y-offsetY, box.width, box.height);
context.restore();
} else {
context.drawImage(this, box.x, box.y, box.width, box.height);
}
};
Revised to:
wdi.SpiceObject.bytesToURI(imageData).then(res=>{
img.onload = function() {
URL.revokeObjectURL(url);
var box = stream.computedBox;
// we only rotate the stream if spice tells us so through the TOP_DOWN flag mask
if (!(stream.flags & wdi.SpiceStreamFlags.SPICE_STREAM_FLAGS_TOP_DOWN)) {
var offsetX = box.x + (this.width/2);
var offsetY = box.y + (this.height/2);
context.save();
context.translate(offsetX, offsetY);
context.rotate(Math.PI);
The context. The scale (1, 1);
context.drawImage(this, box.x-offsetX, box.y-offsetY, box.width, box.height);
context.restore();
} else {
context.drawImage(this, box.x, box.y, box.width, box.height);
}
};
img.src = res;
});

[yanbx]

The problem with the lock screen causing slow performance: that's exactly what you mentioned
https://wiki.archlinux.org/index.php/NVIDIA/Troubleshooting#Driver_415:_HardDPMS
I did this. I changed the host's physical configuration, it didn't generate X11 / XORG.CONFIG and then I set the parameters in there, and it worked.Problem solved.

[fadams]

Hi again. That's neat.
Could you tell me how you actually tracked this down? You said

You can monitor it through Chrome and look at its download resources

To be honest I'm not especially familiar with Chrome's debugging and profiling, could you maybe spell out a bit more exactly what you did 'cause I'm interested in trying to reproduce what you did when I get a bit of time.

Also from what you seem to be saying above the issue is basically with the

URL.createObjectURL(blob)

call in bytesToURI, correct?

I've actually just done a bit of a Google web search on the call and two things I noticed:

1. in https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL there is a section on MemoryManagement that says: "Each time you call createObjectURL(), a new object URL is created, even if you've already created one for the same object. Each of these must be released by calling URL.revokeObjectURL() when you no longer need them."

2. More importantly URL.createObjectURL seems to be deprecated: https://developers.google.com/web/updates/2018/10/chrome-71-deps-rems "Remove URL.createObjectURL from MediaStream" "The URL.createObjectURL() method has been removed from the MediaStream interface. This method has been deprecated in 2013 and superseded by assigning streams to HTMLMediaElement.srcObject. The old method was removed because it is less safe, requiring a call to URL.revokeOjbectURL() to end the stream. Other user agents have either deprecated (Firefox) or removed (Safari) this feature feature."

I noticed this other link on StackOverflow https://stackoverflow.com/questions/49886248/how-to-replace-url-createobjecturl with a comment:

function createObjectURL() is deprecated

replace this:

video.src = window.URL.createObjectURL(stream);

to this:

video.srcObject = stream;

Though it's not clear if this just relates to MediaStream or createObjectURL in general.

What seems strange though is that if you look in handleStreamData in rasterEngine.js the img.onload and img.onerror calls actually do appear to be calling URL.revokeObjectURL(url) so it's not obvious to me why the momory wouldn't be being freed up.

[yanbx]

First of all, since I sent you the screenshot, you can't see it, so I can only use language to describe it. First of all, I will talk about how to monitor using the Google browser. You can access it remotely on the browser, and then F12, the debugging interface appears.

Then, choose soure option in the debug interface and the GPU option, you will find that with your game run, he will download a lot of image, the image is remote to return to the desktop image, these resources will download every second, about 80 k, but you will find the download images will increase gradually, but don't release the memory, CPU monitoring browser, you will find
Very high, until the browser card dies.CreateObjectTurl (BLOB) does not free memory, but you mentioned URL.revokeObjectTurl (). This method simply transfers the downloaded object to another URL, but does not free memory, so it cannot be used at the appropriate time
Release the waste resources, so in a way, I carefully look at me to send the code snippet, in addition to modify this way, I will also be lib package under another way js file to initialize the object changed, let her time to release the CPU resources appropriately, you can monitor the source when you will find that he had not before the resource file, and observe the CPU will be in a stable range

[yanbx]

I'm currently trying to make a full screen display, but spice-web-client doesn't work that way, so I guess I'll have to fix this code, right?