-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Low frame rates on Starcraft 2 while CPU and GPU utilization is low #333
Comments
Actually most motherboards have very few PCIE x16 slots, usually one or two. While there are might be more full sized PCIE slots, they are actually working as x4 ones. @Venemo, You could also take a look what options are available for you in Thunderbold3 definitely complicates things. I had a similar slowdown caused by the fact that my distribution had i586 optimized glibc, that used a memcpy() with hack for original Pentium. It would read from the destination memory, before overwriting it with real data. When used over my PCIE x16 it would get something like 25% CPU in This reminds me... If you are compiling your own Also, I've tried to make a line that would show all graphs on the screen, try it in case it provides some hint.
You can check if you have an extra graphs with |
@iiv3 Thanks for the suggestions! I'll try them.
What was the solution to this one? How do I check if this is the case?
I honestly don't know, it is possible that I have messed up something in that WINEPREFIX. I will re-install the game in a clean prefix and use that for further testing. |
Every frame, vertices must be sent to the graphic card for the draw calls. There's also the draw commands sent by the driver. Good games don't upload anything else. As mentionned already, the vertices for SC2 are stored in a permanently mapped buffer. The allocation is done in GTT, ie graphic memory in cpu ram. At draw time, the graphic card fetches the data there. As for the draw commands, there's not much to do. Maybe as iive suggested you have wrong flags somewhere which leads to unoptimal functions somewhere. |
Okay, so I upgraded to Fedora 29, kernel 5.0-rc1 and wine staging 4.0-rc1. Then I deleted the wine prefix and reinstalled the game in a clean prefix.
There is a
Here is the output. I couldn't copy-paste the bars from the terminal, but the percentages are clearly visible.
I took a look, and tried a few options but some of them caused a crash, while others caused noticable performance decrease, so I didn't get too far with it. Is there a specific option you want me to try?
With
I just use the mesa that comes with Fedora. I'll also try the version from the "che" repo which is supposed to be a newer version with more optimizations enabled, will report back if it helps.
Here is a screenshot from the graphs with |
Oops, I accidentally hit "close and comment", sorry. Reopening it now. |
@axeldavy Thanks! How come that the samples passed and the ps/vs invocations are so low on your machine? Also the primitives generated seem much less than mine. |
It depends on the scene and resolution. |
Okay, I tried the patch suggested by @axeldavy and replaced Here are the graphs from the patched nine:Hotspots from
Here is the hot spot from the annotated
I also examined the same graph while running wined3d with the PBA patch:Hotspots according to
Which means that wined3d+PBA still beats nine by a dozen or so FPS. What stands out to me is:
Further notes:
Where do we go from here? |
Here is what
Here is what
Reducing the bandwidth using Trying to determine the available bandwidth using
However this egpu.io page states the following: Intel throttles 32Gbps-TB3 to 22Gbps which benchmarks as 20.40Gbps (AMD) or 18.91Gbps (Nvidia) in the important H2D direction. |
I took a look at the hot spots for Nine in
Looking at the other items like |
I'm on Debian, @axeldavy on Arch iirc. |
Okay, so @axeldavy Suggested an approach that helps find whether the constant data is the bottleneck or not:
Here is how the patch file looks:
When I run the game with this patch:
Now the
Interesting how
|
@dhewg Is there anything in here that stands out to you? https://src.fedoraproject.org/rpms/mesa/blob/f29/f/mesa.spec#_365 |
I also tried the the other suggestion by @axeldavy - here are the results.
Here is what I experience when I run the game with this:
This is the patch:
|
OOOPS, looks like I forgot to revert the other patch before applying this one. Will do another test with just the si_pipe patch. |
The @Venemo, setup involves thunderbolt to run the card outside the laptop. This limits it to pcie x4 speeds. To recreate his setup you should at least move your video card to pcie x4 slot. Most MB have only 1-2 PCIE x16 slots, and the rest are x4, even though they use the full x16 connector. You may try it if this is not going to void your PC warranty. |
Rebuilt mesa with the correct patch and updated my last comment. |
@Venemo: No idea about rpm spec stuff, but nothing stands out I guess. I just tried against mesa packages from debian, and it looks like the fps is roughly the same on sc2 |
While the |
@dhewg , to reproduce the issue you need to move the video card to PCIE x4 slot. As I said in my previous comment to you, a lot of MB have full sized slots that are x4. |
@iiv3 yeah, I get that it's a bottleneck that I don't run into. Just confirming that it's not a general issue |
I ported the change to I'd say that it is a consistent improvement, even thought 0.5% is around the margin of error. |
With vsync disabled in-game and |
I've run a series of tests for you. I'm posting the patches here too. All of these were applied on top of the ixit/master branch. In every case the same SC2 replay was used and I looked at the FPS counter at the same moment during the replay.
The maximum size of the buffer that comes from
@axeldavy has a better understanding than me, maybe he can chime in with a better explanation, but I think it isn't really faster, this is just a symptom of an inefficiency somewhere in the code. For example, if there are a lot of unused shader constants, then the const_uploader would needlessly upload all of them, but the stream_uploader would not (as far as I understand). In fact, changing everything to use STREAM is slower than changing just the VS constants to use STREAM. Increasing the default size of the const uploaderThe following patch gives ~50 fps, so no noticable improvement.
Changing the const uploader to STREAMGives ~75-80 fps.
Changing the const uploader to STREAM while also increasing its default sizeGives ~75-80 fps, so the same as before.
Hack that makes the VS constants use the steam uploaderThis one yields ~90 fps.
Hacking VS and PS contants to use the stream uploaderThis one yields ~80-85 fps, so worse than using just the VS constants with the stream uploader.
|
Just a few information |
@axeldavy Is it possible to somehow check if the game uses indirect addressing? Just to confirm or rule out that possibility. |
The game uses indirect addressing. |
Can we somehow eliminate the indirect addressing? |
Thanks. I think the problem with DEFAULT is that more buffers are competing for the visible VRAM window, which makes the memory manager move buffers more often. If you use STREAM too much, you might decrease GPU performance, because PCIe is slower. |
The same game has good performance on pcie x16 but not on thunderbolt. That seems surpring to me that the culprit for the performance manager would move the buffers in VRAM, as else pcie x16 would be affected as well. Except maybe if the buffers get moved to GTT ? That would explain a lot. |
Can you test this branch? |
@marekolsak Wow, thank you. I merged your branch with the ixit master, compiled a release build and ran the game with
Some more experiments that I tried:
|
Thanks that you added SDMA=true. I forgot to tell you about it. I've updated the branch with a performance fix. It should now reach 100% GPU load: |
@marekolsak, would it be possible to implement the same changes for the r600 driver? |
@marekolsak I've ran another series of tests for you.
Conclusions:
|
It would be possible, but the performance improvement wouldn't be as high, because the DMA engine is not asynchronous on radeon. For r600, I recommend changing const_uploader from DEFAULT to STREAM. @Venemo My series decreases performance for trivial benchmarks by 20%, so I need some amdgpu kernel changes to get the performance back. |
Surprising. Any idea what bad kernel behaviour is causing the performance drop ? |
We submit 2 CS ioctls. We need to merge them into 1. |
@marekolsak Can you shed some light on what |
amdgpu_mm_reg is used for getting the GPU load. |
Is the limitation in hardware or in the radeon kernel driver? |
It's a limitation of the radeon kernel driver. |
Can someone update this thread when the patch lands upstream? I have a r3g v1.1 pcie 3.0 x16 to m.2 x4 adapter + rx580 so I could test but I don't know how to build mesa or these out of tree patches. |
@montvid Sure thing! You'll hear about it when it gets upstream. I'm curious though, how does it perform on your setup? |
It seems SC2 is free to play so I got it from battle.net. Interesting during the first campaign the cpu (i3-7100U) and gpu (RX580 8GB) usage is always ~40%, fps is around ~60 (with esync enabled a bit more), GTT is ~100 mb, vram ~1200 mb (1900mb with all settings on ultra-extreme) buffer wait time max 20us (Assassins Creed 1 can get up to 20ms! on ultra settings so runs worse). I am using tkg 4.1 wine (lutris) a pcie 3.0 x4 adapter so it is not a thunderbolt issue. @Venemo I was hoping for better game performance from my egpu setup but thunderbolt is even worse according to egpu.io at least i have 8gb of vram to store all the ultra textures. :P |
@montvid is that with or without the SDMA patch? if without then it is roughly in line with what I experience here. |
Without the patch. |
Looks like the sdma patches landed on mesa master. I guess this can be closed then? |
Even though GPU utilization is near 100% now the game is still slower on my setup than yours. I'd prefer to keep this issue open until I manage to do a proper comparison between nine and windows on this hardware. If there is no significant difference then let's close it, otherwise we should try to find the next bottleneck. |
Here is an interesting tidbit. In kernel 5.1 amdgpu adds a way to examine the PCI Express bandwidth usage: So I ran
EDIT: fixed mistake in calculating the Gbit/sec. There are two interesting conclusions here:
@marekolsak What do you think about my numbers, do they make sense? Am I interpreting them correctly? |
Yes, the numbers look correct. |
It's not just the bandwidth that matters. It's also the latency that PCIe incurs. |
@marekolsak True. But 2.5 Gbit / sec is still less than 10% of what it should be capable of. There is definitely a problem in there somewhere. |
Long story short, I observed a scenario where I get low frame rates even though the CPU and GPU utilization was low. This is a continuation of this issue where we started discussing a performance problem with Starcraft 2 when running on Gallium Nine standalone.
Hardware details: the system is a Dell XPS 13 9370 with a 4-core / 8-thread Intel 8550U, a Sapphire AMD Radeon RX 570 ITX card is connected through Thunderbolt 3 (which is roughly equivalent to a x4 PCI-E connection), and a 4K Dell U2718Q screen is connected to the RX 570's DisplayPort output.
I didn't measure the frame rates in a scientific way, but the difference is very noticable:
Some observations from the other thread that are worth to mention here:
GALLIUM_HUD
CPU utilization is around 30% and GPU utilization is around 50%perf top
the majority of time is spent in functions likesi_set_constant_buffer
amdgpu_mm_rreg
andNineDevice9_SetIndices
from which I got the impression that some buffers are copied to/from the GPU and that may be a problem.@axeldavy
That does sound awesome, and makes me think that the problem may be with my setup, or that maybe my setup triggers some sort of corner case within Nine. After thinking about it more, I've got three ideas:
Idea 1. Maybe there's something wrong with my kernel command line? Bascially what I do is disable all the spectre / meltdown mitigations, I enable the PowerPlay features in amdgpu and blacklist the
i915
module.Idea 2. Maybe somehow Nine performs more IO through PCI-E than wined3d and the Thunderbolt 3 port is the bottleneck. Is it a possibility that there are some operations within Nine that are not noticable when using a PCI-E x16 connection but become problematic when running over PCI-E x4? I don't know how to verify this theory, but maybe there is a way to check the PCI-E port usage.
Idea 3. It occours to me that I installed DirectX 9 from
winetricks
in this Wine prefix before I started using Nine. Is it possible that this interferes with Nine somehow?@iiv3
No, I've got 4.19 from Fedora:
4.19.13-300.fc29.x86_64
Good idea, I will try that.
Sure, I'll take a look, though I'm afraid I'm not an expert in
perf
.Sorry for the long post!
The text was updated successfully, but these errors were encountered: