Use memory mapped files for managed textures #2524
Conversation
|
Thanks for the review. I guess my lack of C++ experience is really showing here. |
|
Addressed the feedback. |
|
every time I try to merge this in and run Return of Reckoning (warhammer online) with the latest wine and mesa the launcher crashes straight away wine: Unhandled page fault on read access to 0000000C at address 6244DDDC (thread 0110), starting debugger... Unhandled exception: page fault on read access to 0x0000000c in 32-bit code (0x6244dddc). includes even the frog patch |
|
@wils0nyan Can you make an apitrace? |
|
ok here it is I'm not sure if this is useful this is unrelated but even though the launcher loads without the unmap merge the box is still blacked out/flashes details every now and then |
K0bin
force-pushed
the
unmap-managed
branch
2 times, most recently
from
24735cd
to
fe27aec
Compare
|
The launcher crash should be fixed. |
K0bin
force-pushed
the
unmap-managed
branch
5 times, most recently
from
9aedff8
to
fd7cf5e
Compare
K0bin
changed the title
K0bin
force-pushed
the
unmap-managed
branch
3 times, most recently
from
e66062e
to
7b78f0c
Compare
K0bin
force-pushed
the
unmap-managed
branch
3 times, most recently
from
a0e3c20
to
165c5f1
Compare
|
can confirm that the crash is gone but also have the following going: and merged: no crash all good so far mesa did break hevc 10 bit encoding on my ryzen 5700g recently with that efc stuff otherwise the additions to dxvk and the underlying drivers have been awesome over the last 6 months, it's improved a great deal thanks for the work you guys do |
|
Great work. |
|
Basically anything that's 32bit and currently crashes. |
|
Ich will dig through the issues |
|
Hi, Is it possible to extend this approach to optionally do this for D3DPOOL_SYSTEMMEM? There's a family of games that mirrors SYSTEMMEM texture for each DEFAULT one, perhaps because it does some CPU processing on them, sometimes. If those textures could be pushed out of process' address space it would be a huge win there. Right now, I am trying to change the game itself via reverse engineering. My naive idea was to push everything to GPU memory (DEFAULT), but looks like it is more tricky than that. In any case, such a change looks to be way too invasive and complex... if memory pressure could be reduced via DXVK that would be gold. I will gladly provide apitrace and test such fix. Thanks! |
It's definitely possible but a bit more annoying because you can write to SYSTEMMEM textures on the GPU. Can you open an issue for that and attach an apitrace? I can look into extending this for SYSTEMMEM after landing the initial version. Which games are we talking about anyway? |
|
I am messing with a 32bit game called GTR2. However, same or similar game engine is used by rFactor, Automobilista and dozen of other games, so I am guessing they might benefit as well. One important detail about those games is that they are semi-open. They are not opensource, but they are by design made in such a way, that people can create their own cars/tracks, and customize nearly any aspect of the simulation. There are plenty of tools and a lot of stuff is adjustable via text configuration files. The stock GTR2 won't run out of memory. But if you use some modern community content with highly detailed meshes and high resolution textures that's where the problem surfaces. It is not possible to simulate real events running, say 67 cars, because game runs out of memory at 30 cars. I am in a middle of debugging, but it seems that game creates sysmem texture, and it keeps them around (creating the address space pressure), and dynamically manages matching default textures. I see some calls to LockRect/AddDirtyRect/UpdateTexture etc. I do not yet understand why that is done that way, but I think at least some of the processing is on the CPU side. Buffers contribute to memory pressure too, because even though they seem to go to default pool, they are DYNAMIC usage, so they seem to eat up process' address space as well, but not sure how big of a problem that is. All that said, would issue with an apitrace still help? EDIT: I think I'll spend more time debugging here as I am still not sure I diagnosed the problem correctly, don't want to waste your time before I am 100% sure. |
|
Renamed some symbols from _MANAGED to something a bit more generic in case we ever decide to extend this to other pool types. |
This is annoying to maintain and hopefully won't be necessary anymore.
This will be used to identify managed textures that have not been used for a while.
|
Some games really work better now with this version, but cannot solve #2417 |
|
Superseded by #2663 |
In order to finally fix some of our address space problems, I took a page out of Gallium Nines book.
Managed textures now use memory mapped files to store the internal data. The data is suballocated from 4MB (totally arbitrary) memory mapped files. This allows us to map and unmap that memory as we see fit. This only impacts managed textures, systemmem textures still use the DXVKBuffer to store their data because they can be used with
GetRenderTargetDatato read back data from the GPU. I don't think those are a problem anyway. Buffers are also unchanged. It's possible to extend this to buffers but I'd like to get some feedback first.Unmapping follows a simple heuristic: We track the frame in which a texture was last used with
LockRect,AddDirtyRector when it was actually uploaded. If that was more than 16 frames ago or we've crossed the threshold of 512MB for mapped textures, we unmap. I hope that's good enough for games that keep the pointer around after calling Unlock. If it's not we can either try to tweak the heuristic, come up with something smarter or just disable it using a config option. I'd rather not punish decent D3D9 games that just happen to use too many high res managed textures because some do stupid things that aren't supposed to be allowed.All of this is disabled in 64 bit builds of course.
The branch is basically a prototype. It works pretty well with Borderlands 2 and Witcher 2, both of which use a ton of managed textures. I'd like to get some feedback on this. Should we also extend this to buffers? Do you have any other ideas for the unmap heuristic?