New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Common/MemArenaWin: Improve file mapping logic. #11822
Common/MemArenaWin: Improve file mapping logic. #11822
Conversation
|
Hello, creating an anonymous memory mapping for the emulator's RAM causes problems for external programs that need a more performant method of reading from and writing to said RAM. While more popular tools such as Dolphin Memory Engine have stuck to process hooking, accessing the memory mapping instead nets a sizeable increase in read/write performance. I've included links to tests done in 2021 where both memory access methods were timed doing 500k reads and writes in a python environment which you can find here and here. In short, there are massive speed boosts compared to the older method, with the most prominent improvements being on Windows which saw a 99% reduction in spent time. In practice, this method only worked best on Windows, as on Linux the memory mapping is immediately unlinked upon creation, requiring programs attempting to access Gamecube RAM to actively search for it before Dolphin launches a game, it still works but it's not as intuitive as being able to hook into memory at any point in emulation. I presume a similar thing will occur on macOS and Android, but I have not tested either platforms. Addressing this is definitely out of scope of this PR, so I only came here to make a note about this and to ask the following question:
You can find a link to a github project that currently makes use of the memory mapping access method here as an example of its execution. Thanks for your consideration. |
|
You can see the reason I removed the name in #11737, that creates a second MemoryArena for a completely different purpose than emulated RAM. They both can't have the same name. I'm also eventually planning on making it possible to run multiple game sessions simultaneously in the same process, at which point obviously the different sessions can't share the same memory space. If you have any suggestions on how you want to arrive at a unique name for your purposes, be my guest, though I hope you do realize that you're relying on very specific implementation details here. |
|
For right now, the best I can think of is adding a boolean argument to GrabSHMSegment that can determine whether or not it should use the string it generates as the name for the memory space. The Jit Cache wouldnt need a name for its memory space and can opt out of using a name, and the emulated RAM could act as before just with a true value appended to its arguments. Of course this would not work in your planned case of multiple game sessions under the same process, so the only ideas I came up with in that situation is either providing an extra string argument to act as a discriminator between each memory space, or passing a numeric value that lists the amount of active sessions and having that be applied to the string name. |
|
Can’t you just include a guid or something in the name |
|
Possibly. That comes with the assumption that an external program would be able to query Dolphin's available memory mappings. My knowledge of this is still rudimentary at best, I stopped looking into this shortly after the provided test results were made due to how Dolphin manages the memory mapping on Linux. From what I've learned though, handling guids would be easier on Linux since you are scanning /dev/shm. I don't know what would have to be done on Windows, macOS, and Android(I only mention Android for completeness, I doubt it would be a primary platform for this use case). EDIT: I did some Windows API digging and learned enough such that finding the emulated RAM will not be a problem no matter what the string name of the memory mapping is, so long as there is actually a string name to go along with the memory mapping. Finding a memory mapping without a name is a more complicated process but it seems to be doable if one either got lucky with just calling OpenFileMapping with nullptr as the name, or doing a more thorough check on the available memory regions via VirtualQueryEx. Linux is still a much more restrictive environment, but accessing emulated RAM by file descriptor seems to be a promising theoretical method that I can look into. |
a43517e
to
8342164
Compare
|
I've reverted back to using the name, but the user of MemArena now passes the desired name and is responsible for making sure it's unique within the current process. That should satisfy all use cases. |
Noticed this while reviewing #11737.
First commit: Both CreateFileMapping and MapViewOfFileEx want their memory size/offset (respectively) as two separate high bits and low bits parameters, for what I assume are ancient legacy reasons.
Second commit: The last parameter of CreateFileMapping is a name so that other functions can find this memory mapping using the same name. This is explicitly what we not want and it can be set to NULL to not give it a findable name, so let's just do that.
With these commits #11737 behaves correctly on Windows, before I was not seeing the supposed performance improvements (probably because something with the memory mapping failed).