-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[draft / demo] Add network-enabled trace logging / add compact binary tracelog format #268
base: master
Are you sure you want to change the base?
Conversation
- These were left off because compression wasn't needed - Adding them back in for future work that needs to use zlib to compress - Note: There are much more recent zlib versions, at some point we should probably upgrade zlib in BSNES
- demo-only, needs more cleanup before an upstream merge - win32-only implementation for demo, replace with platform-independent libs - add socket server for clients to connect and receive trace log data - add new compact binary format for tracelogging (8 bytes) - with tracemask DISABLED, still achieves 45FPS (previous 2FPS) for tracelogging over localhost. with tracemask enabled, it's full-speed - extensible binary tracelog data format to allow future sending of arbitrary data to external tools
Neat stuff! Sorry I haven't really had the time to actually look at this yet, hopefully I can do that pretty soon. |
Cool! Yea no rush. I realized I forgot to document a couple of things with this. There's a couple constraints that were made as performance tradeoffs, some of them might also be backed off. My goal was to make this really fast while also being somewhat expandable (for instance I hope to hook the "usage" system into this). Some alternative ideas instead of using sockets for external communication:
Overall, it works pretty good though. I think dumping most of this into a plugin where it can be optionally compiled/enabled might be the way to go. Here's some more info on the pipeline: For every instruction executed, 8 bytes of trace info are generated: These 8 bytes are the smallest "abridged" tracelog format, but, you can enable "full" tracelog (about 20 bytes) and it will dump the rest of the info on registers A, X, Y, S, e flag, and 4 bytes for the actual instruction. We append 2 header bytes at the beginning for: As a first pass, I tried just having the main thread throw those 10 bytes down the socket as they came in, but, it was way too slow and even the non-blocking socket send call was too slow. So I ended up profiling and iterating the design, came up with this which seems to be working great, even if it's a bit more complex than I was hoping for.
This works fine on localhost because Windows seems to handle arbitrary packet lengths fine (and we send about 25k in one packet). If it was sent on a real ethernet network, it probably would fragment instead and that could potentially destroy performance. I think (not sure though) we'd have to tune our compressed packet size to be around 1500 bytes to maintain performance, which might require some other tweaking. Other note: the organization of the code is pretty sloppy, I shoved all this in a new file called w32_socket.h which is poorly named and contains classes that do 3 different things. So basically, let me know if you think this is something you'd like to see cleaned up and integrated into BSNES-plus upstream, or, if this is better off as a fork or a separate plugin. I'm new to SNES-specific stuff, so if I'm doing anything silly there, let me know. Also interested if this idea of remote interfacing with the CPU guts should be thought out at a higher-level layer like libsnes or something, I'm kind of new and unfamiliar with the broader ecosystem. Thanks! |
Hi! Thank you all for your work on BSNES-plus, it's an amazing tool.
I started working on tracelog modifications for BSNES to allow realtime streaming of trace data to outside apps. I originally thought this would be a throwaway hack, but, decided to clean it up and post it here in case others thought it was useful.
This code I would consider demo-only, for merging into upstream BSNES it needs some additional finishing like making it play nicer with the UI and exposing some config options. If there's interest in my approach, I can refine this further for upstream submission.
I was working on disassembly research for a 4MB ROM, and BSNES's tracelog capabilities were extremely useful. However, even a short run would produce gigantic gigabyte-sized dumps, and would cause BSNES to run at ~2FPS. The workflow for doing all this manually with an external tool was tedious as well.
This PR does 2 main things:
When both options are combined, tracelogging [even with tracemask turned off] now runs in (basically) realtime and you can play through a game or run a movie while fully logging everything with an external tool.
I have been working on modifications to an existing external tool called Diztinguish, which is a GUI for automating some parts of the disassembly process by marking an input ROM as code vs data, and marking the code with metadata about the state of the X,M,DB,D flags to enable accurate disassembly. Once the code has been marked up, Diztinguish can export source .asm files with labels and markup that can be compiled back into a byte-identical version of the original ROM.
The changes in this PR allow Diztinguish to talk to BSNES-plus: IsoFrieze/DiztinGUIsh#18
Here's a demo of what this entire system looks like running together. On the left is BSNES-plus, which is talking via socket to Diztinguish on the right. The areas in yellow showing up are 658c16 instructions being marked up in realtime as the game runs. This is a sped up video of about a 1 minute run (full version is here: https://www.youtube.com/watch?v=NCZUESf82Rg&feature=youtu.be)
Below is a highlight of Diz marking up the ROM in realtime, pretty neat visual:
Future work:
This PR adds a few files back in the embedded zlib source files included in BSNES (for compression functionality). They are unmodified from the original zlib v1.2.3 source.