Save States support #313
Comments
|
Was about to request the same feature, but wanted to research forks that already implemented it first, as other feature requests here did that much due diligence. I'm really thankful for you considering to add this feature, considering your fork is in the best position to offer a more reliable version of this feature that's actively supported. I'd like to add more reasons why I see this feature justified:
I suggest the Retroarch frontend as an inspiration for how to handle this. It includes:
|
|
Just putting my 2c in here: personally, I don't use save states in emulated games, but I totally see how this feature is important for many users. It's probably also pretty important for:
I haven't investigated any patch implementing this feature yet. I will appreciate if you'll link me to forum threads or places where such patch was already posted / discussed, so I could import it as a branch named sidenote |
|
This SVN build is supposedly just the SVN build plus save states, so a diff between it and vanilla SVN should provide a recent version of a save state patch. That thread just has builds, but you may be able to ping that user for their source. https://www.vogons.org/viewtopic.php?t=53116 This is apparently the patch by itself: |
|
Just thinking about it logically, in a perfect world we would save all written-variables (and arrays), and then reload them when restoring state. That's pretty darn invasive.. basically, we need to attach an "is-written" bit to every variable (and array) that we throw to the positive state when we assign values. Perhaps we'd derive these state-type variables from a common "SaveableState" class, which registers all objects with the general save & restore functionality. At this point, it feels similar to the instrumentation a compiler adds around every variable for debugging and catching memory access issues! Besides adding a lot of code pollution, my guess is it would probably hurt performance. So we then look at the less-perfect scenario: figuring out the minimal set of variables and arrays we need to store so most games sputter back to life, mid-game, without strange side-effects (knowing that some state variables will only be initialized once the game is underway again.. things like CDROM read sector position, and so on). That looks like what the current patch does; and it has game-specific handling injected at various points too. For console gaming I really appreciate save & restore states, especially some of the more hardcore arcade shmups. Although I haven't found a strong need for them in DOS games, I think it would be very nice to have a library or "chapters" of common states for each game. For example, every level of Silpheed, or each Planet in Space Quest IV, and so on. These could be crowd-sourced and be managed in another git repo. |
|
This may be a really ignorant statement, but don't you just need to save the machine state? CPU type speed, amount of RAM, and what is in RAM, etc? |
|
Without looking at the code yet - I hope we could get away with something like:
And then provide infrastructure to load the state as well, so user can load and unpause. |
|
That's right @ enderandrew; it's just that the definition of "machine" is more sprawling than a console. On top of the memory and CPU state, we've got many video cards each with "BIOS" and their state variables, many sound cards (and their states), CDROM state variables, file system handles (which files are open and their current read/write positions) that map into the host filesystem, midi data in-flight, external sockets/connections, SDL's runtime configuration, and many more. |
|
@kcgen Damn, you are right, this would also probably need to track renderer state to track loaded vga palette… also restore resolution :/ After briefly looking at the patch - it's so long because it includes some external libraries, which we probably don't need - the same functionality is provided already either by zlib or xxhash we already use. I'll import it to Git, so we can read it more easily. |
|
@dreamer; the state of the code right now, I envision the save-state code to be like wrapping a spider's web around a chess board. All the pieces become harder to deal with, the code becomes far less readable (obscured buy saving and loading code around everything), and overall becomes quite a mess. A bottom-up solution, where the saving and loading sits underneath the code - if all saveable variables and arrays derive from a common base-class, is the way to go. |
|
I think this will also have consequences for memory protection in cpu emulation caches… which is a critical problem to address (it's reported only for Linux, but it affects all OSes - one day major OS vendor might flip the switch "we no longer allow writeable/executable memory for unsigned apps" or something like that)… NetBSD does something like this already IIRC. In my opinion, #253 is a critical issue, that needs to be handled before we'll start work on save states (but it's only an opinion, I did look into that bug already, but only briefly glanced at save states patch). In the meantime, work on save state should probably start with splitting it into smaller chunks and removing minizip and other dependencies we don't want. (GNU patch has some problem with importing this diff, I am working on it). |
|
Both GNU patch and Git are convinced the original patch is corrupted, but I couldn't recognize what was the problem; I imported the "corrupted" parts manually. Patch is imported to branch The patch is absolutely massive. If someone intends to work on this, then my initial suggestion is: right after resolving all code conflicts after rebase to master, start with removing minizip and other libraries bundled in here. We already depend on zlib via libpng, so there's no point in duplicating the dependency and making everything more complicated. However, my initial assessment of this patch is: it's in very bad state, extremely intrusive. If we'll work on save states, I think it'll be more productive to start the work from scratch - maybe some ideas or code can be salvaged. If this was Rust, most of this code could be automatically-generated and not written and updated by hand, maintaining this code would be nightmare. |
|
@BenMcLean wrote:
So this is +1 voice for including this feature ;) |
|
Don't know how I missed this issue because I read the title of every single issue before I tried posting. Disappointing to hear that there are problems with the old patch. This would be my number one feature request. Back when I was a kid, I had waaay more time than money. So it was OK that these old games were somewhat of a time sink because I had to make each one I bought stretch out until I could afford another one. But nowadays, I have money to re-buy old games on GOG but don't have time to replay parts of the game I already solved over and over just to get another chance at the difficult part. Save states give me back some of the time that's now in short supply as an adult. |
|
Yeah, I understand how this is a pretty important feature - it would have it's use cases also e.g. for making some bugs easier to reproduce… But a different approach is needed in my opinion - but that patch is pretty helpful anyway, as it identifies chunks of code that need serialization support! So, I mentioned already that this feature would be much easier to implement in Rust than in C++; another approach could be to use Google's Protocol Buffers library for state serialization - it would require a major rewrite of parts of code, but would be easier to maintain long-term. Overall, going forward we need to think about how to bite this without closing development paths for other features. The initial actionable task would be to clean up the patch (out with that minizip lib!) and make it easy to re-apply on top of newest master (but keep it separate initially). Out of curiosity: can we start listing games, that benefit the most from such patch? If they share some common characteristics, maybe we could cut the scope of the feature (e.g. avoid saving VGA state or something). |
|
I could be wrong but I think saving the VGA state is going to be required. Some of these games do real crazy stuff with the VGA |
|
If anyone stumbling upon this issue needs save state support, DOSBox Pure supports RetroArch's save states system. |
|
DOSbox-X also has a save state mechanism. Separately, in addition to save/restore memory and CPU state - what about snapshot capabilities like Virtualbox, e.g. for the HDD/CD images, configuration changes, etc.? To have a "ROM-like" frozen state (snapshot) and subsequent storage changes to be recorded in a subsequent-state-file(s). User to be able to "merge" or not those back into the ROM frozen state (or in a copy of that). |
|
I researched this extensively as a gamer. Currently Staging is the best ever Dosbox-Gaming-Bliss solution we have. Therefore it makes a lot of sense to create at least a limited save-state build: 32-bit VGA/SVGA, Sblaster+Roland Midi+Fluidsynth essentially the entire supremely amazing Staging Sound System. No net, no printer, no anything else required as this is for a Special PC Gamers Limited Build Only!! I want Staging to support Wizardry VII: Dark Savant = VGA, SBlaster/Roland/Fluidsynth + mouse in an easy MINIMALIST, 32-bit build. |
I really hope I won’t have to deal with this. |
|
I'm quite sure you won't have to. 😁 |
|
@FFnewbie - saving and restoring the state of an application is most easily done at the operating system level; it's just too bad this ability isn't exposed to users (Think of "suspend and resume" functions in most operating systems.. they can save the entire OS, drivers, and running application state to disk; the functionality exists!) If you're running Linux, check out the CRUI project.
CRUI gives you save and restore for every application. |
|
First of all, thanks for the kind words @FFnewbie 😄 I'm glad to hear you're enjoying the project! Well, you mentioned Wiz VII and you seem to really appreciate the audio enhancements, which practically means that now I'm officially bribed 😎 You seem to realise that savestates are unsupportable for the general use case. So that's for the cases when external hardware and the filesystem are involved. However, just because that is true, it could be supported in a very reliable way for more restricted use cases as you've identified. Those restricted cases would be still massively useful, so let's not throw out the baby with the bathwater just yet... at least not in principle 😅 When the whole state of the emulated machine is held in memory, in principle we could 100% support save states. When using host filesystem mounts, the content of the host FS can of course change between making the savestate then reloading it later. But, this is something I consider a theorethical non-issue. Savestates are shortlived and ephemeral by definition and they break easily across emulator version upgrades--which is fine as they don't replace regular saves; they are meant to be used as a "quick undo" during a game session, or at least within short timeframes (e.g., I create a savestate today before I need to go to sleep, then continue my gaming session tomorrow). Simply put, the host FS changing in the meantime is a non-issue; the user needs to ensure not to muck around with the FS for that particular game for the duration they want the savestates to remain valid. That's not a dealbreaker at all, that's expected, and that is how other emulators handle savestates too (e.g., WinUAE saves just a symbolic link to your active floppy images; "oh, you've moved/renamed/deleted your floppy image? Too bad, the savestate now is invalid--be more careful next time" 😄 This is fine.) I also like your suggestion of supporting a minimally useful subset of features only first. Where the difficulties come in is that this would still be a huge amount of work to implement it in a clean, non-hacky way, and we simply don't have the resources to focus on this right now because we already have big plans for the next say 2-3 years. However, this is a high-value killer feature that we should keep in the back of our minds. We will add an OSD (on-screen display menu) to Staging in the future which will necessitate the normalisation and fixing of a lot of state management. That would pave the road for a minimal savestate implementation. So yeah, the OSD will definitely come before this, but I'm personally interested in adding support for such a high demand killer feature. But, you'll need to be patient until we get there 😄 |
|
My understanding is that DOSBOX save states would only record the state of the emulated RAM and CPU, (and I don't know, maybe also the sound and video hardware) not the storage. This is normal. But I suppose it'd be nice to also have a way to make a file that contains all of the state. |
And the state of all emulated hardware, soundcards, MIDI devices, VGA adapters, and so on, and so on... Every single emulated device needs to be restored at the register level when resurrecting a saved state, as if nothing happened between the save and the restore. With external hardware you can't really restore state, e.g. running Munt as a separate process or actual MIDI hardware. All games initialise the MIDI hardware at startup. We can store the state of the MIDI devices emulated internally by DOSBox into the savestate, but that would be much harder with external MIDI devices. And how do you restore it? It all becomes quite tricky and yeah, better not to even go there.
Except when you're playing a 4 to 8 CD FMV game 😄 Thinking more about this, conflicts with the config are also a problem. E.g. you have some local In general, in "savestate mode" the available options and tweaks should be vastly reduced, maybe going as far as "no changes to the config" apart from trivial stuff. You wanna change your config substantially (other than say changing fullscreen/windowed or mixer levels), too bad, start a new game 🤷🏻 I can only see support added for this with such a vastly reduced scope—for the general use case, it would be madness. |
|
Even the mouse emulation might be problematic if someone sets mapping for a dual-mouse Settlers game with a friend. If both mice have the same name, we have literally no reliable way to distinguish them when restoring a save state. |
True, and I'm sure there are more of these if we start thinking about it. I guess, my point was:
To be fair, Settlers is another extreme example. Typical use case for this would be playing some arcade game in single-person mode that has no save feature. Then instead of spending 8 hours completing level 1 alone by retrying it 100x times, you could play the whole game in 4 hours (all 10 levels) with the save state feature. When you die, you do not retry from the start, but you can basically "simulate" infinite lives. That's the typical use case, when we get into "let's support all the features", things become horribly complex and it's pointless, really... This would be just a handy feature to support quicksave for fixed configs for action games within the span of a single gaming session or maybe a few days or weeks (or games where the RNG is really not on your side... frustrating to get your full party wiped out in such cases in long RPGs, but quicksave to the rescue!) You just don't screw around with the game directory and the configs until you finish the game, otherwise you've just voided your warranty 😄 That's how I see it—simple, focused support could be added. General support for every weird combination of features that we support, and there are tons of them—no way, not in a 100 years 😄 |
|
There are forks of DOSBOX that already do this. Wouldn't it be a simple matter of just incorporating their changes? |
No. The code has diverged way too much already, and that's only going to be more the case as we're cleaning up the codebase. Plus, AFAIK, the savestate handling in those other ports are done in a rather hacky and dirty way—this will need to be done from scratch. The team had investigated incorporating those patches in the past, and that was the assessment. |
|
@BenMcLean , if you're curious to see, click on the still-maintained .patch file here: https://www.vogons.org/viewtopic.php?p=1106424#p1106424. About half of that 433 KB diff involves DOSBox module state capture and serialization code; it makes the voodoo and IMFC patches look little. |
|
I like the idea of save states in consoles so i don't have to start over on extremely difficult Ninja Gaiden series of games on the NES when i run out of lives like i had to when i was a kid. Thankfully all PCs had these highly sophisticated devices to save your game state called 5.25" floppy disks and later Winchester disk drives from the PC XT onwards to save scum my way through the games. |
But not all DOS games have a save game feature. Action games usually don't have it, and that's where you need them most. |
|
It would be nice if someone compiled a list of DOS games without save support. If they're mostly ports of arcade games likes Frogger or Dig Dug then those could be played on their native platforms with save states on RetroArch. Anyway having that list would help people on deciding on this issue and its total impact. |
Well, what you're saying is not completely crazy, you know 😄 I was more enthusiastic about this in the morning, but yeah, as I started to think about potential use cases more and how much of a pain it would be to implement AND maintain this going forward, there's not much of it left 😅 To put it differently, this would require a lot of time and effort that could be spent on better (read, a lot more important) things. |
|
VHD differencing and dynamic support was added to DOSbox-X - if code can be integrated here may be relevant for savestates (and cover partially the usecases at #166, #314) |
Yeah, it's unlikely we want so much complexity just for savestate support in a few games. Also, no one really wants to run VHD images, host integration is so much more convenient. But thanks for the heads-up 😄 |
VHD is nice if you want to support Win9x/ME due to the dynamic size support, and it is also better supported on Windows hosts for mounting HDD image files, compared to the raw image files that Staging supports. With VHD files on Windows 8+ hosts you can just right click them and mount them. With RAW HDD files, I only know of 7zip that can open them, and even then only in read-only mode. |
|
VHD support seems useful so it could be its own feature request ticket. Can you write one for us. Thanks. |
There are also games having saves but only load the level-start. Prince of Persia 2 comes to mind. (With its flimsy jump-timing and overall high difficulty, save stated would be a big blessing for it.) It's the same with mission-based games like Dark Forces, Comanche, Tie Fighter, X-Wing, Terminator 2029, Alien Breed, Lemmings, Raptor, Syndicate, Wing Commander and many more. Another usecase would be games with passcodes/levelcodes, so you don't have to write them down and enter them. This is especially useful for cases, where no keyboard is available (e.g. HTPC or arcade setups). Personally this is definitely my most wished for feature. |
There are old patches for Save States that Daum builds were using 5+ years ago, though it may make more sense to ask the DOSBOX-X maintainers for the version they are using currently.
While DOSBOX is not an emulator, it is a means for retro-gaming and Saves States are a common feature of emulators. It provides an enhancement that makes retro gaming better in DOSBOX than on native hardware. This is particularly nice in that many classic games either didn't have save features or had limited saves.
If you're trying to quickly get through a backlog of many retro games it can allow you to progress through an old game quickly (reloading after a death or bad event) or allowing you to step away and resume at your convenience.
While this is probably my number #1 most wanted feature, it may make sense to implement this later if you're planning any major refactoring of the code-base if you think that will require a refactoring of how save states work, or invalidate previous save states with updated builds.
The text was updated successfully, but these errors were encountered: