Frame rewind

[thrust26]

After a break, it should be possible to rewind the emulator by one frame. This would be very helpful for debugging.

Example:
Ever so often I face rare timer overruns in my games. I can use breakif to stop the emulator, but I cannot step back in time to identify the root cause of the timer overrun.

[sa666666]

Just wanted to follow up on this; I will be pushing this to post-5.0 release, since I'm trying not to add any new features so we can get the 5.0 release done (we've been working on it since Sept. 2016).

[thrust26]

Maybe the code about reverse emulation on this website helps: http://blargg.8bitalley.com/misc/

IMO reverse emulation would be even cooler! 😄

[sa666666]

I'll look at this at some point, but I already basically know how rewind would work; it's just a matter of sitting down and implementing it.

Basically, we need to figure out the 'resolution' of how we step backwards (1 second, 1 frame, 1 cycle, etc), and then continuously save states for that resolution. Then there needs to be a UI to expose this to the user, and allow to go backwards.

Not that I've discussed this much or given it a lot of thought, but my main issue is that we want rewind at the user level (for going back to hard-to-pass places in a game) and also for the developer (perhaps to rewind after a breakpoint). And how to combine two of them is still a TODO.

Maybe the resolution can be 1 second, but it can keep track of input in the meantime until the next second. So if, for example, you want to rewind to the previous frame, you would have to rewind to the previous second (60 frames back), then skip ahead 59 frames, so the final result is that you're one frame back (59 - 60 = -1).

Anyway, I digress on this. There are several exciting things we want to add, and at this point we have discussions going on in 4 different issues 😄 So for now, I think we need to concentrate on optimizing TIA, improving TIA startup time, and then the new sound code.

[thrust26]

For sure, I only wanted not to forget this finding.

[thrust26]

How about multiple rewind options here?

Example:

• we sample the state every frame and save it
• every 10 frames, the frames -11..-20 are deleted (-10 is kept)
• every 1 second, the frames -70, -80..-120 are deleted (-60 is kept)
• every 10 seconds, the frame -120, -180..-600 are deleted

So you can rewind 1 frame 10-20 times, plus 10 frames 6-12 times, plus 10 seconds 1-2 times.
(I hope I got the math right, but I think you get the idea)

Or just 3 ring buffers which are updated every 1 frame (9 entries), 10 frames (5 entries) and 1 second (2 entries). 😉 (here the problem is that rewinding needs some logic, else it might jump forward when switching between buffers)

Or very simple, a big ring buffer of 600 entries and all step logic in rewind.

[sa666666]

Getting this part right is basically the hardest part. The infrastructure is already there to save a state file to RAM, since I toyed with event recording and replaying some time ago (the code is commented out, but it did work). And of course we'd need a UI to expose it to the user. But I estimate that perhaps 80+% of the required work is already present in the current code.

[thrust26]

If you want to change the UI, then split the single < button into a < and a << button (if you want to have a hotkey here, then e.g. CTRL+SHIFT+R). The latter would rewind e.g. 1 second. But IMO rewinding currently works very fast, so I don't think the change is really required.

I don't see a reason for other UI changes.

[sa666666]

The UI stuff I was referring to was being able to rewind during emulation, for people to repeat a difficult part of a game, etc. This is closely related to being able to rewind in the debugger. Both are just different applications of the same thing, really. But perhaps we should concentrate only on rewind in the debugger for developers for now.

[DirtyHairy]

@thrust26 I just skimmed the code you linked and the linked nesdev thread. The idea is not actually reversing the state machine (which, on second thought, is impossible as transitions like the execution of STA 0xFF cannot be reversed), but taking periodic snapshots and filling in the spaces between by fast-forwarding emulation from the nearest snapshot.

[sa666666]

Yes, that's exactly what I was planning to do, years before I ever read about that article 😃 It is just finding the time to do it.

[DirtyHairy]

😄 It is a smart idea, but not all that exotic I guess.

[thrust26]

That's not even exactly reverse emulation, because during fast forwarding you would miss the previous user interactions.

I knew it before, magics doesn't exist. 😃

[thrust26]

@sa666666 I think the requirements for rewinding during playing and debugging are quite different. Debugging is mostly done on frame level, while playing needs saving in much larger intervals. IMO we should open a second issue for this.

Anyway...
Have people requested automatic rewind during playing? Would they want to load their savestates later or just rewind a few seconds while playing? The former would require a UI menu (e.g. for selecting and clearing the savestates), the latter only some hotkeys.

How about a mode creating automatic savestates every e.g. 5 seconds on disk? The name should contain a timestamp then, which could be either used by Stella to determine the next step back or by the user in the menu.

[sa666666]

Yes, and this UI menu is precisely what I was referring to above 😄 I think for now you are right, and we can just concentrate on the debugger. And have it rewind only 1 frame. And that functionality has to be turned on manually (ie, the developer has to enable it when they want it).

[thrust26]

At least one frame I hope. If e.g. a break happens in vertical blank I want to be able to go back to the previous vertical blank. And not only to vertical sync.

I suppose that means you have to store the last two frames, no?

[sa666666]

The 'only' in 1 frame meant the interval/resolution itself is only 1 frame. I can have 20, 50, 1000 actual frames stored, dependant only on available memory. Maybe I'll start with 60 frames (ie, 60 savestates of 1 frame each).

[thrust26]

OK, that's more than I usually need.

[sa666666]

I'm about to do the 5.0.2 release soon. After that (sometime in the coming week) I will look into implementing this long-requested feature.

[thrust26]

Another small request for improvement: display besides the rewind button how many frames will be rewinded next.

[sa666666]

The problem is that all rewind states aren't for one frame. In the debugger, a rewind state is created for each step/trace also, so how do we show that the next rewind will be by one step (other than in the text after the rewind happens)?

If you mean to show how many rewind 'slots' are currently used, then this is easy to do.

[thrust26]

Agreed, both kinds of rewinds should be the same. Actually I think they could both use the same buffer, and maybe the buffer deletion same algorithm too.

For now 'slots' will do.

[thrust26]

My further plans for rewind:

1. different for players (e.g. recorded every second, see 5.) and developers (recorded every frame)
2. controlled by a developer mode flag, which switches other options too (not only rewind related, probably in a developer option tab, see Developer options #151)
3. user controlled size of the buffer (e.g. 100..1000)
4. user controller horizon of the rewind buffer (e.g. 5 seconds..30 minutes)
5. user controlled intervals (1 scanline .. seconds/minutes)
6. 3..5. combined result into the density of the save states.
7. the very first saved state is never deleted
8. the first n (e.g. 60) states are kept unchanged, from then on the states are compressed exponentially to reach the horizon.

Example: recorded every frame, size = 100, unchanged = 60, horizon = 30 minutes. So within 39 states we must reach 30 minutes (=108,000 frames). That means the difference between states has to be increased by a factor of ~1.3.

Formula (sn = ~108,000; a0 = 1; q = ~1.3; n = 39):

This formula cannot be solved for q, so we have to approximate it.

[thrust26]

All implemented with e2301df, except for:

1. smaller minimal intervals (e.g. 1 scanline)
2. more navigation options (e.g. key repeat, slider...)

[sa666666]

Mostly complete at this point, and we leave the remaining items for 5.2.

[thrust26]

@sa666666 Any chance for getting button repeat working for 6.0?

[sa666666]

I will try, but I recall there were some serious UI issues that prevented it. For now I'm working on GameInfoDialog, but I will come to this one next.

[thrust26]

Fixed by fccfee0