Separate game logic and drawing updates; allow consistent game speed with variable frame rate

[parasyte]

There's a really good writeup on this idea here: http://www.koonsolo.com/news/dewitters-gameloop/

The concept revolves around running the game logic at a constant rate which is slower than the frame rate, and allowing the frame rate to run as fast as it possibly can.

The tricky part is properly interpolating object positions in each frame update, between game logic updates. And the article provides some ideas for handling that.

Note: this is very different from the me.sys.interpolation feature, which keeps the frame rate and game logic running at the same rate, but attempts to make up for dropped frames by dynamically scaling object velocities.

[melonjs]

lol, I wanted to put copy/paste here the url of an article I read a while ago, and just noticed that you mentioned the same one :) but effectively the game loop would deserve a better implementation :)

note as for now, enabling me.sys.interpolation can cause issue with collision detection, due to how detection is done. For example, in case of low frame rate, a player could go through a wall, as collision is done by checking the player next position, and will not check for potential collision between the initial position and the target one.

[parasyte]

Yep, that collision bug will always be there for any "fast moving" objects, especially small objects. The way to fix that is to extend the collision AABB to cover the entire range of motion. As in this image: http://www.wildbunny.co.uk/blog/wp-content/uploads/2011/12/tileCollision3.png

[greghouston]

When this is implemented I would like the ability to set the game speed on the fly in order to go into "fast forward" or "bullet time" mode.

[edited]
So if I set my regular speed is set to 25 ticks per second, my fast forward time might be 50 ticks per second, with the fps remaining at 60 fps for both.

[parasyte]

When I said "bullet time" I really meant "slow motion", as opposite to "fast forward". :)

[greghouston]

https://groups.google.com/forum/?fromgroups=#!topic/melonjs/WYt1V84fbfk

[greghouston]

Since this is so core to the framework it would probably be a good idea to add it to the 0.9.5 or 0.9.6 milestone.

[obiot]

+1 actually on this one, as it could partially be implemented now that at least the me.game logic is much more simple than before (and the dewitters loop is kind of easy to add)

[parasyte]

One of the challenges standing in the way of this is getting rid of the last remaining "frame counting" bits. Especially in me.AnimationSheet.update(): https://github.com/melonjs/melonJS/blob/master/src/renderable/sprite.js#L629

[obiot]

+1
It should not be that hard actually, but would just required maybe some clean-up on the me.timer stuff and add a property or a function that returns the elapsed time since last frame. Once we got that it’s very easy to replace the framecounter by something more “accurate”.

[mluyties]

This would be amazing for porting to mobile =)

+1 !!!

On Tue, Aug 20, 2013 at 6:19 PM, Olivier Biot notifications@github.comwrote:

+1
It should not be that hard actually, but would just required maybe some
clean-up on the me.timer stuff and add a property or a function that
returns the elapsed time since last frame. Once we got that it’s very easy
to replace the framecounter by something more “accurate”.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/99#issuecomment-22989716
.

Sincerely,
Mike Luyties, CEO of RenterMonsters.com
www.rentermonsters.com
http://www.rentermonsters.com

[obiot]

that should do it for the animation part ;)

as far as i can tell there is no more fps count stuff in melonJS now !

[parasyte]

A quick search for "frame" finds: https://github.com/melonjs/melonJS/blob/master/src/renderable/sprite.js#L151 as the last remaining one.

Also, the new animation code doesn't take the time delta into account (the same way me.sys.interpolation does) so it isn't the most accurate timer-based animation. With me.sys.fps = 30, animationspeed set to any number between 1 and 32 will run the animation at the same constant rate (assuming no frames are skipped).

Instead of always expecting to display the next animation frame in sequence, it should use the timer delta since the start of animation to determine which animation frame to display on each draw. And to do that, the animation needs to be given in total duration for the complete animation; not between each frame. In other words, I want my sword swing animation to run for exactly 100ms, no more, no less.

addAnimation : function (name, index, animationspeed) {
    this.anim[name] = {
        // ...
        duration : animationspeed
    };

    // ...
},

setCurrentAnimation : function (name, resetAnim) {
    if (this.anim[name]) {
        // ...
        this.current.startTime = me.timer.getTime();
    }
    // ...
},

update : function () {
    // update animation if necessary
    if (!this.animationpause && (me.timer.getTime() >= this.current.nextFrame)) {

        // get next animation frame based on timer delta
        var delta = me.timer.getTime() - this.current.startTime;
        this.current.idx = ~~(this.current.duration / delta);

        // switch animation if we reach the end of the strip
        // and a callback is defined
        if ((this.current.idx >= this.current.length) && this.resetAnim) {
            // ...
        }
        else {
            // advance to next animation frame
            this.setAnimationFrame(this.current.idx);
        }

        return this.parent() || true;
    }

    return this.parent();
}

[obiot]

beside of the delta issue, on my side I actually assumed that defining a time under 1000/fps would not be useful, as i hardly see what would be the use case behind having one animation frame actually visible for less than "1 frame(fps) time" ?

Good catch for the flickering stuff, this one is the next target then :)

(also temporarily tagging this one for 0.9.9, as some stuff are happening)

[parasyte]

It definitely is not useful to skip animation frames ... Unless the game cannot possibly display all of them in the given amount of time. That happens with animations designed to run at a specific FPS, and when the game cannot run at a constant speed.

If you always assume frame rate will not be constant, that's when it makes sense to allow skipping frames in animations with a high frame count.

Typically you'll see the opposite though; low frame count animations, so each frame is displayed on screen several times.

[obiot]

not really this topic, but still a good input to improve our main loop :P\

http://caspervonb.github.io/2014/02/24/javascript-game-development-asynchronous-execution-loop.html

[agmcleod]

Are we not already doing that? Given we use requestAnimationFrame, and provide the me.timer for frame delta.

[obiot]

Most probably, but I just quickly had a look at it, and wanted to keep a reminder somewhere to have a look at it back later.

[Giwayume]

Currently it looks like both of the first two methods mentioned in the article are implemented, with a switch between the two by setting my.sys.interpolation.

So to implement a "prediction" solution, I'm guessing the API would look like:

me.sys.fps = 25; // Logic fps 
me.sys.interpolation = true; // I'm guessing this is optional.
me.sys.framePrediction = true; // New variable, whatever it's called. Turns on prediction.

For the prediction code, it looks like only me.Container and me.Sprite will have to change to utilize it? Are there other renderables that affect positioning frame-by-frame? It would be another IF statement in each drawing call. [EDIT]: Now that I think about it maybe the system can set a float from 0 to 1 each draw call that represents the time from the current to next frame that's always zero when framePrediction is off...

I something similar to MAX_FRAMESKIP mentioned in the article implemented today?

[obiot]

for me the only place this should be implemented is here :
https://github.com/melonjs/melonJS/blob/master/src/game.js#L248-L269
which is the main loop "update" function

or here :
https://github.com/melonjs/melonJS/blob/master/src/state/state.js#L190-L199
as this is the callback used by the RaF function

but other objects should stay as agnostic as possible from this (in my opinion) and rely only on the given dt information passed to the update function

[Giwayume]

I don't think it is possible to implement at that high of a level.

The idea is you're limiting the actual update logic to 25 fps, so each Renderable needs to "guess" its position based on the last "update" frame. So this is an extra calculation per Renderable.

But come to think of it the Renderable would need to know its velocity, which is a property attached to a Body. So the method mentioned in the article will only work for an Entity.

[obiot]

honestly, my personal intention was rather to fix the timestep as indicated in the following article:
http://caspervonb.com/archive/javascript-game-development-asynchronous-execution-loop/

but then it's more for me a user side implementation if something has to be done on the renderable/entity side ?

[Giwayume]

OK, so you are referring to updating multiple times per draw instead of frame prediction.

Can you answer does something like MAX_FRAMESKIP exist today? If not, I would like to open a separate issue on that.

[obiot]

well if you look at koonsolo article, it does not modify object position in the loop, but rather manage time/delta :

 const int TICKS_PER_SECOND = 25;
    const int SKIP_TICKS = 1000 / TICKS_PER_SECOND;
    const int MAX_FRAMESKIP = 5;

    DWORD next_game_tick = GetTickCount();
    int loops;
    float interpolation;

    bool game_is_running = true;
    while( game_is_running ) {

        loops = 0;
        while( GetTickCount() > next_game_tick && loops < MAX_FRAMESKIP) {
            update_game();

            next_game_tick += SKIP_TICKS;
            loops++;
        }

        interpolation = float( GetTickCount() + SKIP_TICKS - next_game_tick )
                        / float( SKIP_TICKS );
        display_game( interpolation );
    }

and then we do actually kind of already adjust position in terms of frame interpolation, although the main loop does not respect the above loop,
in melonJS :
https://github.com/melonjs/melonJS/blob/master/src/physics/body.js#L478
https://github.com/melonjs/melonJS/blob/master/src/physics/body.js#L452
and then "users" have to apply/use me.timer.tick as well (see the platformer example) :
https://github.com/melonjs/melonJS/blob/master/examples/platformer/js/entities/entities.js#L63

and no we do not support a MAX_FRAMESKIP mechanism, but this is however already covered by this ticket ?

[Giwayume]

If you don't want to separate it out that's fine. I just figured it could be separate because it's easy to implement and having MAX_TIMESKIP alone helps solve the problem of Entitys jumping through walls even if the multiple update call solution is not implemented in the near future.

I will try to tackle both, though.

[obiot]

just my point is that i thought (maybe wrongly) this could be addressed by implementing the main loop as described in the article (since it does take in account a maximum frameskip).

else we also do have this ticket with a similar issue/root cause : #16

[Giwayume]

So if we do the koonsolo article the users will have to override the Entity's draw() method, adjust the drawing position manually based on me.timer.tick and the Entity's velocity, call the _super() draw method, then revert the drawing position back.

I think the multiple update call method in the caspervonb article will work fine. I've heard of other Javascript gaming engines using it. Maybe we can implement both? Let the user decide if they want to create their own drawing prediction logic by overriding the draw() method, or provide a logic "stepSize" and let the engine call the update() multiple times based on that.

The game loop implementation can be swapped out similar to how me.CanvasRenderer and me.WebGLRenderer are swapped out.

[Giwayume]

The more I think about it the more I'm liking this swap out idea.

[obiot]

Talking about the renderer, i'm wondering if we should not move the RaF code into it, after all it would make sense to also have the renderer to control the main loop ? @parasyte what do you think?

[Giwayume]

I think traditionally, no. If you look at a library like pixi.js which is just one big renderer, you must provide your own game loop to call the renderer. It makes sense to me that they're separated out, because a renderer doesn't necessarily care that it's making an animation or not.

This is their simple example:

function animate() {
        // start the timer for the next animation loop
        requestAnimationFrame(animate);

        // "update" logic
        bunny.rotation += 0.01;

        // this is the main render call that makes pixi draw your container and its children.
        renderer.render(stage);
    }

[Giwayume]

So I made a quick implementation of the caspervonb article and it works great!

Even in Firefox where I have 20/60 fps when using the debugger, the movement is correct and very playable!

me.sys.interpolation must be false to use this method, because it is not compatible.

Code:
Giwayume@c60413e

[parasyte]

I'm going to agree with @Giwayume on this one. Another reason against it is that we have (at least one) use case with a temporary renderer to draw things like a minimap:

layer.draw(minimapRenderer);

Also the current implementation is closer to #383

[parasyte]

The multi-stepper is for accuracy (e.g. collisions), which is common in physics engines. Usually the physics simulation is stepped multiple times per frame to resolve multiple collisions with higher precision. The dewitter's game loop is kind of opposite; the idea is to save CPU time by running the updates at a lower frequency while drawing as often as necessary.

I understand the need for both. So I would like to solve this one in a way that's flexible enough to run the updates faster or slower, depending on the need of the developer.

[Giwayume]

If you look at my code snippit Giwayume@c60413e, I believe both can be achieved with this code.

If the developer prefers calling updates multiple times, simply set me.sys.fps to 60.

If the developer prefers to implement frame prediction in their draw events, set me.sys.fps to the target logic fps, say 25, then they can override the Entity's draw event with the frame prediction logic.

I'll provide an example of the latter and create a pull request.

[parasyte]

I am not talking about changing the FPS. The drawing should always be done as fast as the hardware allows (This is ideal, though nothing preventing a developer from intentionally lowering the FPS). The updates can be flexible in any direction, regardless.

[Giwayume]

In the engine as it currently is today, it always draws as fast as the hardware allows no matter what me.sys.fps is set to.

me.sys.fps isn't drawing fps, it's currently update fps. I was assuming that wasn't going to change.

So when I said change me.sys.fps to 25, this means you're changing the update fps to 25, but the entity's draw() function still runs as fast as possible... so if the user overrides Entity.draw() whatever prediction logic they put in there is run as fast as possible.

[parasyte]

The behavior you described is not possible, because the drawing is conditional on updates:

melonJS/src/game.js

Line 280 in b9e4794

if (isDirty) {

This if-statement limits drawing to the same frequency as updates.

[Giwayume]

Oh. Then it just needs to be taken out? Or did you have a different idea?

[Giwayume]

That isDirty variable isn't a big deal to get around. But I think you're suggesting having an updateFps variable in addition to fps. Is that correct?

me.sys.fps  // Max fps (draw and update)
me.sys.updateFps  // Max update fps

[parasyte]

Yes, the condition needs to be made ... conditional :p

I would probably call it "updates per second" instead of "update frames per second". But that's one option. Of course, the code will just use RAF as the reference frame rate, so we could theoretically scale updates and draws simultaneously, relative to RAF.

[parasyte]

Closed by #718