High Input Lag on 3D games

[Dogway]

PCSX2 version:
pcsx2-v1.5.0-dev-3138-gfbafd4420-windows-x86

PCSX2 options:

Software or Hardware mode indistinctly. Using Safe Preset

Plugins used:
Default:
GSdx32-AVX2
Spu2-X
LilyPad 0.12.1
FWNull Driver 0.7.0

Description of the issue:
There's a high game input lag in 3D based games, rarely below 6 frames of lag. Here are some tests done with frame advance using the minimum recorded frames for fastest event paths.

MGS3 Subsistance (6 frames)
GTA Vice City (6 frames)
GTA San Andreas (10 frames, 166ms!)
Sengoku Basara X (6 frames)
Tokyo Xtreme Racer 3 (7 frames)
Taiko Drum Master (6 frames)
Silent Scope (6 frames)
Time Crisis 3 (5 frames)
Capcom Fighting Evolution (4 frames)

Generally it's 6 frames of game input lag which is one of the highest recorded on current emulation software compared to Dolphin (5), Citra (4), Retroarch (4-5) and possibly Cemu and RPCS3 (they feel snappy)

How to reproduce the issue:

-Enable "Recording Tools"
-Load a game
-Press the frame advance hotkey
-Press a fast event triggering button on your pad
-Press again the frame advance hotkey as many times (counting them) until the event starts the animation

PC specifications:
Operating System: Windows 7 x64 SP1
CPU: Intel i5-4670K @ 4.1Ghz
GPU: Nvidia GeForce GTX 1070

[Jacoby1218]

I’ve noticed that too, what controller are you using?

[Dogway]

I use a 360 wireless controller but this is unrelated because I'm only measuring the emulation path input lag through the frame advance procedure. I guess judging by the picture you might already know but here is a good review of controller input lag.

[KrossX]

Is there data on PS2 + CRT input lag? It would be nice to have some, specially for GTA-SA. Might be tricky though as you could need a light in the controller for when a button is pressed and when pressure is greater than deadzone.

[tadanokojin]

Honestly I have a lot of problems with these numbers. Not the least of which is I can't independently verify any of them.

[KrossX]

You need a high speed camera and a controller that has a light on it.

3rd Strike 4.7
Dodonpachi DaiOuJou (White) 3

Dodonpachi seems like a good test point.

[tadanokojin]

I mean even barring a comparison to a control (which they don't provide), they aren't explicitly telling me where these numbers come from.

Here are some tests done with frame advance using the minimum recorded frames for fastest event paths.

What is "fastest event path" and how is it determined to be the fastest?

In DC2 I get numbers ranging from 4 to move the start menu cursor to 10 frames to swing a sword. A lot of which could probably be explained by animation blending and easing or natural delay introduced by the developers.

In Raw Danger! It's about 11 frames to move the cursor in the start menu, but I happen to know that game runs at 20fps internally.

Dolphin (5), Citra (4), Retroarch (4-5) and possibly Cemu and RPCS3 (they feel snappy)

Even if I granted this comparison made sense (I don't think it does). Where are these numbers even coming from?

[QuaiGoner]

In Flatout games input lag almost unplayable (fullspeed)

[Dogway]

What is "fastest event path" and how is it determined to be the fastest?

Do an array of tests in a game and take "the fastest" of the events. As you said in your DC2 example between the 4 and 10 frame lag take the fastest/shorter, 4. When you test enough games and events you start to paint a clear picture. I also think an input lag test with a real PS2 on a CRT should be made to clearly define the lag introduced through emulation, mind you the PS2 controller will likely introduce half a frame of input lag. Unfortunately I don't own a 60fps camera.

Even if I granted this comparison made sense (I don't think it does). Where are these numbers even coming from?

To be as unbiased as possible all tests were performed as explained on the issue description, with TAS tools using frame advance. Cemu and RPCS3 lack these tools.
While I also think PS2 2D games have at least 1 frame of game/emulation input lag, on 3D games it's specially jarring. When you add 6 frames to an extra frame for LCD and another one for the controller (at least), we are talking about 8 frames (133ms) minimum of input lag.

Edit: Found an input lag test with a PS2 3D game on a CRT. (3.86 frames of lag, accounting controller)

[tadanokojin]

As you said in your DC2 example between the 4 and 10 frame lag take the fastest/shorter, 4.

Yeah but there are thousands of things to test. I'm asking you for the specific things that gave you the number. We need to be able to reproduce it.

I also think an input lag test with a real PS2 on a CRT should be made to clearly define the lag introduced through emulation

I agree. That information would be useful.

To be as unbiased as possible all tests were performed as explained on the issue description, with TAS tools using frame advance. Cemu and RPCS3 lack these tools.

What games?
What did you test?
Where can I read the results?
What's your sample size for the other emulators?
Do we even need this comparison?

You're worried about bias but your sample size for PCSX2 is 9 games which I don't think is representative.

[Dogway]

I'm asking you for the specific things that gave you the number. We need to be able to reproduce it.

I test the fastest in gameplay, either jump, punch, turn (if digital input), etc. Eventually I also test Start to check it aligns to the measured minimums. These aspects are not usually given in tests because it's asummed you measured enough events and times, but if you need a specific event to reproduce I will post them later on.

I agree. That information would be useful.

I linked a video on my edit to a test with a real PS2, I didn't find what the display type was but in case it was an LCD I assume it to be one with zero lag. For reference Mega Man X8 (a 3D game) has an input lag of 64.459 (3.867 frames)

What games?
What did you test?
...

An average of an array of games similar to the one presented here. I test enough games until I find a pattern, obviously I'm not going to test the whole library but it's wise to pick games developed with the PS2 on mind, suggest games and I will try to test them. I did a study on input lag for 3D systems in retroarch that you can find on the link on top. As you can see they vary from 3 to 5 frames.

Now that we have a few comparison samples of real measurings on real hardware (Mega Man X8, Mega Man X7, Dodonpachi DaiOuJou (White)...), maybe we could run specific comparisons on PCSX2, but as I said, the events used for these measurings were not redacted so it depends on if you find them valid tests and/or valid games.

[sandman332]

Can confirm this, makes many first person shooter games unplayable.

[tadanokojin]

@Dogway

but if you need a specific event to reproduce I will post them later on.

I'd like to know what you are testing, yes, so others can test it an eliminate any external factors.
You can also post some block dumps so others can test.

In the meantime you can try disabling MTVU if it isn't already and seeing if that reduces latency any.

An average of an array of games similar to the one presented here.

Honestly just lose the comparison, you clearly can't or won't give me any details about it and frankly I don't think it would tell us much anyway.

Feel free to test other games in PCSX2, though. Jak series might be interesting.

maybe we could run specific comparisons on PCSX2

I agree. A good comparison against real hw would give us a delta and help us understand how much of the lag is emulation overhead.

[willkuer]

Wouldn’t it make more sense to check something the in-game button press memory address and check how many frames it takes to register the button press event?

Also wouldn’t you expect that the emulation overhead is game-independent as long as it’s not bottlenecked by the user’s system?

[Dogway]

Did more tests, MTVu has no effect, neither backend except for Yakuza.
I start invoking the events from the first playable scene in the game, then I test several events including Start and moving through the Start options to rule out intrinsic animation delays.

Testing through frame advance in Windowed Mode (OpenGl SW and HW, hyphen - means random not backend). If you need the profiles, let me know.

5     (Jump) (SW/HW) Jak and Daxter - The Precursor Legacy (USA)

4-5   (Jump) (SW/HW) Mega Man X8 (PAL)

4     (Jump) (SW/HW) God of War (PAL)

4     (Jump) (HW)    God of War II (PAL)

6     (Punch/Square)    (HW) Yakuza (USA)
6     (Start Menu Move) (HW) Yakuza (USA)
7     (Move/Start)      (HW) Yakuza (USA)
11    (Punch/Square)    (SW) Yakuza (USA)
11    (Start Options)   (SW) Yakuza (USA)
12-13 (Move/Start)      (SW) Yakuza (USA)

6     (Start Options)     (SW/HW) Gran Turismo 4 (USA)
5     (Brake/Camera View) (SW/HW) Gran Turismo 4 (USA)

7-8   (Jump)        (SW/HW) Kingdom Hearts (PAL)
6     (Start)       (SW/HW) Kingdom Hearts (PAL)
5     (Start Items) (SW/HW) Kingdom Hearts (PAL)

9-10  (Start/Start Options/Jump) (SW/HW) Ico (PAL)
15    (Move)                     (SW/HW) Ico (PAL)

4     (Start)     (HW) Monster Hunter (PAL)
4     (Move/Turn) (HW) Monster Hunter (PAL)
5     (Start)     (SW) Monster Hunter (PAL)
4-5   (Move/Turn) (SW) Monster Hunter (PAL)

5     (Start)        (SW/HW) Tekken 5 (USA)
6     (Cross/Square) (SW/HW) Tekken 5 (USA)

4     (Start)              (SW/HW) Grand Theft Auto III (PAL)
5     (Car Brake -Square-) (SW/HW) Grand Theft Auto III (PAL)
6     (Jump -Square-)      (SW/HW) Grand Theft Auto III (PAL)

10    (Start/Start Options/Move Turn/Car Brake) (SW/HW) Grand Theft Auto - San Andreas (PAL)
14    (Jump)                                    (SW/HW) Grand Theft Auto - San Andreas (PAL)

10-11 (Shoot/Crouch)        (SW/HW) Black (PAL)
12-13 (Start/Start Options) (SW/HW) Black (PAL)

[water111]

The input latency is confusing. I measured Jak 1's input latency with a light on a controller and a high speed camera.

On PS2, I would expect around 5 frames:

• 0 : button pushed
• 1 : IOP reads controller (once per frame)
• 2 : EE receives IOP DMA with button press and calls ja-push
• 3 : animation system runs, puts animation frame in next frame's DMA list
• 4 : VU processes DMA list and writes to frame buffer
• 5 : frame with animation is visible

With the camera, I measured between 5 and 6 frames of latency (usually 6) on a PS2 with a CRT TV.

On PCSX2, I would expect another frame to read the controller, and probably a few more on graphics.
But I got between 11 and 18 frames of latency. (11,13,14,12,13,17,13,18,11,15)

I am not sure where the huge latency is coming from, but I did find two interesting things:

1). In PCSX2, some game frames are just not shown. Even if the game is internally running at 60 fps and PCSX2 is running full speed, it will just skip frames at random. I found an animation which is 17 frames, and plays as frame 1,2,3,....,17 on a real PS2. In PCSX2, some frames a just skipped, so you saw 1,1,3,4,5,6,6,8,9,9,11,12,13,13,13,16,17. Watch this video and see how it is choppy in the second half https://www.youtube.com/watch?v=IMjHCIwb7hc, which matches the high speed video of the screen. The game is running at 60 fps internally here, or the numbers at the top would turn red.

2). The EE/VU timing is very off, so the games may sometimes be running at a different frame rate on PCSX2 and PS2. Jak mostly runs at the right frame rate, but in certain areas, it internally runs at 30 fps on PCSX2 where PS2 runs at 60 fps. I think this is because some EE code takes around 2x more cycles than it should. See the game profiler here when the game goes at 30 fps in PCSX2: https://imgur.com/a/WwBHpY8 . The upper bar is EE, and the yellow/white sections are an EE-based renderer. This fits within 100% on PS2, but is over 120% on PCSX2. The lower bar is VU1, which has around 2x higher utilization on PS2.

[tadanokojin]

I found an animation which is 17 frames, and plays as frame 1,2,3,....,17 on a real PS2. In PCSX2, some frames a just skipped, so you saw 1,1,3,4,5,6,6,8,9,9,11,12,13,13,13,16,17.

Interesting. Just spitballing but I wonder if the alt vsync PR addresses this any,

[tadanokojin]

I ran some tests from @Dogway's results. I ran these tests by buffering the input for 2 frames. I'm including that buffering in my results. All tests are NTSC-U.

In GTA III I tested the pause menu. I found it was 4 or 5 frames before the background appears. I did note however that the audio cue happens on the 3rd frame.
https://youtu.be/_KJMuUMM5T8

In GT4 I got about 6 frames for the audio cue and the menu begins to appear on the 7th. Exiting it I'm noticing it's only 3 frames for the audio cue.
https://youtu.be/FC2Y6aQakjw

In Ico, which was the one I was most interested to test because of the high number. I tested it by holding the analog stick in the direction opposite where the character was facing and waiting for the character to move. The result I got was 8 frames. Which is a little more than half what Dogway is reporting. Not sure if this is due to differences in regions or if Dogway tested a different move animation.
https://youtu.be/hW15-DHc_a8

I got 10 frames for the start menu to appear.
https://youtu.be/myE6cU5hZ1E

[sandman332]

So, are there any band-aid fixes or anything that can be done from the users end to somewhat fix this issue? Makes all the games I have played almost unplayable and some definetly are.

[mirh]

Well, for starters, mods could have not deleted my comment referencing the stuttering problems of the nvidia driver #2307. Considering we have yet to solve that hot potato, I'm not sure we really can take at face values results from geforces (unless under linux)

I wonder what best case input lag of the emulator could be then? Say, with unlocked framerate and in the bios menu.

[ramapcsx2]

The upper bar is EE, and the yellow/white sections are an EE-based renderer. This fits within 100% on PS2, but is over 120% on PCSX2. The lower bar is VU1, which has around 2x higher utilization on PS2.

@water111
Could you share how you got into that performance counter debug mode?
It's probably using hardware that PCSX2 doesn't fully / correctly emulate, so I wouldn't trust it too much, but I'd like to try this with modified VRender / VBlank timing.
Maybe it makes a difference.

[water111]

These performance bars should work correctly for EE - they are just based on the timers which PCSX2 does correctly. Other performance menus aren't emulated correctly and display all zeros, I think because PCSX2 doesn't support the EE performance counter registers. The VU bar is a little weird because of how PCSX2 handles VU1 timings.

Here's a link to a patch that you can apply to PCSX2 to get into this mode. It's a bit of hack because it needs to emulate a dev kit with extra RAM and patch several memory values at specific times.

https://gist.github.com/water111/e4a0af85968daad1ac478573ae07b48c

(I think it also changes the vsync timing, so maybe you want to remove this)

If you boot Jak and daxter NTSC/PAL/Japan version, it should start up in the debug mode. Then press L3 to open the menu, then click Display, then profile.

I am fairly certain this isn't a vsync timing issue, but instead an EE timing issue. Most of the time is spent in a renderer where almost every instruction can be dual issued.

[ABU3BGE]

pcsx2-inputlagfinal.zip

Alright, last build, im hoping it is the same performance as inputlag2, you will need to go to Config->Emulator Settings->GS and make sure "Vsyncs in MTGS Queue" is set to 0

If all is good, I'll PR this and call it there for now.

THANKS THERE IS A LITTLE IMPROVMENT I TESTED SHINOBIDOU AND SC AND TRY 2D GAME NEOGEOBC TO SEE AND IT IMROVED A LOT FOR 2D ALMOST NO INPUTLAG IN 2D BTW IS THIS THE LATEST VER OF PCSX2 ?
AND WHAT DID YOU DO TO THE PROGRAM ?

[refractionpcsx2]

Glad to hear it seems to have gotten rid of it. So judging from what you guys are saying, I'm probably good to close this issue once the PR is merged?

and im not sure if that was the 100% latest build, I think I was accidentally using a branch, if you want the latest version with the changes use the PR build (PR linked above)

[Immersion95]

Glad to hear it seems to have gotten rid of it. So judging from what you guys are saying, I'm probably good to close this issue once the PR is merged?

and im not sure if that was the 100% latest build, I think I was accidentally using a branch, if you want the latest version with the changes use the PR build (PR linked above)

I think it can be closed when this PR is merged and also when the aspect ratio triggering it is fixed.

Do we still need "EnableVsyncWindowFlag=enabled" to enforce the Pseudo "FSE" state ?

[refractionpcsx2]

I don't know if that will work on master, probably not, and I'm unsure when that PR for the Flip Model stuff will be merged, as that is not my PR, I was just asked to bundle it in with these test builds to see if it helped any.

I can ask the OP of the Flip Model PR to rebase once my PR is merged so you can get a build from there if you want (or you can keep the inputlagfinal build i guess, not sure if that's missing some fixes from master) until the Flip Model PR is merged, that's the best I can offer for the moment.

[ABU3BGE]

what do you mean by pr ?

[refractionpcsx2]

PR = Pull Request, basically a set of changes to put in to the program, you can find it by clicking on this number #4115 And there will be links for builds at the bottom (referred to as checks) which you can download.

[ABU3BGE]

thanks btw are they going to put what you did in future update ?

[refractionpcsx2]

well yes, because I'm going to merge it :P

if you mean progress report? more than likely in the Q1 2021 report.

[mvdhout1992]

If you want to keep this open I can test a few other PS2 games.

Thanks again for all the effort you guys are putting into development, amazing emulator for a few tough to emulate system.

[refractionpcsx2]

No problem, I just wanted to get this issue sorted because to be frank, it's kinda driving us nuts with people reporting it and nobody doing high speed footage to show the lag differences (except water111 who did some), so at this point we've just tried to look at potential sources of lag and hope you guys who feel it can "feel" a difference.

Yes I'm throwing s**t at the wall and seeing what sticks :P

[ABU3BGE]

i tested mgs 3 and way of samurai 2 there is no difference but devil summoner 2 and gtasa and armored core 3 and onimusha 3 there is little difference and btw i have game fixes disable because it effect inputlag to me

[refractionpcsx2]

btw i have game fixes disable because it effect inputlag to me

That makes absolutely no sense... I think you're feeling placebo there.

[ABU3BGE]

no for real i have it disabled for long time because it caused high inputlag

[refractionpcsx2]

I don't understand why that would be a thing, unless there is a patch the only thing it does is changes a few settings when the game boots, which the game should need.

if it's doing more than that then we have a problem.

Do the games you disable it for show they have fixes/patches on the top of the console/log window?

[ABU3BGE]

no like shinobidu and other game i forgot what it is so not all the games have that problem with gamefix

[ABU3BGE]

it only have this in shinobidu
Patches: No CRC found, using 00000000 instead

[refractionpcsx2]

That only happens when you full boot and it's going through the BIOS procedure, that will change to your games CRC later on.

[mvdhout1992]

The fixed zip file also fixes input lag in the GTA games for me.

[DukeSkinny]

Agreed. There is a significant improvement as compared to a build from late last summer. Movement in 'Okage: Shadow King' is now near-instantaneous for me.

[Skalt711]

Hello.

The build in pcsx2-inputlagfinal.zip does reduce input lag for me when I play Gradius III and IV bundle and Dodonpachi Daioujou. I didn't test any 3D games if Gradius IV from the bundle doesn't count.

[refractionpcsx2]

Thanks for testing guys, the changes in pcsx2-inputlagfinal are now in master, so no need to test that anymore :)

[ABU3BGE]

by master you mean the nightly build ?

[refractionpcsx2]

Yes

[ABU3BGE]

thanks thats good news

[limyz]

@refractionpcsx2 I tested the latest nightly build with GuitarFreaks and verified the input latency is much significantly reduced with a custom controller using rawinput. I probably see if a stream is possible in the coming week.

[Qodify]

@refractionpcsx2 I tested the latest nightly build with GuitarFreaks and verified the input latency is much significantly reduced with a custom controller using rawinput. I probably see if a stream is possible in the coming week.

Tested on Ratchet & Clank Deadlocked (fast-paced 3rd person platformer/shooter) and works consistently. I hope this gets in stable.

[MrCK1]

@Qodify Once the next release is ready, all the changes in 1.7 dev will be rolled into 1.8 stable :)

[i-am-nut]

@refractionpcsx2 I tested the latest nightly build with GuitarFreaks and verified the input latency is much significantly reduced with a custom controller using rawinput. I probably see if a stream is possible in the coming week.

@refractionpcsx2 could you please elaborate how to set a custom controller using rawinput? i'm trying here but not managing :(

[MrCK1]

Closing this due to improvements in 1.7 that have eliminated this issue