rsx: PS3 Native frame limiter improvements, add Infinite frame limiter #12052

elad335 · 2022-05-21T03:27:22Z

Do not wait on DEVICE 0x30 semaphore, it seems like it is something to do with queue command synchronization.

This also fixes cellGcmSetFlipWithWaitLabel which is built specifically to enable accurate RSX flipping time, its waiting command is confirmed to be placed AFTER DEVICE 0x30 waiting.

Fix default VSYNC state to be enabled. (and set it to be enabled in cellGcmSetVBlankFrequency and cellVideOutConfigure as well)
Add experimental "Infinite" frame limiter mode.
Fix spurious enabling of second vblank.

elad335 · 2022-05-21T07:03:33Z

I've added a new frame limiter called "Infinite" which basically imposes no limits whatsoever and sends an additional VBLANK each frame .
This is an extreme "Off" frame limiter type.
I've also added "Vblank" type which is the same behavior as before #12045.

kd-11 · 2022-05-21T07:11:53Z

I've added a new frame limiter called "Infinite" which basically imposes no limits whatsoever and sends an additional VBLANK each frame . This is an extreme "Off" frame limiter type. I've also added "Vblank" type which is the same behavior as before #12045.

We already know games are using vblank count for timing. This makes things worse imo.
The correct solution here is to not wait for the flip semaphore but to implement hardware triple buffering for RSX. A display queue is used and we check the semaphore during flip()

kd-11 · 2022-05-21T07:13:10Z

If the semaphore is not set and we have a frame to consume, go ahead and consume it. Hopefully by the time that frame is ready the old one will have been cleared

kd-11 · 2022-05-21T07:22:13Z

The display queue already exists by the way, it just needs some tweaks to bind it to a RSX semaphore. I'd like to extend this in future by using the native GPU's semaphore system which does a very similar thing so its important we get the architecture right.

solarmystic · 2022-05-21T08:27:23Z

With Framelimit set to VBLANK, performance is back to normal again. Frame times are also smoother.

Asinin3 · 2022-05-21T08:36:41Z

God of War 3 still has more spikes and slower than before when using auto framelimiter.

Before HW Framelimiter:

After:

After this fixup PR:

elad335 · 2022-05-21T09:05:05Z

I've switched the auto and vblank's demeanor, also please test the new "infinite" frame limiter which is supposed to be a replacement for upping the vblank rate for higher framerates in many games.

Asinin3 · 2022-05-21T09:40:54Z

Vblank limiter is worse now.

old commit:

New Commit:

Infinite in new commit:

1-3fps worse than Off for me.

elad335 · 2022-05-21T09:50:19Z

For clarification - off is the fastest option, infinite is supposed to be used with games that have an internal vblank-based frame limiter.

solarmystic · 2022-05-21T10:05:53Z

With these latest commits, God of War Ascension behaves as follows with each framelimiter option:-

Auto - frame time graph is not so spikey but performance is still lower compared to pre HW limiter behaviour

Vblank - the spikiest frametimes, and the least performant option.

Infinite - better than Vblank and Auto, and matches pre HW limiter behaviour in that scene.

Pre HW limiter build (0.0.22-13626) for comparison:-

rpcs3/Emu/RSX/rsx_methods.cpp

rpcs3/Emu/system_config.h

kd-11 · 2022-05-23T17:37:31Z

rpcs3/Emu/RSX/RSXThread.cpp

@@ -3258,6 +3275,11 @@ namespace rsx
 		flip_status = CELL_GCM_DISPLAY_FLIP_STATUS_DONE;
 		m_queued_flip.in_progress = false;

+		if (g_cfg.video.vblank_loop)
+		{
+			post_vblank_event(rsx::uclock());


This feels like it should just be part of "infinite" vblank mode and does not need the extra loop option

rpcs3/Emu/RSX/RSXThread.cpp

rpcs3/rpcs3qt/emu_settings.cpp

rpcs3/rpcs3qt/tooltips.h

kd-11 · 2022-06-22T17:07:23Z

rpcs3/Emu/Cell/lv2/sys_rsx.cpp

+		if (a5 == 1u)
+		{
+			// This function resets vsync state to enabled
+			render->requested_vsync = true;


Where is the counterpart to disable this? We know PS3 had engines that had tearing when fps dropped below a threshold (e.g cryengine 2)

I see sometimes in logs that the games reset it afterward and after cellVideoOutConfigure, I will add to it as well.

Hmm, it is unlikely that they are calling videoOutConfigure during gameplay as that is a heavy operation. It is more likely to be toggled using this method. Is it known what happens if a5 is 0?

I'm assuming that games also react to the unused CellVideoOutCallback. (maybe it's called when the display is changed)

It's controlled by CellGcmSetFlipMode. You can choose 3 modes there, 1 for vsync, 1 for scanline sync and the last mode for all out anarchy. We only need to set vsync to engage when the first mode is chosen.

I'm assuming that games also react to the unused CellVideoOutCallback. (maybe it's called when the display is changed)

I've checked and no game uses cellVideoOutRegisterCallback.

kd-11 · 2022-06-22T17:29:02Z

rpcs3/Emu/RSX/RSXThread.cpp

@@ -3251,18 +3259,25 @@ namespace rsx
 				}
 			}
 		}
-		else if (wait_for_flip_sema)
+		else if (frame_limit == frame_limit_type::_ps3)


If I'm reading this correctly the logic here is essentially "do not flip until a fresh vblank signal is received". You have the vblank signal as the flip release and this is an acquire operation here. Two problems:

There is still the correct event sent to us which you are now short-circuiting in semaphore-acquire. I'm not sure why this approach is to be considered better as it is essentially a hackier approach. Note that this "PS3 native" sync is to be used with games that have trouble with the normal PC-friendly framelimiter system, so extra bells and whistles are unneeded for this path. Is it that it doesn't work properly with HLE? I'm scratching my head on why I should prefer this version over the existing one which more closely follows what the PS3 does.

This whole set of variables and checks is not correctly labeled to match how display systems are architected with clients acquiring the display and the server releasing it in a cycle. Not a blocker by itself, but its not ideal and I would end up having to change it.

I'm quite confident DEVICE 0x30 is something to do with the queue command and not vblank, as it's inserted by gcm for all display sync options and occurs before semaphore wait by cellGcmSetFlipWithWaitLabel.
This function exists specifically to allow accurate timing of flipping and this is a usage example:

Here is label 0 is used, notice that it resides after DEVICE 0x30 waiting
Furthermore, flips are executed along with the vblank signal, not after, I guess that's why TLoU regressed by the original pr.

From the trace above, we can see there are 2 signals:
The only difference is that there are 2 cond vars that I see, one to ack the enqueue and another to wait for vertical sync.
Server:

label_0x30.wait_signal(); handle_enqueued_head(); label_0x30.reset()

Client:

enqueue(); label_0x30.signal(); label_0x30.wait_ack(); call(); <- CB reset?? label_0x0.wait_signal(); flip();

My guess here is that 0x0 is signaled by the hardware vblank system, 0x30 just acknowledges that a head has been enqueued. So, your PR addresses this by having the system use vblank instead of enqueue signal.
While this is fine, the second critique still holds. The implementation is convoluted and not clear at all.
My suggestion to solve this once and for all is to have a special kind of cond var with waiting for known values. Simply replace my rushed solution with the wait_for_flip_sema with proper condvar.wait() semantics to make the flow clearer.

For simplicity I have omitted the 0x10 logic, I have no idea where that signal is for.
For 0x30 handling, we have the frame enqueue logic in sys_rsx and rsx_thread::end_frame() for that.
For 0x0 we can signal from vblank thread.

No I have HLEd the game this is a cellGcmSetFlipWithWaitLabel call with 0 address. It doesn't appear in other games which uses other flip functions.

Which is named _cellGcmSetFlipCommandWithWaitLabel on firmware.

Here is an LLE flip without waiting on label.

So for "normal" flip the 0x0 var is absent? That simplifies the design then and makes the 0x30_ack a wait for vblank.
Again, my biggest problem with the way it is now is that the "documentation is the code" philosophy doesn't hold up because the design is all over the place. This isn't a new problem, but if it is now well understood, we have to do it properly and this PR already modifies the whole thing anyway. I only understand what is going on because I'm familiar with most of the code there.

0x30 has nothing to do with vblank, it appears in all sync modes even with VSYNC turned off and besides the whole purpose of _cellGcmSetFlipCommandWithWaitLabel is to time the flip very closely to what CELL decides. So if 0x30 is a vblank semaphore it defies all logic as to why the function exists in the first place.

It seems my point is getting lost in translation, that doesn't matter it's just an ack, whether its from vblank or not. I'm not happy with the implementation, but if we're being honest, you're unlikely to restructure it, so I'll merge and change it later when I have time.

* Do not wait on DEVICE 0x30 semaphore, it seems like it is something to do with queue command synchronization. - This also fixes cellGcmSetFlipWithWaitLabel which is built specifically to enable accurate RSX flipping time, its waiting command is confirmed to be placed **AFTER** DEVICE 0x30 waiting. * Fix default vsync state to be enabled. (and set it to enabled in cellGcmSetVBlankFrequency as well) * Add experimental "Infinite" frame limiter mode. * Fix spurious enabling of second vblank.

elad335 force-pushed the vblank branch from f562e6c to f246efe Compare May 21, 2022 03:49

elad335 mentioned this pull request May 21, 2022

[Regression] Reduced performance in various 60 FPS games since #12045 when rpcs3 frame limiter is enabled #12050

Closed

elad335 force-pushed the vblank branch 2 times, most recently from 3d9b80f to e640eb4 Compare May 21, 2022 08:54

kd-11 mentioned this pull request May 21, 2022

rsx: Deadlock avoidance of accurate RSX reservations #12048

Merged

elad335 force-pushed the vblank branch from e640eb4 to f3db2b5 Compare May 21, 2022 09:00

kd-11 mentioned this pull request May 21, 2022

rsx: Trivial non-blocking display synchronization #12055

Merged

Megamouse added the RSX label May 21, 2022

elad335 force-pushed the vblank branch from c878883 to 888da1a Compare May 21, 2022 16:40

elad335 changed the title ~~rsx: Redesign frame limiter~~ rsx: Implement frame semaphore May 21, 2022

elad335 changed the title ~~rsx: Implement frame semaphore~~ rsx: Implement frame semaphore (cellGcmSetWaitFlip) May 21, 2022

elad335 force-pushed the vblank branch from 888da1a to 3ce4735 Compare May 21, 2022 17:58

elad335 changed the title ~~rsx: Implement frame semaphore (cellGcmSetWaitFlip)~~ rsx: Implement frame semaphore May 22, 2022

kd-11 reviewed May 23, 2022

View reviewed changes

rpcs3/Emu/RSX/rsx_methods.cpp Outdated Show resolved Hide resolved

kd-11 reviewed May 23, 2022

View reviewed changes

rpcs3/Emu/system_config.h Outdated Show resolved Hide resolved

kd-11 reviewed May 23, 2022

View reviewed changes

rpcs3/Emu/RSX/RSXThread.cpp Outdated Show resolved Hide resolved

elad335 force-pushed the vblank branch from de05908 to 3ebc8e1 Compare June 21, 2022 17:24

elad335 requested a review from kd-11 June 21, 2022 17:25

elad335 changed the title ~~rsx: Implement frame semaphore~~ rsx: PS3 Native frame limiter improvements, add Infinite frame limiter Jun 21, 2022

elad335 marked this pull request as ready for review June 21, 2022 17:25

elad335 force-pushed the vblank branch from 4a2caef to 1971717 Compare June 21, 2022 17:41

Megamouse reviewed Jun 21, 2022

View reviewed changes

rpcs3/rpcs3qt/emu_settings.cpp Outdated Show resolved Hide resolved

Megamouse reviewed Jun 21, 2022

View reviewed changes

rpcs3/rpcs3qt/tooltips.h Outdated Show resolved Hide resolved

elad335 force-pushed the vblank branch from 1971717 to d6c1455 Compare June 21, 2022 17:49

Megamouse reviewed Jun 21, 2022

View reviewed changes

rpcs3/rpcs3qt/tooltips.h Show resolved Hide resolved

kd-11 reviewed Jun 22, 2022

View reviewed changes

elad335 force-pushed the vblank branch from d6c1455 to 688e66b Compare June 22, 2022 17:21

elad335 requested a review from kd-11 June 22, 2022 17:22

elad335 force-pushed the vblank branch from 688e66b to eb3bb28 Compare June 22, 2022 17:26

kd-11 reviewed Jun 22, 2022

View reviewed changes

elad335 requested a review from kd-11 June 23, 2022 06:39

elad335 force-pushed the vblank branch from 53d91c9 to 943d4ad Compare June 23, 2022 11:50

kd-11 approved these changes Jun 25, 2022

View reviewed changes

elad335 force-pushed the vblank branch from a6eeec9 to 6a8c39a Compare June 25, 2022 12:09

elad335 force-pushed the vblank branch 3 times, most recently from 23d662d to ffa4342 Compare June 25, 2022 12:29

rsx: Do not discard flip notifications

5deafde

elad335 force-pushed the vblank branch from ffa4342 to 5deafde Compare June 25, 2022 12:34

Megamouse merged commit 7422ab9 into RPCS3:master Jun 25, 2022

elad335 added a commit to elad335/rpcs3 that referenced this pull request Jun 25, 2022

rsx: Fixup after RPCS3#12052 for frame limiter off

81bef68

kd-11 pushed a commit that referenced this pull request Jun 25, 2022

rsx: Fixup after #12052 for frame limiter off

f5a55b3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rsx: PS3 Native frame limiter improvements, add Infinite frame limiter #12052

rsx: PS3 Native frame limiter improvements, add Infinite frame limiter #12052

elad335 commented May 21, 2022 •

edited

Loading

elad335 commented May 21, 2022 •

edited

Loading

kd-11 commented May 21, 2022

kd-11 commented May 21, 2022

kd-11 commented May 21, 2022

solarmystic commented May 21, 2022

Asinin3 commented May 21, 2022

elad335 commented May 21, 2022

Asinin3 commented May 21, 2022

elad335 commented May 21, 2022

solarmystic commented May 21, 2022 •

edited

Loading

kd-11 May 23, 2022

elad335 Jun 21, 2022

kd-11 Jun 22, 2022

elad335 Jun 22, 2022

kd-11 Jun 22, 2022

Megamouse Jun 22, 2022

kd-11 Jun 22, 2022

elad335 Jun 23, 2022

kd-11 Jun 22, 2022

elad335 Jun 23, 2022 •

edited

Loading

kd-11 Jun 25, 2022 •

edited

Loading

kd-11 Jun 25, 2022

elad335 Jun 25, 2022 •

edited

Loading

elad335 Jun 25, 2022 •

edited

Loading

elad335 Jun 25, 2022

kd-11 Jun 25, 2022

elad335 Jun 25, 2022

kd-11 Jun 25, 2022

rsx: PS3 Native frame limiter improvements, add Infinite frame limiter #12052

rsx: PS3 Native frame limiter improvements, add Infinite frame limiter #12052

Conversation

elad335 commented May 21, 2022 • edited Loading

elad335 commented May 21, 2022 • edited Loading

kd-11 commented May 21, 2022

kd-11 commented May 21, 2022

kd-11 commented May 21, 2022

solarmystic commented May 21, 2022

Asinin3 commented May 21, 2022

elad335 commented May 21, 2022

Asinin3 commented May 21, 2022

elad335 commented May 21, 2022

solarmystic commented May 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elad335 Jun 23, 2022 • edited Loading

Choose a reason for hiding this comment

kd-11 Jun 25, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elad335 Jun 25, 2022 • edited Loading

Choose a reason for hiding this comment

elad335 Jun 25, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elad335 commented May 21, 2022 •

edited

Loading

elad335 commented May 21, 2022 •

edited

Loading

solarmystic commented May 21, 2022 •

edited

Loading

elad335 Jun 23, 2022 •

edited

Loading

kd-11 Jun 25, 2022 •

edited

Loading

elad335 Jun 25, 2022 •

edited

Loading

elad335 Jun 25, 2022 •

edited

Loading