SPU LLVM: Improve runtime SPU compilation preferences

[elad335]

1. Prefer using inactive worker threads to ompile new SPU blocks for maximum concurrency.
2. Postpone thread notifications to when block queue is drained so all threads would start at once, not delaying the managing thread to push more blocks.
3. If more than one block has the same use count as others, apply for compilation first the one which has been queued earlier. Similarly to distance / time = speed, here the time here is compared (estimation) for rate of block usage.

Some of these affect only CPUs with 12 or more threads at the current implementation, point 3 affects all.

Tests: Tested to improve massively the performance of SPU LLVM ingame compilation for Red Dead Redemption on 14600KF

[Megamouse]

Instead of just making changes that supposedly are faster, how about adding some benchmarks for once, so people can actually grasp the benefits when seeing auch a PR?

[Megamouse]

Wouldn't this reduce instructions if it was behind if constexpr Notify?

[elad335]

Not really, compilers detect that Notify is known. if-constexpr is not technically for optimizations, but to avoid compiling code that would otherwise not compile for template functions.

[elad335]

Instead of just making changes that supposedly are faster, how about adding some benchmarks for once, so people can actually grasp the benefits when seeing auch a PR?

Need someone with 12 or more threads to test this on empty SPU cache.

[elad335]

I guess I'll also make it possible to test its performance somewhat accurately mid-game, because the only way the user known now is how much stuttery the game is while compiling.

[readywer]

Instead of just making changes that supposedly are faster, how about adding some benchmarks for once, so people can actually grasp the benefits when seeing auch a PR?

Need someone with 12 or more threads to test this on empty SPU cache.

I have a 13600k later today i will try to compere the performace to master in some games(GT5,6, R&C games).

[elad335]

keep in mind it's not an FPS test, it's to test how relatively long SPU compilation takes (those green "compiled block successfully messages")

[EmulationChannel]

@elad335 I tested 14600KF 14 /20 THE LAST OF US and RED DEAD REDEMPTION very quickly SPU CACHE