Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use accuracy instead of score for mania pp calculation #18749

Merged
merged 8 commits into from
Jun 29, 2022

Conversation

molneya
Copy link
Contributor

@molneya molneya commented Jun 17, 2022

As lazer is using a new scoring system, it doesn't make sense to use score as an attribute to calculate pp. This PR makes pp derive from the score hit judgements using a custom accuracy formula instead. This can allow for future changes if we need to weight some judgements higher than others for balancing purposes.

The new curve for scaling accuracy: https://www.desmos.com/calculator/ymgamiinki

Note that accuracy pp is removed entirely. Once more ideas are put forth we can consider re-implementing it again with something that makes a bit more sense.

After analysis of the pp changes shown in smoogi's spreadsheet, we decided to give a weight of -100 to misses. This should lower the prominence of mashed/low accuracy scores, which usually have high miss counts. This was accounted for in stable score with HitPunishment. See Evening's comment for the analysis of this change. While results look promising with a different weighting for misses, for this PR to act as a framework for future changes it would be better to have this in a state that users expect (that is to correlate with accuracy that they are used to seeing with scoreV2).

Feedback from players revealed that NoFail could be considered a massive game changer on certain difficult to pass maps. This has been reduced to 0.75x so players cannot abuse those maps as much.


Sheet containing all scores: https://docs.google.com/spreadsheets/d/1jMgTrQ8HxfKRadOsyNv3h7AsdHGAJ4F7I85BBBkQACs/edit#gid=1165396648
Sheet containing only scores which already have pp: https://docs.google.com/spreadsheets/d/1CQOsjhC3StyaJJZLpGqCGP7AQoNZh3cYuOYFm49Infs/edit#gid=1165396648

As of 984634c

@abraker95
Copy link

abraker95 commented Jun 17, 2022

Can we get some comparisons between nomod, EZ, and HT, and EZDT, and NFDT? Removing the score multiplier stuff might make PP for some of those score rather high. iirc stuff like <500k score giving 0 PP was a bandaid fix for stuff like NFDT giving way too much PP.

I misunderstood the change. With this only scores 80% acc and above will receive PP, which approximately emulates previous behavior using acc instead of score.

@Kominaru
Copy link

Good stuff

We obviously need to tweak the curve (optimal is unlikely to be linear), but this is already a huge step forward

@smoogipoo
Copy link
Contributor

smoogipoo commented Jun 18, 2022

I'll provide two sheets here, since this change makes it so that pp is awarded regardless of total score (as it should be...).

Edit: See OP.

@smoogipoo smoogipoo requested a review from a team June 18, 2022 03:08
@Eve-ning
Copy link
Member

@smoogipoo is it possible to include SR & Accuracy within the sheet?

@smoogipoo
Copy link
Contributor

smoogipoo commented Jun 18, 2022

It's a little bit involved to do so it'll take me a while, make do with the above for now.

@Eve-ning
Copy link
Member

It's a little bit involved to do so it'll take me a while, make do with the above for now.

Sure, I'll try it on my end, I'll let you know if it's possible from here, carry on with other things

@Eve-ning
Copy link
Member

Eve-ning commented Jun 18, 2022

Results

This is the %pp comparison.

A point at (0.95, 0.8) implies at 95% Acc, you get 80% Max PP

Current System vs. Proposed System

image

Delta: Proposed System - Current System

image

Analysis

1 Removal of bonus consistency:

  • The current system derives %pp from Score; Score is influenced by Accuracy and Combo
  • The deviation from the linear path is caused by Combo Scoring
  • As we depend on accuracy, %pp will not be affected by consistency

2 Buff of better scores:

  • Observe the orange line ~100% Acc intersecting higher than the rest
  • This implies an (on average) increase in %pp gained

3 Nerf of scores < 85%:

  • All scores below that threshold will be nerfed as observed.
  • This is amplified by setting a Miss as -100

Annex

@smoogipoo
Copy link
Contributor

smoogipoo commented Jun 22, 2022

Have you considered scaling the NF multiplier based on how badly the user performed? For example in osu! it scales by the number of misses:

if (score.Mods.Any(m => m is OsuModNoFail))
multiplier *= Math.Max(0.90, 1.0 - 0.02 * effectiveMissCount);

Otherwise, I'm not sure if you're going to run into the same "sandbagging" issues as in osu! where players would play with NF attached 100% of the time and qualify for lower ranked tournaments (75% reduction is pretty big). That being said, I also don't know if rank-restricted tournaments exist in mania and if not - why.

Related Discord discussion: https://discord.com/channels/546120878908506119/917623675958747146/987635123728039946

tl;dr: This isn't currently used for sandbagging in mania, and there's other ways to sandbag due to a different culture. We'll go with this multiplier for now since it's better than what's currently in master anyway - NF reduces score which has an even more drastic effect on PP.

@RushFTK
Copy link

RushFTK commented Jun 27, 2022

might have some misunderstanding, what's influence about some mod such as HardRock?

@Eve-ning
Copy link
Member

might have some misunderstanding, what's influence about some mod such as HardRock?

mania doesn't use hr pp

@smoogipoo smoogipoo merged commit fd0a1e2 into ppy:master Jun 29, 2022
This was referenced Jun 29, 2022
@molneya molneya deleted the accuracy-pp-curve branch November 11, 2023 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants