Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rounding error (maybe) in levelup cost calculator? #97

Open
UguuUguu opened this issue Jun 3, 2021 · 1 comment · May be fixed by #115 or #161
Open

Rounding error (maybe) in levelup cost calculator? #97

UguuUguu opened this issue Jun 3, 2021 · 1 comment · May be fixed by #115 or #161

Comments

@UguuUguu
Copy link

UguuUguu commented Jun 3, 2021

I don't know what could possibly be causing this.

I attempted to calculate the cost to raise a six-star operator from elite 2, level 37, to elite 2, level 69, with a base of 93 Currency Experience. If the on-hand LMD is entered at 83,908 or less, it displays a correct subtraction. If the on-hand LMD is 83,909 or higher, it seems to have some kind of rounding error, displaying the cost at (for example)
131071.70000000001 = 214980.7 - 83909
I've attached a screenshot of the error, and the settings that caused it. Error occurs in both Firefox and Chrome.
image

@treveleon
Copy link

According to the values in aklevel.json, The values for each level cost are all whole integers.

The issue is being encountered because Current Exp = 93.

When the site gets to these ternaries:

let es = cl === maxLevel[star - 1][cev] ? 0 : charExpMap[cev][cl - 1] - ce;
let cs = cl === maxLevel[star - 1][cev] ? 0 : charUpCostMap[cev][cl - 1] * es / (charExpMap[cev][cl - 1]);

A decimal is introduced because of the offset to the experience and then it has potential to create floating point math issues in the output.

Likely the easiest workaround would be to force single point output in the display variable gn like is already done for gs. (Embarrassingly I almost exclusively use GitLab so I'm unfamiliar with how this repo flow works or I'd make the commit myself) 😅

@op there's a lot of really great resources out there RE floating point math concerns like this. It's pretty handy to know from time to time.

BryghtShadow added a commit to BryghtShadow/AN-EN-Tags that referenced this issue Dec 12, 2021
Resolves Aceship#97 by rounding LMD cost, and remove unnecessary `toFixed(1)`.
@BryghtShadow BryghtShadow linked a pull request Dec 12, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants