Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Occasional test failures in ranged_test.cpp #22064
Game version: Everything since 41e8209
Operating system: All that run tests.
Tiles or curses: N/A
Mods active: N/A
All tests pass consistently.
Very occasional (so far 3 / 270 executions in travis) failures in either the "unskilled shooter with an inaccurate pistol" or "unskilled shooter with an inaccurate smg" tests.
Steps to reproduce the behavior
Run ranged unit tests repeatedly.
Most likely fix is to back out or adjust the penultimate balance tweak from #21468 (07b94a2), I had run the tests many times before that adjustment with no failures, and I probably just cut it too close.
Essentially I dropped the multiplier on the DEX penalty a tiny bit and increased the multiplier on the PER penalty a similarly small amount, just reversing this will probably fix the immediate problem, but some compromise value may also work.
Quick outline of the feedback from the test:
This is the name of the test section that failed, we spawn an "unskilled shooter" and give them an "inaccurate smg", then have them fire at a target at a specified range.
This is the actual test results, we expect the average of "fast_stats_upper" to be lower than 0.6, but it isn't. This test is checking the hit rate of a "quickdraw", but it's failing because it's hitting too often.
This gives the basic outline of the test,
Max aim speed and min aim speed give feedback about how fast the player is aiming, if min aim speed is above 0 it means they haven't aimed fully, so they are constrained by the time provided to aim.
Some misc stats for troubleshooting, none are particularly pertinent here.
This is detail about the test results, it sampled 100 shots and estimates that there is 0.15 error for the lower test bound, and it sampled 144 times and estimates that there is 0.12 error for the upper bound (the one that failed).
What we can see from this is that the adjustment to PER had no effect on this test, since it isn't even close to full aim. Reverting the DEX multiplier back to its previous value would result in that penalty being 30 instead of 27 (not a particularly large value, that might not be enough TBH). Alternately some other change that increases the effective dispersion of this scenario without breaking other tests would resolve the issue.
As a last resort, we can bless the results and loosen the tests, but I don't think that's a good option.
Going for fully normal distribution could allow getting rid of RNG altogether.
Natural distributions are a bit too realistic at times and could make aiming at distant targets too easy and aiming at targets close up too hard, but those aren't huge errors. Plus, they would indirectly fix current lack of sniping at lower levels that way.