Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial rng replace #2084

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Conversation

qndel
Copy link
Member

@qndel qndel commented May 30, 2021

Replaces crappy vanilla rng with a decent one for everything that doesn't break save compatibility (At least I hope so :D)

@qndel qndel added the enhancement New feature or request label May 30, 2021
Source/engine.cpp Outdated Show resolved Hide resolved

int32_t GetRndSeedV2()
{
std::uniform_int_distribution<int32_t> dist(0, INT_MAX);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the default constructor here has the same effect, creating a distribution with a lower bound of 0 and upper bound of std::numeric_limits<int32_t>::max().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dunno, I tested and the results were off by 1

{
if (v <= 0)
return 0;
return GetRndSeedV2() % v;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given you're using std::uniform_int_distribution why not use distribution params with v-1 as the max? This would also let you avoid modulo bias, fitting into the stated goal of improving randomness. v-1 would be necessary as std::uniform_int_distribution uses a closed interval but the existing GenerateRnd function uses a half-open interval.

You could also simplify call sites that use a lower bound (weapon damage rolls for example) by providing an overload with a minimum bound parameter. If dist was created and stored alongside rng with the default distribution then something like the following would be suitable for bounded random number generation:

int32_t GenerateRndV2(int32_t max, int32_t min)
{
    if (max <= min)
        return min;

    return dist(rng, std::uniform_int_distribution<int32_t>::param_type{ min, max });
}

These callers currently add 1 to the argument to account for GenerateRnd's half-open interval, an approach like this would make that no longer necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not the goal of this PR

@@ -2324,19 +2325,19 @@ bool PlrHitMonst(int pnum, int m)
if (hit < hper) {
#endif
if (player._pIFlags & ISPL_FIREDAM && player._pIFlags & ISPL_LIGHTDAM) {
int midam = player._pIFMinDam + GenerateRnd(player._pIFMaxDam - player._pIFMinDam);
int midam = player._pIFMinDam + GenerateRndV2(player._pIFMaxDam - player._pIFMinDam);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Highest value this can generate is player._pIFMaxDam - 1, is this a pre-existing bug?

@@ -2733,7 +2734,7 @@ bool PM_DoRangeAttack(int pnum)
mistype = MIS_LARROW;
}
if ((player._pIFlags & ISPL_FIRE_ARROWS) != 0 && (player._pIFlags & ISPL_LIGHT_ARROWS) != 0) {
dmg = player._pIFMinDam + GenerateRnd(player._pIFMaxDam - player._pIFMinDam);
dmg = player._pIFMinDam + GenerateRndV2(player._pIFMaxDam - player._pIFMinDam);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same potential (existing) bug here, damage range is [min, max)

@AJenbo
Copy link
Member

AJenbo commented Jun 19, 2021

Is this algo guaranteed to give the same sequence when setting the same seed?

@ephphatha
Copy link
Contributor

std::mt19937 is a default set of arguments for the mersenne twister engine, rng is guaranteed to give the same sequence when (re-)initialised with the same seed. I believe that distributions are guaranteed to map the same generated number to the same output within their interval but I can't find documentation covering this specifically.

Running a quick test shows that calling the distribution with different params doesn't affect the generator, and a uniform_int_distribution appears to be consistent for the same generator values.

In the following output all three lines were generated with the default mt19937 seed and a distribution having a default interval of [0, 100].
The top line used the interval [333, 999] for positions 0, 3, 6, 9, 12, 15, 18
The middle line used the same interval as the top line for positions 0, 6, 12, 18 and the same interval as the bottom line for positions 3, 9, 15
The bottom line used the interval [1, 4] for positions 0, 3, 6, 9, 12, 15, 18

547, 20, 50,647, 63, 43,616, 39, 83,950, 86, 22,635, 91, 15,589, 89, 56,650, 32
  1, 20, 50,647, 63, 43,  2, 39, 83,950, 86, 22,  1, 91, 15,589, 89, 56,  2, 32
  1, 20, 50,  2, 63, 43,  2, 39, 83,  4, 86, 22,  1, 91, 15,  1, 89, 56,  2, 32

The same numbers were generated in the same order, and mapped to the same output for a given interval even when the distribution params varied between runs (cols 3-6 for example).

@AJenbo
Copy link
Member

AJenbo commented Jun 19, 2021

My biggest worry is that there could be a difference between two std implementations, but sounds like it's part of the spec to be stable.

@qndel
Copy link
Member Author

qndel commented Jun 19, 2021

Is this algo guaranteed to give the same sequence when setting the same seed?

obviously

@qndel
Copy link
Member Author

qndel commented Jun 19, 2021

vanilla rng

monobit_test                             0.1016892493688818 PASS
frequency_within_block_test              0.5427869036748679 PASS
runs_test                                0.20869832039571723 PASS
longest_run_ones_in_a_block_test         0.10379179874654554 PASS
binary_matrix_rank_test                  0.10733785915603897 PASS
dft_test                                 0.16373315942288413 PASS
non_overlapping_template_matching_test   0.9999981915288942 PASS
overlapping_template_matching_test       0.2136566153526672 PASS
maurers_universal_test                   0.004284665889349451 FAIL
linear_complexity_test                   0.7170447464996982 PASS
serial_test                              0.28304219315039814 PASS
approximate_entropy_test                 0.3002676702843548 PASS
cumulative_sums_test                     0.08849918274824375 PASS
random_excursion_test                    0.029870664339639433 PASS
random_excursion_variant_test            0.08634782098366245 PASS

@qndel
Copy link
Member Author

qndel commented Jun 19, 2021

my rng

monobit_test                             0.2581142844998201 PASS
frequency_within_block_test              0.6608354754963154 PASS
runs_test                                0.8426423556097362 PASS
longest_run_ones_in_a_block_test         0.4422302667466412 PASS
binary_matrix_rank_test                  0.5712732704344778 PASS
dft_test                                 0.5626456741682362 PASS
non_overlapping_template_matching_test   1.0000000299879452 PASS
overlapping_template_matching_test       0.26038389890445107 PASS
maurers_universal_test                   0.13866138941713624 PASS
linear_complexity_test                   0.2408765957644154 PASS
serial_test                              0.1545838268001897 PASS
approximate_entropy_test                 0.38857399727239755 PASS
cumulative_sums_test                     0.17274721942550308 PASS
random_excursion_test                    0.291628919843869  PASS
random_excursion_variant_test            0.13470504984007847 PASS

@qndel
Copy link
Member Author

qndel commented Jun 19, 2021

tested with https://github.com/dj-on-github/sp800_22_tests

These numbers don't mean much but there were many rng flaws that could be observed during actual gameplay - stat shrine boosted dex 90% of the time etc. that's why one of the shrines even has "favors shields" in description. That's just poo rng.
None of these problems exist when using the new one.

ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 28, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 28, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 28, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 29, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 29, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 29, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jun 30, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jul 1, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align with member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.

Add libgmock-dev to the install script for linux_x86_64_test to use EXPECT_THAT and ::testing::AnyOf
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jul 8, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align with member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.

Add libgmock-dev to the install script for linux_x86_64_test to use EXPECT_THAT and ::testing::AnyOf
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jul 16, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align with member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.

Add libgmock-dev to the install script for linux_x86_64_test to use EXPECT_THAT and ::testing::AnyOf
ephphatha added a commit to ephphatha/devilutionX that referenced this pull request Jul 16, 2021
…ators

This builds on the work done by qndel in diasurgical#2084. I have not updated any of the call sites identified in that PR as I wanted to make it obvious that this does not change existing code.

This is more of a discussion point at the moment. I was thinking that namespaces would be a good way to delineate behaviour. I've implemented "vanilla" to cover the base game logic and bugs. "randomV1" is based on diasurgical#2084 and uses STL concepts to add convenience functions as discussed in diasurgical#2193.

The API design is intended to be common between versions. The next step would be to update the vanilla namespace functions to align with member functions of vanilla::RandomEngine. The reason for having this class is to allow creating a RandomEngine object and passing it in to the level/item generation code (for example) to make it clearer how a given object/dungeon layout is generated.

Looking at code such as quests.cpp:165 makes me think a "vanillaSafe" namespace which fixes the negative number bug would be useful, but this would be a future enhancement iff this proposal is acceptable.

Add libgmock-dev to the install script for linux_x86_64_test to use EXPECT_THAT and ::testing::AnyOf
@AJenbo AJenbo modified the milestones: 1.3.0, 1.4.0 Sep 23, 2021
@AJenbo AJenbo modified the milestones: 1.4.0, 1.5.0 Dec 17, 2021
@AJenbo AJenbo modified the milestones: 1.5.0, 1.6.0 Apr 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants