Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

default_rng.integers(2**32) always return 0 #16066

Closed
vstinner opened this issue Apr 24, 2020 · 7 comments · Fixed by #16076
Closed

default_rng.integers(2**32) always return 0 #16066

vstinner opened this issue Apr 24, 2020 · 7 comments · Fixed by #16076
Labels
00 - Bug 06 - Regression component: numpy.random Priority: high High priority, also add milestones for urgent issues
Milestone
1.18.4 release

Comments

@vstinner
Copy link
Contributor

Reproducing code example:

I don't understand why default_rng.integers(2**32) always return 0, whereas 2**32-1, 2**32+1, 2**40, etc. return random numbers as expected. The result is a numpy.int64, so I expected to be able to generate numbers of up 2**64-1.

$ python3.7 -c 'from numpy.random import default_rng; rng=default_rng(); print([rng.integers(2**32) for _ in range(5)])'
[0, 0, 0, 0, 0]

It's fine with 2**40:

$ python3.7 -c 'from numpy.random import default_rng; rng=default_rng(); print([rng.integers(2**40) for _ in range(5)])'
[386296210341, 896689857600, 958588149890, 364800985883, 643738305251]

Numpy/Python version information:

I'm running Fedora 31 with python3-3.7.6-2.fc31.x86_64 and python3-numpy-1.17.4-2.fc31.x86_64:

vstinner@apu$ python3.7
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, numpy; print(numpy.__version__, sys.version)
1.17.4 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]
@seberg seberg added 00 - Bug Priority: high High priority, also add milestones for urgent issues component: numpy.random labels Apr 24, 2020
@seberg
Copy link
Member

seberg commented Apr 24, 2020

Probably it uses a 32bit generator in the background for higher speeds, but then casts 2**32 + 1 to 32bit by accident! Its definitely a pretty serious bug.

@seberg seberg added this to the 1.19.0 release milestone Apr 24, 2020
@WarrenWeckesser
Copy link
Member

The problem is in master, too:

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.19.0.dev0+6d6df47'

In [3]: rng = np.random.default_rng()

In [4]: rng.integers(2**32, size=8)
Out[4]: array([0, 0, 0, 0, 0, 0, 0, 0])

This was probably introduced in #14777. I'll take a look.

@seberg
Copy link
Member

seberg commented Apr 24, 2020

Thanks @WarrenWeckesser, the problem is also in 1.17, so it may be older than your commit. But I do think by now it goes this deep into the code and you seem to know that part. So great, was just considering to ping Kevin :).

@seberg seberg modified the milestones: 1.19.0 release, 1.18.4 release Apr 24, 2020
@seberg
Copy link
Member

seberg commented Apr 24, 2020

If we manage, we should try to include the fix in 1.18.4 I think.

@WarrenWeckesser
Copy link
Member

WarrenWeckesser commented Apr 24, 2020 •
edited

@seberg, gh-14777 was backported to 1.17.

@WarrenWeckesser
Copy link
Member

It turns out the cause is #14501, which was backported to 1.17.3. I'm working on a fix.

@WarrenWeckesser WarrenWeckesser mentioned this issue Apr 25, 2020
Merged
@WarrenWeckesser
Copy link
Member

Proposed fix is in #16076

WarrenWeckesser added a commit to WarrenWeckesser/numpy that referenced this issue Apr 26, 2020
@WarrenWeckesser
BUG: random: Generator.integers(2**32) always returned 0.
e8b8401 
When the input to Generator.integers was 2**32, the value 2**32-1
was being passed as the `rng` argument to the 32-bit Lemire method,
but that method requires `rng` be strictly less then 2**32-1.

The fix was to handle 2**32-1 by calling next_uint32 directly.
This also works for the legacy code without changing the stream
of random integers from `randint`.

Closes numpygh-16066.
charris pushed a commit to charris/numpy that referenced this issue Apr 27, 2020
@WarrenWeckesser @charris
BUG: random: Generator.integers(2**32) always returned 0.
23d7021 
When the input to Generator.integers was 2**32, the value 2**32-1
was being passed as the `rng` argument to the 32-bit Lemire method,
but that method requires `rng` be strictly less then 2**32-1.

The fix was to handle 2**32-1 by calling next_uint32 directly.
This also works for the legacy code without changing the stream
of random integers from `randint`.

Closes numpygh-16066.
@charris charris mentioned this issue Apr 27, 2020
Merged
@iyanmv iyanmv mentioned this issue Nov 18, 2021
Merged
iyanmv added a commit to iyanmv/galois that referenced this issue Nov 18, 2021
@iyanmv
Bump NumPy to 1.18.4
e916418 
NumPy 1.18.4 is the first release with the new random number generator
policy (NEP 19) that is not affected by bugs gh-16066 or gh-14774.

Refs:
 - https://numpy.org/devdocs/release/1.18.4-notes.html
 - numpy/numpy#14774
 - numpy/numpy#16066
iyanmv added a commit to iyanmv/galois that referenced this issue Nov 22, 2021
@iyanmv
Bump NumPy to 1.18.4
8eaab38 
NumPy 1.18.4 is the first release with the new random number generator
policy (NEP 19) that is not affected by bugs gh-16066 or gh-14774.

Refs:
 - https://numpy.org/devdocs/release/1.18.4-notes.html
 - numpy/numpy#14774
 - numpy/numpy#16066
mhostetter pushed a commit to mhostetter/galois that referenced this issue Dec 3, 2021
@iyanmv @mhostetter
Bump NumPy to 1.18.4
7ecf8ec 
NumPy 1.18.4 is the first release with the new random number generator
policy (NEP 19) that is not affected by bugs gh-16066 or gh-14774.

Refs:
 - https://numpy.org/devdocs/release/1.18.4-notes.html
 - numpy/numpy#14774
 - numpy/numpy#16066
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug 06 - Regression component: numpy.random Priority: high High priority, also add milestones for urgent issues
Projects
None yet
Milestone
1.18.4 release
Development

Successfully merging a pull request may close this issue.

BUG: random: Generator.integers(2**32) always returned 0. WarrenWeckesser/numpy
3 participants
@seberg @vstinner @WarrenWeckesser